Separate per-batch and per-tuple memory contexts in COPY
In batching mode, COPY was using the same (per-tuple) memory context for allocations with longer lifetime. This was confusing but harmless, until commit 31f38174 added COPY FROM ... WHERE feature, introducing a risk of memory leak. The "per-tuple" memory context was reset only when starting new batch, but as the rows may be filtered out by the WHERE clauses, that may not happen at all. The WHERE clause however has to be evaluated for all rows, before filtering them out. This commit separates the per-tuple and per-batch contexts, removing the ambiguity. Expressions (both defaults and WHERE clause) are evaluated in the per-tuple context, while tuples are formed in the batch context. This allows resetting the contexts at appropriate times. The main complexity is related to partitioning, in which case we need to reset the batch context after forming the tuple (which happens before routing to leaf partition). Instead of switching between two contexts as before, we simply copy the last tuple aside, reset the context and then copy the tuple back. The performance impact is negligible, and juggling with two contexts is not free either. Discussion: https://www.postgresql.org/message-id/flat/CALAY4q_DdpWDuB5-Zyi-oTtO2uSk8pmy+dupiRe3AvAc++1imA@mail.gmail.com
Showing
Please register or sign in to comment