• Tomas Vondra's avatar
    Separate per-batch and per-tuple memory contexts in COPY · 36a1281f
    Tomas Vondra authored
    In batching mode, COPY was using the same (per-tuple) memory context for
    allocations with longer lifetime. This was confusing but harmless, until
    commit 31f38174 added COPY FROM ... WHERE feature, introducing a risk
    of memory leak.
    
    The "per-tuple" memory context was reset only when starting new batch,
    but as the rows may be filtered out by the WHERE clauses, that may not
    happen at all.  The WHERE clause however has to be evaluated for all
    rows, before filtering them out.
    
    This commit separates the per-tuple and per-batch contexts, removing the
    ambiguity.  Expressions (both defaults and WHERE clause) are evaluated
    in the per-tuple context, while tuples are formed in the batch context.
    This allows resetting the contexts at appropriate times.
    
    The main complexity is related to partitioning, in which case we need to
    reset the batch context after forming the tuple (which happens before
    routing to leaf partition).  Instead of switching between two contexts
    as before, we simply copy the last tuple aside, reset the context and
    then copy the tuple back.  The performance impact is negligible, and
    juggling with two contexts is not free either.
    
    Discussion: https://www.postgresql.org/message-id/flat/CALAY4q_DdpWDuB5-Zyi-oTtO2uSk8pmy+dupiRe3AvAc++1imA@mail.gmail.com
    36a1281f
copy.c 144 KB