• David Rowley's avatar
    Allow run-time pruning on nested Append/MergeAppend nodes · a929e17e
    David Rowley authored
    Previously we only tagged on the required information to allow the
    executor to perform run-time partition pruning for Append/MergeAppend
    nodes belonging to base relations.  It was thought that nested
    Append/MergeAppend nodes were just about always pulled up into the
    top-level Append/MergeAppend and that making the run-time pruning info for
    any sub Append/MergeAppend nodes was a waste of time.  However, that was
    likely badly thought through.
    
    Some examples of cases we're unable to pullup nested Append/MergeAppends
    are: 1) Parallel Append nodes with a mix of parallel and non-parallel
    paths into a Parallel Append.  2) When planning an ordered Append scan a
    sub-partition which is unordered may require a nested MergeAppend path to
    ensure sub-partitions don't mix up the order of tuples being fed into the
    top-level Append.
    
    Unfortunately, it was not just as simple as removing the lines in
    createplan.c which were purposefully not building the run-time pruning
    info for anything but RELOPT_BASEREL relations.  The code in
    add_paths_to_append_rel() was far too sloppy about which partitioned_rels
    it included for the Append/MergeAppend paths.  The original code there
    would always assume accumulate_append_subpath() would pull each sub-Append
    and sub-MergeAppend path into the top-level path.  While it does not
    appear that there were any actual bugs caused by having the additional
    partitioned table RT indexes recorded, what it did mean is that later in
    planning, when we built the run-time pruning info that we wasted effort
    and built PartitionedRelPruneInfos for partitioned tables that we had no
    subpaths for the executor to run-time prune.
    
    Here we tighten that up so that partitioned_rels only ever contains the RT
    index for partitioned tables which actually have subpaths in the given
    Append/MergeAppend.  We can now Assert that every PartitionedRelPruneInfo
    has a non-empty present_parts.  That should allow us to catch any weird
    corner cases that have been missed.
    
    In passing, it seems there is no longer a good reason to have the
    AppendPath and MergeAppendPath's partitioned_rel fields a List of IntList.
    We can simply have a List of Relids instead.  This is more compact in
    memory and faster to add new members to.  We still know which is the root
    level partition as these always have a lower relid than their children.
    Previously this field was used for more things, but run-time partition
    pruning now remains the only user of it and it has no need for a List of
    IntLists.
    
    Here we also get rid of the RelOptInfo partitioned_child_rels field. This
    is what was previously used to (sometimes incorrectly) set the
    Append/MergeAppend path's partitioned_rels field.  That was the only usage
    of that field, so we can happily just remove it.
    
    I also couldn't resist changing some nearby code to make use of the newly
    added for_each_from macro so we can skip the first element in the list
    without checking if the current item was the first one on each
    iteration.
    
    A bug report from Andreas Kretschmer prompted all this work, however,
    after some consideration, I'm not personally classing this as a bug fix.
    So no backpatch.  In Andreas' test case, it just wasn't that clear that
    there was a nested Append since the top-level Append just had a single
    sub-path which was pulled up a level, per 8edd0e79.
    
    Author: David Rowley
    Reviewed-by: Amit Langote
    Discussion: https://postgr.es/m/flat/CAApHDvqSchs%2BubdybcfFaSPB%2B%2BEA7kqMaoqajtP0GtZvzOOR3g%40mail.gmail.com
    a929e17e
pathnodes.h 107 KB