• Tom Lane's avatar
    Make the overflow guards in ExecChooseHashTableSize be more protective. · 8442317b
    Tom Lane authored
    The original coding ensured nbuckets and nbatch didn't exceed INT_MAX,
    which while not insane on its own terms did nothing to protect subsequent
    code like "palloc(nbatch * sizeof(BufFile *))".  Since enormous join size
    estimates might well be planner error rather than reality, it seems best
    to constrain the initial sizes to be not more than work_mem/sizeof(pointer),
    thus ensuring the allocated arrays don't exceed work_mem.  We will allow
    nbatch to get bigger than that during subsequent ExecHashIncreaseNumBatches
    calls, but we should still guard against integer overflow in those palloc
    requests.  Per bug #5145 from Bernt Marius Johnsen.
    
    Although the given test case only seems to fail back to 8.2, previous
    releases have variants of this issue, so patch all supported branches.
    8442317b
nodeHash.c 36.6 KB