• Thomas Munro's avatar
    Rotate instead of shifting hash join batch number. · e69d6445
    Thomas Munro authored
    Our algorithm for choosing batch numbers turned out not to work
    effectively for multi-billion key inner relations.  We would use
    more hash bits than we have, and effectively concentrate all tuples
    into a smaller number of batches than we intended.  While ideally
    we should switch to wider hashes, for now, change the algorithm to
    one that effectively gives up bits from the bucket number when we
    don't have enough bits.  That means we'll finish up with longer
    bucket chains than would be ideal, but that's better than having
    batches that don't fit in work_mem and can't be divided.
    
    Batch-patch to all supported releases.
    
    Author: Thomas Munro
    Reviewed-by: Tom Lane, thanks also to Tomas Vondra, Alvaro Herrera, Andres Freund for testing and discussion
    Reported-by: James Coleman
    Discussion: https://postgr.es/m/16104-dc11ed911f1ab9df%40postgresql.org
    e69d6445
pg_bitutils.h 3.16 KB