• Jeff Davis's avatar
    Disk-based Hash Aggregation. · 1f39bce0
    Jeff Davis authored
    While performing hash aggregation, track memory usage when adding new
    groups to a hash table. If the memory usage exceeds work_mem, enter
    "spill mode".
    
    In spill mode, new groups are not created in the hash table(s), but
    existing groups continue to be advanced if input tuples match. Tuples
    that would cause a new group to be created are instead spilled to a
    logical tape to be processed later.
    
    The tuples are spilled in a partitioned fashion. When all tuples from
    the outer plan are processed (either by advancing the group or
    spilling the tuple), finalize and emit the groups from the hash
    table. Then, create new batches of work from the spilled partitions,
    and select one of the saved batches and process it (possibly spilling
    recursively).
    
    Author: Jeff Davis
    Reviewed-by: Tomas Vondra, Adam Lee, Justin Pryzby, Taylor Vesely, Melanie Plageman
    Discussion: https://postgr.es/m/507ac540ec7c20136364b5272acbcd4574aa76ef.camel@j-davis.com
    1f39bce0
select_distinct.sql 3.38 KB