• Tom Lane's avatar
    Be more wary about 32-bit integer overflow in pg_stat_statements. · 17fd203b
    Tom Lane authored
    We've heard a couple of reports of people having trouble with
    multi-gigabyte-sized query-texts files.  It occurred to me that on
    32-bit platforms, there could be an issue with integer overflow
    of calculations associated with the total query text size.
    Address that with several changes:
    
    1. Limit pg_stat_statements.max to INT_MAX / 2 not INT_MAX.
    The hashtable code will bound it to that anyway unless "long"
    is 64 bits.  We still need overflow guards on its use, but
    this helps.
    
    2. Add a check to prevent extending the query-texts file to
    more than MaxAllocHugeSize.  If it got that big, qtext_load_file
    would certainly fail, so there's not much point in allowing it.
    Without this, we'd need to consider whether extent, query_offset,
    and related variables shouldn't be off_t not size_t.
    
    3. Adjust the comparisons in need_gc_qtexts() to be done in 64-bit
    arithmetic on all platforms.  It appears possible that under duress
    those multiplications could overflow 32 bits, yielding a false
    conclusion that we need to garbage-collect the texts file, which
    could lead to repeatedly garbage-collecting after every hash table
    insertion.
    
    Per report from Bruno da Silva.  I'm not convinced that these
    issues fully explain his problem; there may be some other bug that's
    contributing to the query-texts file becoming so large in the first
    place.  But it did get that big, so #2 is a reasonable defense,
    and #3 could explain the reported performance difficulties.
    
    (See also commit 8bbe4cbd, which addressed some related bugs.
    The second Discussion: link is the thread that led up to that.)
    
    This issue is old, and is primarily a problem for old platforms,
    so back-patch.
    
    Discussion: https://postgr.es/m/CAB+Nuk93fL1Q9eLOCotvLP07g7RAv4vbdrkm0cVQohDVMpAb9A@mail.gmail.com
    Discussion: https://postgr.es/m/5601D354.5000703@BlueTreble.com
    17fd203b
pg_stat_statements.c 81.3 KB