• Heikki Linnakangas's avatar
    Fix overflow check and comment in GIN posting list encoding. · bde7493d
    Heikki Linnakangas authored
    The comment did not match what the code actually did for integers with
    the 43rd bit set. You get an integer like that, if you have a posting
    list with two adjacent TIDs that are more than 2^31 blocks apart.
    According to the comment, we would store that in 6 bytes, with no
    continuation bit on the 6th byte, but in reality, the code encodes it
    using 7 bytes, with a continuation bit on the 6th byte as normal.
    
    The decoding routine also handled these 7-byte integers correctly, except
    for an overflow check that assumed that one integer needs at most 6 bytes.
    Fix the overflow check, and fix the comment to match what the code
    actually does. Also fix the comment that claimed that there are 17 unused
    bits in the 64-bit representation of an item pointer. In reality, there
    are 64-32-11=21.
    
    Fitting any item pointer into max 6 bytes was an important property when
    this was written, because in the old pre-9.4 format, item pointers were
    stored as plain arrays, with 6 bytes for every item pointer. The maximum
    of 6 bytes per integer in the new format guaranteed that we could convert
    any page from the old format to the new format after upgrade, so that the
    new format was never larger than the old format. But we hardly need to
    worry about that anymore, and running into that problem during upgrade,
    where an item pointer is expanded from 6 to 7 bytes such that the data
    doesn't fit on a page anymore, is implausible in practice anyway.
    
    Backpatch to all supported versions.
    
    This also includes a little test module to test these large distances
    between item pointers, without requiring a 16 TB table. It is not
    backpatched, I'm including it more for the benefit of future development
    of new posting list formats.
    
    Discussion: https://www.postgresql.org/message-id/33bfc20a-5c86-f50c-f5a5-58e9925d05ff%40iki.fi
    Reviewed-by: Masahiko Sawada, Alexander Korotkov
    bde7493d
test_ginpostinglist--1.0.sql 325 Bytes