• Tom Lane's avatar
    Repair bugs in GiST page splitting code for multi-column indexes. · 166d534f
    Tom Lane authored
    When considering a non-last column in a multi-column GiST index,
    gistsplit.c tries to improve on the split chosen by the opclass-specific
    pickSplit function by considering penalties for the next column.  However,
    there were two bugs in this code: it failed to recompute the union keys for
    the leftmost index columns, even though these might well change after
    reassigning tuples; and it included the old union keys in the recomputation
    for the columns it did recompute, so that those keys couldn't get smaller
    even if they should.  The first problem could result in an invalid index
    in which searches wouldn't find index entries that are in fact present;
    the second would make the index less efficient to search.
    
    Both of these errors were caused by misuse of gistMakeUnionItVec, whose
    API was designed in a way that just begged such errors to be made.  There
    is no situation in which it's safe or useful to compute the union keys for
    a subset of the index columns, and there is no caller that wants any
    previous union keys to be included in the computation; so the undocumented
    choice to treat the union keys as in/out rather than pure output parameters
    is a waste of code as well as being dangerous.
    
    Hence, rather than just making a minimal patch, I've changed the API of
    gistMakeUnionItVec to remove the "startkey" parameter (it now always
    processes all index columns) and treat the attr/isnull arrays as purely
    output parameters.
    
    In passing, also get rid of a couple of unnecessary and dangerous uses
    of static variables in gistutil.c.  It's remarkable that the one in
    gistMakeUnionKey hasn't given us portability troubles before now, because
    in addition to posing a re-entrancy hazard, it was unsafely assuming that
    a static char[] array would have at least Datum alignment.
    
    Per investigation of a trouble report from Tomas Vondra.  (There are also
    some bugs in contrib/btree_gist to be fixed, but that seems like material
    for a separate patch.)  Back-patch to all supported branches.
    166d534f
gistutil.c 20 KB