• Tom Lane's avatar
    Use SnapshotDirty rather than an active snapshot to probe index endpoints. · fccebe42
    Tom Lane authored
    If there are lots of uncommitted tuples at the end of the index range,
    get_actual_variable_range() ends up fetching each one and doing an MVCC
    visibility check on it, until it finally hits a visible tuple.  This is
    bad enough in isolation, considering that we don't need an exact answer
    only an approximate one.  But because the tuples are not yet committed,
    each visibility check does a TransactionIdIsInProgress() test, which
    involves scanning the ProcArray.  When multiple sessions do this
    concurrently, the ensuing contention results in horrid performance loss.
    20X overall throughput loss on not-too-complicated queries is easy to
    demonstrate in the back branches (though someone's made it noticeably
    less bad in HEAD).
    
    We can dodge the problem fairly effectively by using SnapshotDirty rather
    than a normal MVCC snapshot.  This will cause the index probe to take
    uncommitted tuples as good, so that we incur only one tuple fetch and test
    even if there are many such tuples.  The extent to which this degrades the
    estimate is debatable: it's possible the result is actually a more accurate
    prediction than before, if the endmost tuple has become committed by the
    time we actually execute the query being planned.  In any case, it's not
    very likely that it makes the estimate a lot worse.
    
    SnapshotDirty will still reject tuples that are known committed dead, so
    we won't give bogus answers if an invalid outlier has been deleted but not
    yet vacuumed from the index.  (Because btrees know how to mark such tuples
    dead in the index, we shouldn't have a big performance problem in the case
    that there are many of them at the end of the range.)  This consideration
    motivates not using SnapshotAny, which was also considered as a fix.
    
    Note: the back branches were using SnapshotNow instead of an MVCC snapshot,
    but the problem and solution are the same.
    
    Per performance complaints from Bartlomiej Romanski, Josh Berkus, and
    others.  Back-patch to 9.0, where the issue was introduced (by commit
    40608e7f).
    fccebe42
selfuncs.c 213 KB