• Tom Lane's avatar
    Fix <> and pattern-NOT-match estimators to handle nulls correctly. · 512c7356
    Tom Lane authored
    These estimators returned 1 minus the corresponding equality/match
    estimate, which is incorrect: we need to subtract off the fraction
    of nulls in the column, since those are neither equal nor not equal
    to the comparison value.  The error only becomes obvious if the
    nullfrac is large, but it could be very bad in a mostly-nulls
    column, as reported in bug #14676 from Marko Tiikkaja.
    
    To fix the <> case, refactor eqsel() and neqsel() to call a common
    support routine, which can be made to account for nullfrac correctly.
    The pattern-match cases were already factored that way, and it was
    simply an oversight that patternsel() wasn't subtracting off nullfrac.
    
    neqjoinsel() has a similar problem, but since we're elsewhere discussing
    changing its behavior entirely, I left it alone for now.
    
    This is a very longstanding bug, but I'm hesitant to back-patch a fix for
    it.  Given the lack of prior complaints, such cases must not come up often,
    so it's probably not worth the risk of destabilizing plans in stable
    branches.
    
    Discussion: https://postgr.es/m/20170529153847.4275.95416@wrigleys.postgresql.org
    512c7356
selfuncs.c 232 KB