• Tom Lane's avatar
    Fix strange behavior (and possible crashes) in full text phrase search. · 89fcea1a
    Tom Lane authored
    In an attempt to simplify the tsquery matching engine, the original
    phrase search patch invented rewrite rules that would rearrange a
    tsquery so that no AND/OR/NOT operator appeared below a PHRASE operator.
    But this approach had numerous problems.  The rearrangement step was
    missed by ts_rewrite (and perhaps other places), allowing tsqueries
    to be created that would cause Assert failures or perhaps crashes at
    execution, as reported by Andreas Seltenreich.  The rewrite rules
    effectively defined semantics for operators underneath PHRASE that were
    buggy, or at least unintuitive.  And because rewriting was done in
    tsqueryin() rather than at execution, the rearrangement was user-visible,
    which is not very desirable --- for example, it might cause unexpected
    matches or failures to match in ts_rewrite.
    
    As a somewhat independent problem, the behavior of nested PHRASE operators
    was only sane for left-deep trees; queries like "x <-> (y <-> z)" did not
    behave intuitively at all.
    
    To fix, get rid of the rewrite logic altogether, and instead teach the
    tsquery execution engine to manage AND/OR/NOT below a PHRASE operator
    by explicitly computing the match location(s) and match widths for these
    operators.
    
    This requires introducing some additional fields into the publicly visible
    ExecPhraseData struct; but since there's no way for third-party code to
    pass such a struct to TS_phrase_execute, it shouldn't create an ABI problem
    as long as we don't move the offsets of the existing fields.
    
    Another related problem was that index searches supposed that "!x <-> y"
    could be lossily approximated as "!x & y", which isn't correct because
    the latter will reject, say, "x q y" which the query itself accepts.
    This required some tweaking in TS_execute_ternary along with the main
    tsquery engine.
    
    Back-patch to 9.6 where phrase operators were introduced.  While this
    could be argued to change behavior more than we'd like in a stable branch,
    we have to do something about the crash hazards and index-vs-seqscan
    inconsistency, and it doesn't seem desirable to let the unintuitive
    behaviors induced by the rewriting implementation stand as precedent.
    
    Discussion: https://postgr.es/m/28215.1481999808@sss.pgh.pa.us
    Discussion: https://postgr.es/m/26706.1482087250@sss.pgh.pa.us
    89fcea1a
tsgistidx.c 18.2 KB