• Tom Lane's avatar
    Use annotations to reduce instability of isolation-test results. · 741d7f10
    Tom Lane authored
    We've long contended with isolation test results that aren't entirely
    stable.  Some test scripts insert long delays to try to force stable
    results, which is not terribly desirable; but other erratic failure
    modes remain, causing unrepeatable buildfarm failures.  I've spent a
    fair amount of time trying to solve this by improving the server-side
    support code, without much success: that way is fundamentally unable
    to cope with diffs that stem from chance ordering of arrival of
    messages from different server processes.
    
    We can improve matters on the client side, however, by annotating
    the test scripts themselves to show the desired reporting order
    of events that might occur in different orders.  This patch adds
    three types of annotations to deal with (a) test steps that might or
    might not complete their waits before the isolationtester can see them
    waiting; (b) test steps in different sessions that can legitimately
    complete in either order; and (c) NOTIFY messages that might arrive
    before or after the completion of a step in another session.  We might
    need more annotation types later, but this seems to be enough to deal
    with the instabilities we've seen in the buildfarm.  It also lets us
    get rid of all the long delays that were previously used, cutting more
    than a minute off the runtime of the isolation tests.
    
    Back-patch to all supported branches, because the buildfarm
    instabilities affect all the branches, and because it seems desirable
    to keep isolationtester's capabilities the same across all branches
    to simplify possible future back-patching of tests.
    
    Discussion: https://postgr.es/m/327948.1623725828@sss.pgh.pa.us
    741d7f10
nowait-4.out 545 Bytes