The WAL consistency checking code needed to be updated for the new
page status bit, but that didn't get done previously.

Ashutosh Sharma, reviewed by Amit Kapila

Discussion: http://postgr.es/m/CAA4eK1LP_oz4EfMen14OjJuzN5CqPdfRkFFuA-MfkcfeE8zGyg@mail.gmail.com

Clear LH_PAGE_HAS_DEAD_TUPLES during replay, similar to what gets done
for btree.  Update hashdesc.c for xl_hash_vacuum_one_page.

Oversights in commit 6977b8b7 spotted
by Amit Kapila.  Patch by Ashutosh Sharma.

Bump WAL version.  The original patch to make hash indexes write-ahead
logged probably should have done this, and the single page vacuuming
patch probably should have done it again, but better late than never.

Discussion: http://postgr.es/m/CAA4eK1Kd=mJ9xreovcsh0qMiAj-QqCphHVQ_Lfau1DR9oVjASQ@mail.gmail.com

We'll need this for the upcoming patch to speed up expression evaluation.
Might as well push it now to see if it behaves sanely in the buildfarm.

Andres Freund

Discussion: https://postgr.es/m/20170320062511.hp5qeurtxrwsvfxr@alap3.anarazel.de

Michael Paquier

Discussion: https://www.postgresql.org/message-id/CAB7nPqRZ_LNTCJ7gGjE_SpRanGoALfTgsxdauNsKq%2BLBRH9fxg%40mail.gmail.com

Commit 818fd4a6 missed cleaning up the symlinks it added for various .c
files imported from src/port and src/common.  Neatnik-ishly make the
file lists in the "clean" target look exactly like the earlier lists of
what to symlink in.

TidScan plan nodes were not systematically tested before.  These additions
raise the LOC coverage number for the basic regression tests from 52% to
92% in nodeTidscan.c, and from 60% to 93% in tidpath.c.

Andres Freund, tweaked a bit by me

Discussion: https://postgr.es/m/20170320062511.hp5qeurtxrwsvfxr@alap3.anarazel.de

this makes buffer access strategy have no effect.
Change was a part of commit 48354581 during 9.6
release cycle, so backpath to 9.6

Reported-by: Jim Nasby
Author: Alexander Korotkov
Reviewed-by: Jim Nasby, Andres Freund

https://commitfest.postgresql.org/13/1029/

Not every platform supports non-default collations, as pointed out by
the buildfarm, so skip collation-related regression tests in pg_dump
when they aren't supported.

Commit ff27db5d changed this function to no longer return the start
position, but forgot the comment.

Oversight in commit 17f8ffa1.  Per buildfarm member prion.

The non-concurrent code path for REFRESH MATERIALIZED VIEW failed to
report its updates to the stats collector.  This is bad since it means
auto-analyze doesn't know there's any work to be done.  Adjust it to
report the refresh as a table truncate followed by insertion of an
appropriate number of rows.

Since a matview could contain more than INT_MAX rows, change the
signature of pgstat_count_heap_insert() to accept an int64 rowcount.
(The accumulator it's adding into is already int64, but existing
callers could not insert more than a small number of rows at once,
so the argument had been declared just "int n".)

This is surely a bug fix, but changing pgstat_count_heap_insert()'s API
seems too risky for the back branches.  Given the lack of previous
complaints, I'm not sure it's a big enough problem to justify a kluge
solution that would avoid that.  So, no back-patch, at least for now.

Jim Mlodgenski, adjusted a bit by me

Discussion: https://postgr.es/m/CAB_5SRchSz7-WmdO5szdiknG8Oj_GGqJytrk1KRd11yhcMs1KQ@mail.gmail.com

Link to CREATE and ALTER ROLE consistently, instead of mixing CREATE
USER and ALTER ROLE.

Update outdated claim that TRUNCATE is a PostgreSQL extension.

Add cross-links between DELETE and TRUNCATE references pages.

Change the header style of references pages in HTML and PDF output to be
more like the old style under DSSSL.  In particular, the page should
start with a header containing the command name, instead of just "Name".

When removing a test, need to make sure the count of tests is adjusted
when it isn't calculated.

This is really testing getopt more than pg_dump, and what getopt returns
exactly appears to differ based on platform, so remove this test.

Per buildfarm.

These improvements bring the lines-of-code coverage of pg_dump.c up to
87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly).  Nearly
every function is covered, three of the four which aren't are only
called when talking to older PG instances.

There is more which can, and should, be done here to improve the
coverage but it's past time to see what the buildfarm thinks of this.

What has been added:

- Coverage for many more command-line options
- Use command_fails_like instead of command_exit_is
- Operator classes, operator families
- Text search configuration, templates, parsers, dictionaries
- FDWs, servers, foreign tables
- Materialized views
- Improved Publications / Subscriptions test (though this needs work,
  see PG10 open items and tests marked with XXX in 002_pg_dump.pl)
- Unlogged tables
- Partitioned tables
- Additional ACL testing for various object types

There is room for improvement, specifically:

- Various type-based option (alignment, storage, etc)
- Composite type collation
- Extra Procedural language functions (inline, validator)
- Different function options (SRF, Transform, config, security definer,
  cost, leakproof)
- OpClass options (default, storage, order by, recheck)
- OpFamily options (order by, recheck)
- Aggregate functions (combinefunc, serialfunc, deserialfunc, etc)
- Text Search parser 'headline'
- Text Search template 'init'
- FDW options (handler, validator, options)
- Server options (type, version, options)
- User mapping options
- Default ACLs for sequences, types
- Security labels
- View circular dependencies (last function that needs coverage)
- Toast table autovacuum options
- Replica identity options
- Independent indexes (plus marking them as clustered on)
- Deferrable / initially deferred constraints
- Independent domain constraints

There's bits of extension pg_dump'ing also not covered, but those will
need to go into test_pg_dump (such as having a filter for config
tables).

Last, but not least, this approximately halves the number of tests run
with 'ok()' by removing the ok()-based checking of if all runs are
covered by each test.  Instead, 002_pg_dump.pl will just exit out in
such a case (with a message in the log file).  In general, when adding
tests, cover all runs unless there is a very good reason not to (such as
adding a 'catch-all' case).  With these changes, the resulting output
and number of "tests" run is actually reduced.

The incorrect count causes FOP to error out.

Previous commits, notably 53be0b1a and
6f3bd98e, made it possible to see from
pg_stat_activity when a backend was stuck waiting for another backend,
but it's also fairly common for a backend to be stuck waiting for an
I/O.  Add wait events for those operations, too.

Rushabh Lathia, with further hacking by me.  Reviewed and tested by
Michael Paquier, Amit Kapila, Rajkumar Raghuwanshi, and Rahila Syed.

Discussion: http://postgr.es/m/CAGPqQf0LsYHXREPAZqYGVkDqHSyjf=KsD=k0GTVPAuzyThh-VQ@mail.gmail.com

Windows apparently will not detect socket write-ready events unless a
preceding send attempt returned WSAEWOULDBLOCK.  In many usage patterns
that's satisfied by the caller of WaitEvenSetWait(), but not always.

Apply the same solution that we already had in pgwin32_select(), namely to
perform a dummy WSASend() call with len=0.  This will return WSAEWOULDBLOCK
if there's no buffer space (even though it could legitimately do nothing
and report success, which makes me a bit nervous about this solution;
but since it's been working fine in libpq, let's roll with it).

In passing, improve the comments about this in pgwin32_select(), and remove
duplicated code there.

Back-patch to 9.6 where WaitEventSetWait() was introduced.  We might need
to back-patch something similar into predecessor code.  But given the lack
of complaints so far, it's not clear that the case ever gets exercised
in the back branches, so I'm not going to expend effort on it right now.

This should resolve recurring failures on buildfarm member bowerbird,
which has been failing since 1e8a8500 went in.

Diagnosis and patch by Petr Jelinek, cosmetic adjustments by me.

Discussion: https://postgr.es/m/5b6a6d6d-fb45-0afb-2e95-5600063c3dbd@2ndquadrant.com

Author: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>

Concurrent auto-analyze could be holding a snapshot, affecting the
removal of deleted row versions.  Remove the deletion to avoid this
happening.  Per buildfarm.

In passing, make the test independent of assumptions of physical row
order, just out of sheer paranoia.

Names containing the letters "log" sometimes confuse users into
believing that only non-critical data is present.  It is hoped
this renaming will discourage ill-considered removals of transaction
status data.

Michael Paquier

Discussion: http://postgr.es/m/CA+Tgmoa9xFQyjRZupbdEFuwUerFTvC6HjZq1ud6GYragGDFFgA@mail.gmail.com

David Christensen

Discussion: http://postgr.es/m/82299377-1480-4439-9ABA-5828D71AA22E@endpoint.com

Tests all combinations of users with MD5, plaintext and SCRAM verifiers
stored in pg_authid, with plain 'password', 'md5' and 'scram'
authentication methods.

Michael Paquier

Oversight in the main SCRAM patch.

If the process token contains SECURITY_SERVICE_RID, but it has been
disabled by the SE_GROUP_USE_FOR_DENY_ONLY attribute, win32_is_service()
would incorrectly report that we're running as a service. That situation
arises, e.g. if postmaster is launched with a restricted security token,
with the "Log in as Service" privilege explicitly removed.

Replace the broken code with CheckProcessTokenMembership(), which does
this correctly. Also replace similar code in win32_is_admin(), even
though it got this right, for simplicity and consistency.

Per bug #13755, reported by Breen Hagan. Back-patch to all supported
versions. Patch by Takayuki Tsunakawa, reviewed by Michael Paquier.

Discussion: https://www.postgresql.org/message-id/20151104062315.2745.67143%40wrigleys.postgresql.org

This would have ben grammatical if the closing tag name were actually
part of the output, but of course it's not.

While \help CREATE would complete usefully, \help DROP or \help ALTER
did not complete anything.

Expand the list of things after CREATE and DROP to cover ALTER as well,
and use that for the ALTER completion.  Also make minor tweaks to that
list.

Also add support for completing \help on multiword commands like CREATE
TEXT SEARCH ...

Author: Andreas Karlsson <andreas@proxel.se>

presence of page pins, which leads to serious estimation errors in the
planner.  This particularly affects small heavily-accessed tables,
especially where locking (e.g. from FK constraints) forces frequent
vacuums for mxid cleanup.

Fix by keeping separate track of pages whose live tuples were actually
counted vs. pages that were only scanned for freezing purposes.  Thus,
reltuples can only be set to 0 if all pages of the relation were
actually counted.

Backpatch to all supported versions.

Per bug #14057 from Nicolas Baccelli, analyzed by me.

Discussion: https://postgr.es/m/20160331103739.8956.94469@wrigleys.postgresql.org

Primarily, this didn't work correctly when the tidbitmap ended up
empty.

Dilip Kumar, per a report from Emre Hasegeli

Discussion: http://postgr.es/m/CAFiTN-ujHFKb8WSLhK54rfqQT3r2yiPQOyeBrCDsA4p9Fwp_jw@mail.gmail.com

These functions are intended to be used by monitoring tools, and,
unlike pg_ls_dir(), access to them can be granted to non-superusers,
so that those monitoring tools can observe the principle of least
privilege.

Dave Page, revised by me, and also reviewed a bit by Thomas Munro.

Discussion: http://postgr.es/m/CA+OCxow-X=D2fWdKy+HP+vQ1LtrgbsYQ=CshzZBqyFT5jOYrFw@mail.gmail.com

The previous deparsing logic wasn't smart enough to produce subqueries
when deparsing; make it smart enough to do that.  However, we only do
it that way when necessary, because it generates more complicated SQL
which will be harder for any humans reading the queries to understand.

Etsuro Fujita, reviewed by Ashutosh Bapat

Discussion: http://postgr.es/m/c449261a-b033-dc02-9254-2fe5b7044795@lab.ntt.co.jp

The original coding was trying to use a TypeName as a string Value,
which doesn't work; an oversight in my commit a61fd533.  Repair.

Also, make sure we cover the broken case in the relevant test script.

Backpatch to 9.5.

Discussion: https://postgr.es/m/20170315151829.bhxsvrp75xdxhm3n@alvherre.pgsql

The buildfarm has reminded me that not all systems consider char to be
signed and we need to be explicit.  Adjust the various bits of mac8.c
for what we intend, mostly using casts to unsigned char as suggested by
Tom, and adjust the tests for valid input accordingly.  Explicitly make
the hexlookup table signed as it's useful to use -1 there to indicate an
invalid value.

Andres' compiler points out, quite correctly, that there's no need for
some of the overly paranoid checks which were put into mac8.c.  Remove
those, as they're useless, add some comments and make a few other minor
improvements- reduce the size of hexlookup by making it a char array
instead of an int array, and pass in the ptr location directly instead
of making hex2_to_uchar re-calculate the location based off the offset
every time.