Make every page in a hash index which isn't all-zeroes have a valid
special space, so that tools like pageinspect don't error out.

Also, make pageinspect cope with all-zeroes pages, because
_hash_alloc_buckets can leave behind large numbers of those until
they're consumed by splits.

Ashutosh Sharma and Robert Haas, reviewed by Amit Kapila.
Original trouble report from Jeff Janes.

Discussion: http://postgr.es/m/CAMkU=1y6NjKmqbJ8wLMhr=F74WzcMALYWcVFhEpm7i=mV=XsOg@mail.gmail.com

All error messages use the American English spelling of recognize,
apply to the single one not doing so to be consistent.

Author: Daniel Gustafsson <daniel@yesql.se>

They were not rendered with DSSSL, but now they show up by default using
XSLT.  Just remove them, since they are not useful.

The size of the data is not the same thing as the size of the size of
the data.

Reported off-list by Tushar Ahuja.  Fix by Ashutosh Sharma, reviewed
by Amit Kapila.

Discussion: http://postgr.es/m/CAE9k0PnmPDXfvf8HDObme7q_Ewc4E26ukHXUBPySoOs0ObqqaQ@mail.gmail.com

This improves code coverage and lays a foundation for testing
similar issues in a distributed environment.

Author: Thomas Munro <thomas.munro@enterprisedb.com>
Reviewed-by: Michael Paquier <michael.paquier@gmail.com>

suggested by Tom Lane

Author: Daniel Gustafsson <daniel@yesql.se>

Author: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>

Commit 59702716 added transition table support to PL/pgsql so that
SQL queries in trigger functions could access those transient
tables.  In order to provide the same level of support for PL/perl,
PL/python and PL/tcl, refactor the relevant code into a new
function SPI_register_trigger_data.  Call the new function in the
trigger handler of all four PLs, and document it as a public SPI
function so that authors of out-of-tree PLs can do the same.

Also get rid of a second QueryEnvironment object that was
maintained by PL/pgsql.  That was previously used to deal with
cursors, but the same approach wasn't appropriate for PLs that are
less tangled up with core code.  Instead, have SPI_cursor_open
install the connection's current QueryEnvironment, as already
happens for SPI_execute_plan.

While in the docs, remove the note that transition tables were only
supported in C and PL/pgSQL triggers, and correct some ommissions.

Thomas Munro with some work by Kevin Grittner (mostly docs)

Previously they were defined using multiples of XLogSegSize.
Remove GUC_UNIT_XSEGS. Introduce GUC_UNIT_MB

Extracted from patch series on XLogSegSize infrastructure.

Beena Emerson

This was missed in a924c327 ff.

During allocation VALGRIND_MAKE_MEM_DEFINED was called with a pointer
as size. That kind of works, but makes valgrind exceedingly slow for
workloads involving the slab allocator.

Secondly there was an access to memory marked as unreachable within
SlabCheck(). Fix that too.

Author: Tomas Vondra
Discussion: https://postgr.es/m/a6543b6d-6015-99b1-63ef-3ed55a76a730@2ndquadrant.com

Per gripe from Sven R. Kunze

2PC state info held in shmem at PREPARE, then cleaned at COMMIT PREPARED/ABORT PREPARED,
avoiding writing/fsyncing any state information to disk in the normal path, greatly enhancing replay speed.
Prepared transactions that live past one checkpoint redo horizon will be written to disk as now.
Similar conceptually to 978b2f65 and building upon
the infrastructure created by that commit.

Authors, in equal measure: Stas Kelvich, Nikhil Sontakke and Michael Paquier
Discussion: https://postgr.es/m/CAMGcDxf8Bn9ZPBBJZba9wiyQq-Qk5uqq=VjoMnRnW5s+fKST3w@mail.gmail.com

When changing the type of a sequence, adjust the min/max values of the
sequence if it looks like the previous values were the default values.
Previously, it would leave the old values in place, requiring manual
adjustments even in the usual/default cases.
Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
Reviewed-by: Vitaly Burovoy <vitaly.burovoy@gmail.com>

Dilip Kumar

Discussion: http://postgr.es/m/CAFiTN-uKAvRhWprb0i-U9zFOekgQRRwqjP1wvOBsKZb-UEKbug@mail.gmail.com

Author: Masahiko Sawada <sawada.mshk@gmail.com>

Add tab completion for COMMENT/SECURITY LABEL ON
PUBLICATION/SUBSCRIPTION.
Reported-by: Stephen Frost <sfrost@snowman.net>

Per discussion, the TAP tests are really more verbose than necessary, so
remove the --verbose flag from PROVE_FLAGS.  Also add comments to let
folks know how they can enable it if they really wish to, as suggested
by Craig Ringer.

Author: Michael Paquier, additional comments by me.
Discussion: https://postgr.es/m/CAMsr%2BYGAzcMDOZ_BirnMCL6Sb%3DMUjP0FRE82YBDSbXcf6pm9Yg%40mail.gmail.com

We need to set the origin remote position to end_lsn, not commit_lsn, as
commit_lsn is the start of commit record, and we use the origin remote
position as start position when restarting replication stream.  If we'd
use commit_lsn, we could request data that we already received from the
remote server after a crash of a downstream server.

Author: Petr Jelinek <petr.jelinek@2ndquadrant.com>

This was correct in earlier versions of the patch that lead to
commit ea69a0de, but somehow got
broken in the last version which I actually committed.

Mithun Cy, per an off-list report from Ashutosh Sharma

Discussion: http://postgr.es/m/CAD__OujbAwNU71v1y-RoQxZ8LZ6-V2UFTkex3v34MK6uZ3Xb5w@mail.gmail.com

Since hash indexes typically have very few overflow pages, adding a
new splitpoint essentially doubles the on-disk size of the index,
which can lead to large and abrupt increases in disk usage (and
perhaps long delays on occasion).  To mitigate this problem to some
degree, divide larger splitpoints into four equal phases.  This means
that, for example, instead of growing from 4GB to 8GB all at once, a
hash index will now grow from 4GB to 5GB to 6GB to 7GB to 8GB, which
is perhaps still not as smooth as we'd like but certainly an
improvement.

This changes the on-disk format of the metapage, so bump HASH_VERSION
from 2 to 3.  This will force a REINDEX of all existing hash indexes,
but that's probably a good idea anyway.  First, hash indexes from
pre-10 versions of PostgreSQL could easily be corrupted, and we don't
want to confuse corruption carried over from an older release with any
corruption caused despite the new write-ahead logging in v10.  Second,
it will let us remove some backward-compatibility code added by commit
293e24e5.

Mithun Cy, reviewed by Amit Kapila, Jesper Pedersen and me.  Regression
test outputs updated by me.

Discussion: http://postgr.es/m/CAD__OuhG6F1gQLCgMQNnMNgoCvOLQZz9zKYJQNYvYmmJoM42gA@mail.gmail.com
Discussion: http://postgr.es/m/CA+TgmoYty0jCf-pa+m+vYUJ716+AxM7nv_syvyanyf5O-L_i2A@mail.gmail.com

Amit Langote

Discussion: http://postgr.es/m/80f6b049-e882-f6c3-f82c-f44baa94d369@lab.ntt.co.jp

Craig Ringer, reviewed by me.

I intended to include this adjustment in the previous commit
(7a39b5e40) but messed up.

Currently, the only type of child relation is an "other member rel",
which is the child of a baserel, but in the future joins and even
upper relations may have child rels.  To facilitate that, introduce
macros that test to test for particular RelOptKind values, and use
them in various places where they help to clarify the sense of a test.
(For example, a test may allow RELOPT_OTHER_MEMBER_REL either because
it intends to allow child rels, or because it intends to allow simple
rels.)

Also, remove find_childrel_top_parent, which will not work for a
child rel that is not a baserel.  Instead, add a new RelOptInfo
member top_parent_relids to track the same kind of information in a
more generic manner.

Ashutosh Bapat, slightly tweaked by me.  Review and testing of the
patch set from which this was taken by Rajkumar Raghuwanshi and Rafia
Sabih.

Discussion: http://postgr.es/m/CA+TgmoagTnF2yqR3PT2rv=om=wJiZ4-A+ATwdnriTGku1CLYxA@mail.gmail.com

In a couple of places, _hash_kill_items was mistakenly called with
the buffer lock not held.  Repair.

Ashutosh Sharma, per a report from Andreas Seltenreich

Discussion: http://postgr.es/m/87o9wo8o0j.fsf@credativ.de

gset_data (aka gd) in planner.c is always non-null if and only if
parse->groupingSets is non-null, but Coverity doesn't know that and
complains.  Feed it an assertion to see if that keeps it happy.

Add necessary include files for things used in the header.

Since change of slot name is a supported operation, handle it more
gracefully, instead of in the this-should-not-happen way.

Author: Petr Jelinek <petr.jelinek@2ndquadrant.com>

Change the style of links generated by xrefs to section number only, as
it was with DSSSL, instead of number and title, as is the default of the
XSLT stylesheets.

Our documentation is mostly written expecting the old style, so keep
that for the time being, per discussion.

We already have CppAsString2, there's no need for the MSVC support to
re-invent a macro to do that (and especially not to inject it in as
ugly a way as this).

Discussion: https://postgr.es/m/CADkLM=c+hm2rc0tkKgC-ZgrLttHT2KkfppE+BC-=i-xj+7V-TQ@mail.gmail.com

This is another bit of ancient behavior that was documented poorly (in
a couple of cases) or not at all (in several others).

Discussion: https://postgr.es/m/9b4ea968-753f-4b5f-b46c-d7d3bf7c8f90@manitou-mail.org

Various psql slash commands that normally act on the current query buffer
will automatically recall and re-use the most recently executed SQL command
instead, if the current query buffer is empty.  Although this behavior is
ancient (dating apparently to commit 77a47299), it was documented nowhere
in the psql reference page.  For that matter, we'd never bothered to define
the concept of "current query buffer" explicitly.  Fix that.  Do some
wordsmithing on relevant command descriptions to improve clarity and
consistency.

Discussion: https://postgr.es/m/9b4ea968-753f-4b5f-b46c-d7d3bf7c8f90@manitou-mail.org

In commit e984ef58 I (tgl) simplified the behavior of \p to just print
the current query buffer; but Daniel Vérité points out that this made it
inconsistent with the behavior of \g and \w.  It should print the same
thing \g would execute.  Fix that, and improve related comments.

Daniel Vérité

Discussion: https://postgr.es/m/9b4ea968-753f-4b5f-b46c-d7d3bf7c8f90@manitou-mail.org

Author: Erik Rijkers

Previously, text between backquotes in a psql metacommand's arguments
was always passed to the shell literally.  That considerably hobbles
the usefulness of the feature for scripting, so we'd foreseen for a long
time that we'd someday want to allow substitution of psql variables into
the shell command.  IMO the addition of \if metacommands has brought us to
that point, since \if can greatly benefit from some sort of client-side
expression evaluation capability, and psql itself is not going to grow any
such thing in time for v10.  Hence, this patch.  It allows :VARIABLE to be
replaced by the exact contents of the named variable, while :'VARIABLE'
is replaced by the variable's contents suitably quoted to become a single
shell-command argument.  (The quoting rules for that are different from
those for SQL literals, so this is a bit of an abuse of the :'VARIABLE'
notation, but I doubt anyone will be confused.)

As with other situations in psql, no substitution occurs if the word
following a colon is not a known variable name.  That limits the risk of
compatibility problems for existing psql scripts; but the risk isn't zero,
so this needs to be called out in the v10 release notes.

Discussion: https://postgr.es/m/9561.1490895211@sss.pgh.pa.us

On ProcessUtility document the parameter, to match others.

On CreateCachedPlan drop the queryEnv parameter.  It was not
referenced within the function, and had been added on the
assumption that with some unknown future usage of QueryEnvironment
it might be useful to do something there.  We have avoided other
"just in case" implementation of unused paramters, so drop it here.

Per gripe from Tom Lane