based on patch from Josh Soref <jsoref@gmail.com>

From: Josh Soref <jsoref@gmail.com>

From: Josh Soref <jsoref@gmail.com>

The original messaging design, introduced in commit 068cfadf, seems too
chatty now that some time has elapsed since the bug fix; most installations
will be in good shape and don't really need a reminder about this on every
postmaster start.

Hence, arrange to suppress the "wraparound protections are now enabled"
message during startup (specifically, during the TrimMultiXact() call).
The message will still appear if protection becomes effective at some
later point.

Discussion: https://postgr.es/m/17211.1489189214@sss.pgh.pa.us

Commit 42fcad68 failed to do this.

Michael Paquier

Discussion: http://postgr.es/m/CAB7nPqTXK9Qn8TmonPf29jNXGw_CA3fTDaRfgnbHCFYr-Tx6qw@mail.gmail.com

This could result in corruption of the init fork of an unlogged index
if the ambuildempty routine for that index used shared buffers to
create the init fork, which was true for brin, gin, gist, and hash
indexes.

Patch by me, based on an earlier patch by Michael Paquier, who also
reviewed this one.  This also incorporates an idea from Artur
Zakirov.

Discussion: http://postgr.es/m/CACYUyc8yccE4xfxhqxfh_Mh38j7dRFuxfaK1p6dSNAEUakxUyQ@mail.gmail.com

Commit 355d3993 probably should have
done this, but nobody noticed that it was needed.

This logic was adapated from create_merge_append_plan, but the two
cases aren't really analogous, because create_merge_append_plan is not
projection-capable and must therefore have a tlist identical to that
of the underlying paths.  Overwriting the tlist of Gather Merge with
whatever the underlying plan happens to produce is no good at all.

Patch by me, reviewed by Rushabh Lathia, who also reported the issue
and made an initial attempt at a fix.

Discussion: http://postgr.es/m/CA+Tgmob_-oHEOBfT9S25bjqokdqv8e8xEmh9zOY+3MPr_LmuhA@mail.gmail.com

Oversight in commit 9ca5c872.  Per buildfarm.

Fallout from fcec6caa: mark a variable in
set_tablefunc_size_estimates as used for asserts only.

Also, the planner_rte_fetch() call is pointless with assertions
disabled, so enclose it in a USE_ASSERT_CHECKING #ifdef; fix the same
problem in set_subquery_size_estimates().

First problem noted by David Rowley, whose compiler is noisier than mine
in this regard.

The immediate motivation for this is to provide clean infrastructure
for the proposed \if...\endif patch for psql; but it seems like a good
thing to have even if that patch doesn't get in.  Previously the callback
functions could only make use of application-global state, which is a
pretty severe handicap.

For the moment, the pointer is only passed through to the get_variable
callback function.  I considered also passing it to the write_error
callback, but for now let's not.  Neither psql nor pgbench has a use
for that, and in the case of psql we'd have to invent a separate wrapper
function because we would certainly not want to change the signature of
psql_error().

Discussion: https://postgr.es/m/10108.1489418309@sss.pgh.pa.us

Rather than waiting around for statement_timeout to expire, we can just
try to take the table's lock in nowait mode.  This saves some fraction
under 4 seconds when running this test with prepared xacts available,
and it guards against timeout-expired-anyway failures on very slow
machines when prepared xacts are not available, as seen in a recent
failure on axolotl for instance.

This approach could fail if autovacuum were to take an exclusive lock
on the test table concurrently, but there's no reason for it to do so.

Since the main point here is to improve stability in the buildfarm,
back-patch to all supported branches.

Patch by Masahiko Sawada

The problem was that "begin transaction" was issued automatically
before executing COMMIT/ROLLBACK PREPARED if not in auto commit. This fix by
Masahiko Sawada fixes this.

Some compilers require it. At least Visual Studio, according to the
buildfarm, and gcc with the -pedantic flag.

From: David Rowley <david.rowley@2ndquadrant.com>

Replace the mapping tables used to convert between UTF-8 and other
character encodings with new radix tree-based maps. Looking up an entry in
a radix tree is much faster than a binary search in the old maps. As a
bonus, the radix tree representation is also more compact, making the
binaries slightly smaller.

The "combined" maps work the same as before, with binary search. They are
much smaller than the main tables, so it doesn't matter so much. However,
the "combined" maps are now stored in the same .map files as the main
tables. This seems more clear, since they're always used together, and
generated from the same source files.

Patch by Kyotaro Horiguchi, with lot of hacking by me at various stages.
Reviewed by Michael Paquier and Daniel Gustafsson.

Discussion: https://www.postgresql.org/message-id/20170306.171609.204324917.horiguchi.kyotaro%40lab.ntt.co.jp

We don't use those files anymore, since commit 1de9cc0d.

Masahiko Sawada

This makes almost all core code follow the policy introduced in the
previous commit.  Specific decisions:

- Text search support functions with char* and length arguments, such as
  prsstart and lexize, may receive unaligned strings.  I doubt
  maintainers of non-core text search code will notice.

- Use plain VARDATA() on values detoasted or synthesized earlier in the
  same function.  Use VARDATA_ANY() on varlenas sourced outside the
  function, even if they happen to always have four-byte headers.  As an
  exception, retain the universal practice of using VARDATA() on return
  values of SendFunctionCall().

- Retain PG_GETARG_BYTEA_P() in pageinspect.  (Page images are too large
  for a one-byte header, so this misses no optimization.)  Sites that do
  not call get_page_from_raw() typically need the four-byte alignment.

- For now, do not change btree_gist.  Its use of four-byte headers in
  memory is partly entangled with storage of 4-byte headers inside
  GBT_VARKEY, on disk.

- For now, do not change gtrgm_consistent() or gtrgm_distance().  They
  incorporate the varlena header into a cache, and there are multiple
  credible implementation strategies to consider.

When commit 3e23b68d introduced
single-byte varlena headers, its fmgr.h changes presented
PG_GETARG_TEXT_PP() and PG_GETARG_TEXT_P() as equals.  Its postgres.h
changes presented PG_DETOAST_DATUM_PACKED() and VARDATA_ANY() as the
exceptional case.  Now, instead, firmly recommend PG_GETARG_TEXT_PP()
over PG_GETARG_TEXT_P(); likewise for other ...PP() macros.  This shaves
cycles and invites consistency of style.

Detect fclose() failures; given "ln -s /dev/full $PGDATA/devfull",
"pg_file_write('devfull', 'x', true)" now fails as it should.  Don't
leak a stream when fwrite() fails.  Remove a born-ineffective test that
aimed to skip zero-length writes.  Back-patch to 9.2 (all supported
versions).

In functions that issue a deconstruct_array() call, consistently use
plain VARSIZE()/VARDATA() on the array elements.  Prior practice was
divided between those and VARSIZE_ANY_EXHDR()/VARDATA_ANY().

When commit 3e23b68d introduced
single-byte varlena headers, it rendered this comment incomplete.

This could only matter if the guessed_type variable had a value that wasn't
a member of the PasswordType enum; but just in case, let's be sure that
control falls out to reach the elog(ERROR) at the end of the function.
Per gripe from Coverity.

Noted by Coverity (a rather impressive catch).

Michael Paquier

Coverity noted that the last line of gather_merge_getnext() was
unreachable, since each arm of the preceding "if" ends in a "return".
Drop it as an oversight.  In passing, improve some nearby comments.

Upcoming patches are revamping expression evaluation significantly. It
therefore seems prudent to try to ensure that the coverage of the
existing evaluation code is high.

This commit adds coverage for the cases that can reasonably be
tested. There's still a bunch of unreachable error messages and such,
but otherwise this achieves nearly full regression test coverage (with
the exception of the unused GetAttributeByNum/GetAttributeByName).

Author: Andres Freund
Discussion: https://postgr.es/m/20170310194021.ek4bs4bl2khxkmll@alap3.anarazel.de

When using unnamed connections with dblink, every time a new
connection is made, the old one is leaked. Fix that.

This has been an issue probably since dblink was first committed.
Someone complained almost ten years ago, but apparently I decided
not to pursue it at the time, and neither did anyone else, so it
slipped between the cracks. Now that someone else has complained,
fix in all supported branches.

Discussion: (orig) https://postgr.es/m/flat/F680AB59-6D6F-4026-9599-1BE28880273D%40decibel.org#F680AB59-6D6F-4026-9599-1BE28880273D@decibel.org
Discussion: (new) https://postgr.es/m/flat/0A3221C70F24FB45833433255569204D1F6ADF8C@G01JPEXMBYT05
Reported by: Jim Nasby and Takayuki Tsunakawa

This allows rolling back the effects of some SPI commands without
having to fail the entire PL/Tcl function.

Victor Wagner, reviewed by Pavel Stehule

Discussion: https://postgr.es/m/20170108205750.2dab04a1@wagner.wagner.home

When one of the kernel calls in the socket()/bind()/listen() sequence
fails, include the specific address we're trying to bind to in the log
message.  This greatly eases debugging of network misconfigurations.

Also, after successfully setting up a listen socket, report its address
in the log, to ease verification that the expected addresses were bound.
There was some debate about whether to print this message at LOG level or
only DEBUG1, but the majority of votes were for the former.

Discussion: https://postgr.es/m/9564.1489091245@sss.pgh.pa.us

Dumping a publication with more than one table crashed pg_dump.

patch by Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>, test by me

There's no really good reason why the autovacuum launcher and logical
replication launcher should announce themselves at startup and shutdown
by default.  Users don't care that those processes exist, and it's
inconsistent that those background processes announce themselves while
others don't.  So, reduce those messages from LOG to DEBUG1 level.

I was sorely tempted to reduce the "starting logical replication worker
for subscription ..." message to DEBUG1 as well, but forebore for now.
Those processes might possibly be of direct interest to users, at least
until logical replication is a lot better shaken out than it is today.

Discussion: https://postgr.es/m/19479.1489121003@sss.pgh.pa.us

This reverts commit ccce90b3.  This
optimization is unsafe, at least, of rollbacks and rollbacks to
savepoints, but I'm concerned there may be other problematic cases as
well.  Therefore, I've decided to revert this pending further
investigation.

Previously they were disabled due to performance concerns on 32bit
arm, where 64bit atomics are often implemented via kernel traps.

Author: Roman Shaposhnik
Discussion: http://postgr.es/m/CA+ULb+uErkFuXUCCXWHYvnV5KnAyjGUzzRcPA-M0cgO+Hm4RSA@mail.gmail.com

Commits 89e0bac8 et al replaced newlines with spaces in object names
printed in SQL comments, but we neglected to consider that the same
names are also printed by "pg_restore -l", and a newline would render
the output unparseable by "pg_restore -L".  Apply the same replacement
in "-l" output.  Since "pg_restore -L" doesn't actually examine any
object names, only the dump ID field that starts each line, this is
enough to fix things for its purposes.

The previous fix was treated as a security issue, and we might have
done that here as well, except that the issue was reported publicly
to start with.  Anyway it's hard to see how this could be exploited
for SQL injection; "pg_restore -L" doesn't do much with the file
except parse it for leading integers.

Per bug #14587 from Milos Urbanek.  Back-patch to all supported versions.

Discussion: https://postgr.es/m/20170310155318.1425.30483@wrigleys.postgresql.org