606c0123 attempted to reduce cost of index scans using > and <
strategies, though got that completely wrong in a few complex cases.

Revert whole patch until we find a safe optimization.

Rename pl/pgsql global variables to always have a plpgsql_ prefix,
so they don't conflict with other shared libraries loaded.

When adding replication origins in 5aa23504, I somehow managed to set
the timestamp of decoded transactions to InvalidXLogRecptr when decoding
one made without a replication origin. Fix that, and the wrong type of
the new commit_time variable.

This didn't trigger a regression test failure because we explicitly
don't show commit timestamps in the regression tests, as they obviously
are variable. Add a test that checks that a decoded commit's timestamp
is within minutes of NOW() from before the commit.

Reported-By: Weiping Qu
Diagnosed-By: Artur Zakirov
Discussion: 56D4197E.9050706@informatik.uni-kl.de,
    56D42918.1010108@postgrespro.ru
Backpatch: 9.5, where 5aa23504 originates.

A thinko concerning nesting depth caused json_to_record() to produce bogus
output if a field of its input object contained a sub-object with a field
name matching one of the requested output column names.  Per bug #13996
from Johann Visagie.

I added a regression test case based on his example, plus parallel tests
for json_to_recordset, jsonb_to_record, jsonb_to_recordset.  The latter
three do not exhibit the same bug (which suggests that we may be missing
some opportunities to share code...) but testing seems like a good idea
in any case.

Back-patch to 9.4 where these functions were introduced.

Commits 9ff60273 and dbe23289 adjusted the declarations
of some core functions referenced by contrib/tsearch2's install script,
forgetting that in a pg_upgrade situation, we'll be trying to restore
operator class definitions that reference the old signatures.  We've
hit this problem before; solve it in the same way as before, namely by
installing stub functions that have the expected signature and just
invoke the correct function.  Per report from Jeff Janes.

(Someday we ought to stop supporting contrib/tsearch2, but I'm not
sure today is that day.)

This makes it easier to relate the temporary data dirs to each node in
a test script.

Author: Kyotaro Horiguchi
Reviewed-By: Craig Ringer, Alvaro Herrera

PL/Tcl appears to contain logic to convert strings between the database
encoding and UTF8, which is the only encoding modern Tcl will deal with.
However, that code has been disabled since commit 03489512, which
made it "#if defined(UNICODE_CONVERSION)" and neglected to provide any way
for that symbol to become defined.  That might have been all right back
in 2001, but these days we take a dim view of allowing strings with
incorrect encoding into the database.

Remove the conditional compilation, fix warnings about signed/unsigned char
conversions, clean up assorted places that didn't bother with conversions.
(Notably, there were lots of assumptions that database table and field
names didn't need conversion...)

Add a regression test based on plpython_unicode.  It's not terribly
thorough, but better than no test at all.

As of commit 28782206, PL/Tcl will not
compile against pre-8.0 Tcl, whereas it used to work (more or less anyway)
with quite prehistoric versions.  As long as we're moving these goalposts,
let's reinstall them at someplace that has some thought behind it.  This
commit sets the minimum allowed Tcl version at 8.4, and rips out some bits
of compatibility cruft that are in consequence no longer needed.  Reasons
for requiring 8.4 include:

* 8.4 was released in 2002; there seems little reason to believe that
anyone would want to use older versions with Postgres 9.6+.

* We have no buildfarm members testing anything older than 8.4, and
thus no way to know if it's broken.

* We need at least 8.1 to allow enforcement of database encoding
security (8.1 standardized Tcl on using UTF8 internally, before that
it was pretty unpredictable).

* Some versions between 8.1 and 8.4 allowed the backend to become
multithreaded, which is disastrous.  We need at least 8.4 to be able
to disable the Tcl notifier subsystem to prevent that.

A small side benefit is that we can make the code more readable by
doing s/CONST84/const/g.

The original implementation of Tcl was all strings, but they improved
performance significantly by introducing typed "objects" (integers,
lists, code, etc).  It's past time we made use of that; that happened
in Tcl 8.0 which was released in 1997.

This patch also modernizes some of the error-reporting code, which may
cause small changes in the spelling of complaints about bad calls to
PL/Tcl-provided commands.

Jim Nasby and Karl Lehenbauer, reviewed by Victor Wagner

Commit 7132810c (Retain tempdirs for failed tests) used Test::More's
is_passing method, but that was added in Test::More 0.89_01 which is
sometime later than Perl 5.10.1.  Popular platforms such as RHEL6 don't
have that, nevermind some of our older dinosaurs.  Do it the hard way.

Michael Paquier, based on research by Craig Ringer

The new bit indicates whether every tuple on the page is already frozen.
It is cleared only when the all-visible bit is cleared, and it can be
set only when we vacuum a page and find that every tuple on that page is
both visible to every transaction and in no need of any future
vacuuming.

A future commit will use this new bit to optimize away full-table scans
that would otherwise be triggered by XID wraparound considerations.  A
page which is merely all-visible must still be scanned in that case, but
a page which is all-frozen need not be.  This commit does not attempt
that optimization, although that optimization is the goal here.  It
seems better to get the basic infrastructure in place first.

Per discussion, it's very desirable for pg_upgrade to automatically
migrate existing VM forks from the old format to the new format.  That,
too, will be handled in a follow-on patch.

Masahiko Sawada, reviewed by Kyotaro Horiguchi, Fujii Masao, Amit
Kapila, Simon Riggs, Andres Freund, and others, and substantially
revised by me.

Test composite-type arguments and the argisnull and spi_lastoid Tcl
commmands.  This stuff was not covered before, but needs to be exercised
since the upcoming Tcl object-conversion patch changes these code paths
(and broke at least one of them).

These tests verify that 1) WAL replay preserves the stored value,
2) a streaming standby server replays the value obtained from the
master, and 3) the behavior is sensible in the face of repeated
configuration changes.

One annoyance is that tmp_check/ subdir from the TAP tests is clobbered
when the pg_regress test runs in the same subdirectory.  This is
bothersome but not too terrible a problem, since the pg_regress test is
not run anyway if the TAP tests fail (unless "make -k" is used).

I had these tests around since commit 69e7235c; add them now that we
have the recovery test framework in place.

This makes it easier to study the reason for the failure.

Author: Kyotaro Horiguchi
Reviewed-By: Craig Ringer

PQmblen and PQdsplen return information about characters, not words.

Kyotaro Horiguchi

I believe that I (rhaas) introduced this bug while editing the patch
that became bcac23de.

Report and patch from KaiGai Kohei.

Fabien Coelho, reviewed mostly by Michael Paquier and me, but also by
Heikki Linnakangas, BeomYong Lee, Kyotaro Horiguchi, Oleksander
Shulgin, and Álvaro Herrera.

That way, the rules will trigger when the scripts change.

Some of these comments were copied and pasted without updating them,
some of them were duplicates.

I noticed that the async-notify test results in log messages like these:

LOG:  could not send data to client: Broken pipe
FATAL:  connection to client lost

This is because it unceremoniously disconnects a client session that is
about to have some NOTIFY messages delivered to it.  Such log messages
during a regression test might well cause people to go looking for a
problem that doesn't really exist (it did cause me to waste some time that
way).  We can shut it up by adding an UNLISTEN command to session teardown.

Patch HEAD only; this doesn't seem significant enough to back-patch.

This error message was written with only ON SELECT rules in mind, but since
then we also made RETURNING-clause targetlists go through the same logic.
This means that you got a rather off-topic error message if you tried to
add a rule with RETURNING to a table having dropped columns.  Ideally we'd
just support that, but some preliminary investigation says that it might be
a significant amount of work.  Seeing that Nicklas Avén's complaint is the
first one we've gotten about this in the ten years or so that the code's
been like that, I'm unwilling to put much time into it.  Instead, improve
the error report by issuing a different message for RETURNING cases, and
revise the associated comment based on this investigation.

Discussion: 1456176604.17219.9.camel@jordogskog.no

Author: Michael Paquier

Author: Amit Langote

The docs were advising to use /usr/local/pgsql/man instead, but that's
wrong.

Reported-By: Slawomir Sudnik
Backpatch-To: 9.1
Bug: #13894

The original coding of the test was relying too much on the ordering in
which backends are awakened once an advisory lock which they wait for is
released.  Change the code so that each backend uses its own advisory
lock instead, so that the output becomes stable.  Also add a few seconds
of sleep between lock releases, so that the test isn't broken in
overloaded buildfarm animals, as suggested by Tom Lane.

Per buildfarm members spoonbill and guaibasaurus.

Discussion: https://www.postgresql.org/message-id/19294.1456551587%40sss.pgh.pa.us

It's harmless, but might confuse readers.  Seems to have been introduced
in 6bc8ef0b.  Back-patch, just to avoid cosmetic cross-branch
differences.

Amit Langote

In commit 19a54114 I replaced RelOptInfo.width with
RelOptInfo.reltarget.width, but I missed updating debug_print_rel()
for that because it's not compiled by default.
Reported by Salvador Fandino, patch by Michael Paquier.

When converting an RTE with securityQuals into a security barrier
subquery RTE, ensure that the Vars in the new subquery's targetlist
all have varlevelsup = 0 so that they correctly refer to the
underlying base relation being wrapped.

The original code was creating new Vars by copying them from existing
Vars referencing the base relation found elsewhere in the query, but
failed to account for the fact that such Vars could come from sublink
subqueries, and hence have varlevelsup > 0. In practice it looks like
this could only happen with nested security barrier views, where the
outer view has a WHERE clause containing a correlated subquery, due to
the order in which the Vars are processed.

Bug: #13988
Reported-by: Adam Guthrie
Backpatch-to: 9.4, where updatable SB views were introduced

A failure partway through PGLC_localeconv() led to a situation where
the next call would call free_struct_lconv() a second time, leading
to free() on already-freed strings, typically leading to a core dump.
Add a flag to remember whether we need to do that.

Per report from Thom Brown.  His example case only provokes the failure
as far back as 9.4, but nonetheless this code is obviously broken, so
back-patch to all supported branches.

This means that if, for example, TEMP_CONFIG is set and a Makefile
explicitly sets a temp-config file, both will now be used.

Patch from John Gorman.

This reverts commit 9117985b in favor of
a more general solution.

In 0e5680f4, I fixed a bug in heapam that caused spurious deadlocks
when multiple updates concurrently attempted to modify the old version
of an updated tuple whose new version was key-share locked.  I proposed
an isolationtester spec file that reproduced the bug, but back then
isolationtester wasn't mature enough to be able to run it.  Now that
38f8bdca is in the tree, we can have this spec file too.

Discussion: https://www.postgresql.org/message-id/20141212205254.GC1768%40alvh.no-ip.org

I applied the previous-to-last revision of Michaël's submitted patch
instead of the last; these two tweaks pointed out by Craig were left out
of the previous commit by accident.

This long-awaited framework is an expansion of the existing PostgresNode
stuff to support additional features for recovery testing; the recovery
tests included in this commit are a starting point that cover some of
the recovery features we have.  More scripts are expected to be added
later.

Author: Michaël Paquier, a bit of help from Amir Rohan
Reviewed by: Amir Rohan, Stas Kelvich, Kyotaro Horiguchi, Victor Wagner,
Craig Ringer, Álvaro Herrera
Discussion: http://www.postgresql.org/message-id/CAB7nPqTf7V6rswrFa=q_rrWeETUWagP=h8LX8XAov2Jcxw0DRg@mail.gmail.com
Discussion: http://www.postgresql.org/message-id/trinity-b4a8035d-59af-4c42-a37e-258f0f28e44a-1443795007012@3capp-mailcom-lxa08

Some code in the RewindTest test suite is more generally useful than
just for that suite, so put it where other test suites can reach it.

Some postgresql.conf parameters change their default values when a
cluster is initialized with 'allows_streaming' than the previous
behavior; most notably, autovacuum is no longer turned off.

(Also, we no longer call pg_ctl promote with -w, but that flag doesn't
actually do anything in promote so there's no behavior change.)

Author: Michael Paquier

CREATE TABLE .. AS EXECUTE can turn an apparently read-only query into
a write operation, which parallel query can't handle.  It's a bit of a
shame that requires us to avoid parallel query for queries prepared via
PREPARE in all cases, but for right now it does.

This is basically a bug fix; the old code assumes that a ForeignScan
is always parallel-safe, but for postgres_fdw, for example, this is
definitely false.  It should be true for file_fdw, though, since a
worker can read a file from the filesystem just as well as any other
backend process.

Original patch by Thomas Munro.  Documentation, and changes to the
comments, by me.

Thomas Munro

Also, the dump_info method got split into another method that returns
the stuff as a string instead of just printing it to stdout.

Add a new README in src/test/perl too.

Author: Craig Ringer
Reviewed by: Michaël Paquier