When enabling wal_compression, there is a risk to leak data similarly to
the BREACH and CRIME attacks on SSL where the compression ratio of
a full page image gives a hint of what is the existing data of this page.
This vulnerability is quite cumbersome to exploit in practice, but doable.

So this patch makes wal_compression PGC_SUSET in order to prevent
non-superusers from enabling it and exploiting the vulnerability while
DBA thinks the risk very seriously and disables it in postgresql.conf.

Back-patch to 9.5 where wal_compression was introduced.

Gives more information in the log, to debug possible failures.

Tom fixed another one of these in commit 7f32dbcd, but there was another
almost identical one in libpq docs. Per his comment:

HP's web server has apparently become case-sensitive sometime recently.
Per bug #13479 from Daniel Abraham.  Corrected link identified by Alvaro.

Create a log file for each test run. Stdout and stderr of the test script,
as well as any subprocesses run as part of the test, are redirected to
the log file. This makes it a lot easier to debug test failures. Also print
the test output (ok 12 - ... messages) to the log file, and the command
line of any external programs executed with the system_or_bail and run_log
functions. This makes it a lot easier to debug failing tests.

Modify some of the pg_ctl and other command invocations to not use 'silent'
or 'quiet' options, and don't redirect output to /dev/null, so that you get
all the information in the log instead.

In the passing, construct some command lines in a way that works if $tempdir
contains quote-characters. I haven't systematically gone through all of
them or tested that, so I don't know if this is enough to make that work.

pg_rewind tests had a custom mechanism for creating a similar log file. Use
the new generic facility instead.

Michael Paquier and me.

Autoconf generates additional code for the first AC_CHECK_HEADERS call in
the script. If the first call is within an if-block, the additional code is
put inside the if-block too, even though it is needed by subsequent
AC_CHECK_HEADERS checks and should always be executed. When I moved the
pthread-related checks earlier in the script, the pthread.h test inside
the block became the very first AC_CHECK_HEADERS call in the script,
triggering that problem.

To fix, use AS_IF instead of plain shell if. AS_IF knows about that issue,
and makes sure the additional code is always executed. To be completely
safe from this issue (and others), we should always be using AS_IF instead
of plain if, but that seems like excessive caution given that this is the
first time we have trouble like this. Plain if-then is more readable than
AS_IF.

This should fix compilation with --disable-thread-safety, and hopefully the
buildfarm failure on forgmouth, related to mingw standard headers, too.
I backpatched the previous fixes to 9.5, but it's starting to look like
these changes are too fiddly to backpatch, so commit this to master only,
and revert all the pthread-related configure changes in 9.5.

The AIX 7.1 libm is static, and AIX postgres executables do not export
symbols acquired from libraries.  Back-patch to 9.5, where commit
cfe12763 added a sqrt() call.

This evades a ppc64le "IBM XL C/C++ for Linux" compiler bug.  Back-patch
to 9.5, where the atomics facility was introduced.

Back-patch to 9.5, where commit b64d92f1
introduced this file.

This formalizes a decision implicit in commit
4ea51cdf and adds clean detection of
affected systems.  Vendor updates are available for each such known bug.
Back-patch to 9.5, where the aforementioned commit first appeared.

POSIX does not specify the -q option, and many implementations do not
offer it.  Don't bother changing the MSVC build system, because having
non-GNU diff on Windows is vanishingly unlikely.  Back-patch to 9.2,
where this invocation was introduced.

The psql crash happened when no current connection existed.  (The second
new check is optional given today's undocumented NULL argument handling
in PQhost() etc.)  Back-patch to 9.0 (all supported versions).

On some Linux systems, "-lrt" exposed pthread-functions, so that linking
with -lrt was seemingly enough to make a program that uses pthreads to
work. However, when linking libpq, the dependency to libpthread was not
marked correctly, so that when an executable was linked with -lpq but
without -pthread, you got errors about undefined pthread_* functions from
libpq.

To fix, test for the flags required to use pthreads earlier in the autoconf
script, before checking any other libraries.

This should fix the failure on buildfarm member shearwater. gharial is also
failing; hopefully this fixes that too although the failure looks somewhat
different.

Our version was different from the upstream version in that we tried to use
all possible pthread-related flags that the compiler accepts, rather than
just the first one that works. That change was made in commit
e48322a6, to work-around a bug affecting GCC
versions 3.2 and below (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8888),
although we didn't realize that it was a GCC bug at the time. We hardly care
about that old GCC versions anymore, so we no longer need that workaround.

This fixes the macro for compilers that print warnings with the chosen
flags. That's pretty annoying on its own right, but it also inconspicuously
disabled thread-safety, because we refused to use any pthread-related flags
if the compiler produced warnings. Max Filippov reported that problem when
linking with uClibc and OpenSSL. The warnings-check was added because the
workaround for the GCC bug caused warnings otherwise, so it's no longer
needed either. We can just use the upstream version as is.

If you really want to compile with GCC version 3.2 or older, you can still
work-around it manually by setting PTHREAD_CFLAGS="-pthread -lpthread"
manually on the configure command line.

Backpatch to 9.5. I don't want to unnecessarily rock the boat on stable
branches, but 9.5 seems like fair game.

Test the interactions with permissions and LOCK TABLE. Specifically
ROW EXCLUSIVE, ACCESS SHARE, and ACCESS EXCLUSIVE modes against
SELECT, INSERT, UPDATE, DELETE, and TRUNCATE permissions. Discussed
by Stephen Frost and Michael Paquier, patch by the latter. Backpatch
to 9.5 where matching behavior was first committed.

Back-patch to 9.5 where the bug was introduced.

David Christensen

SUSv2-era shells don't set the PWD variable, though anything more modern
does.  In the buildfarm environment this could lead to test.sh executing
with PWD pointing to $HOME or another high-level directory, so that there
were conflicts between concurrent executions of the test in different
branch subdirectories.  This appears to be the explanation for recent
intermittent failures on buildfarm members binturong and dingo (and might
well have something to do with the buildfarm script's failure to capture
log files from pg_upgrade tests, too).

To fix, just use `pwd` in place of $PWD.  AFAICS test.sh is the only place
in our source tree that depended on $PWD.  Back-patch to all versions
containing this script.

Per buildfarm.  Thanks to Oskari Saarenmaa for diagnosing the problem.

If an allocation fails in the main message handling loop, pqParseInput3
or pqParseInput2, it should not be treated as "not enough data available
yet". Otherwise libpq will wait indefinitely for more data to arrive from
the server, and gets stuck forever.

This isn't a complete fix - getParamDescriptions and getCopyStart still
have the same issue, but it's a step in the right direction.

Michael Paquier and me. Backpatch to all supported versions.

Based on the original code from David Christensen, modified by me.

Per Tom.

The substitution for the pid is %p.

Author: Julien Rouhaud
Discussion: 116262CF971C844FB6E793F8809B51C6E99D48@BPXM02GP.gisp.nec.co.jp

When spilling transaction data to disk a simple typo caused the output
file to be closed and reopened for every serialized change. That happens
to not have a huge impact on linux, which is why it probably wasn't
noticed so far, but on windows that appears to trigger actual disk
writes after every change. Not fun.

The bug fortunately does not have any impact besides speed. A change
could end up being in the wrong segment (last instead of next), but
since we read all files to the end, that's just ugly, not really
problematic. It's not a problem to upgrade, since transaction spill
files do not persist across restarts.

Bug: #13484
Reported-By: Olivier Gosseaume
Discussion: 20150703090217.1190.63940@wrigleys.postgresql.org

Backpatch to 9.4, where logical decoding was added.

The previous coding tried to handle possible failures when fsyncing a
tty or pipe fd by accepting EINVAL - but apparently some
platforms (windows, OSX) don't reliably return that. So instead check
whether the output fd refers to a pipe or a tty when opening it.

Reported-By: Olivier Gosseaume, Marko Tiikkaja
Discussion: 559AF98B.3050901@joh.to

Backpatch to 9.4, where pg_recvlogical was added.

Also updated regression expected output to match. Noted and patch by Daniele Varrazzo.

Build.bat and vcregress.bat got similar treatment years ago. I'm not sure
why install.bat wasn't treated at the same time, but it seems like a good
idea anyway.

The immediate problem with the old install.bat was that it had quoting
issues, and wouldn't work if the target directory's name contained spaces.
This fixes that problem.

We're interested in the buffer size of the socket that's connected to the
client, not the one that's listening for new connections. It happened to
work, as default buffer size is the same on both, but it was clearly not
wrong.

Spotted by Tom Lane

It's unnecessary to set it if the default is higher in the first place.
Furthermore, setting SO_SNDBUF disables the so-called "dynamic send
buffering" feature, which hurts performance further. This can be seen
especially when the network between the client and the server has high
latency.

Chen Huajun

The .backup file name can be passed to pg_archivecleanup even if
it includes the extension which is specified in -x option.
However, previously the document incorrectly warned a user
not to do that.

Back-patch to 9.2 where pg_archivecleanup's -x option and
the warning were added.

The lack of consistency, and lack of attention to our message style
guidelines, was a bit striking.  Try to make 'em better.

Back-patch to avoid unnecessary cross-branch differences.

CharSyam

This builds on commit 21dcda27 by keeping
a plpgsql function's shared ParamListInfo's entries for simple variables
(PLPGSQL_DTYPE_VARs) valid at all times.  That adds a few cycles to each
assignment to such variables, but saves significantly more cycles each time
they are used; so except in the pathological case of many dead stores, this
should always be a win.  Initial testing says it's good for about a 10%
speedup of simple calculations; more in large functions with many datums.

We can't use this method for row/record references unfortunately, so what
we do for those is reset those ParamListInfo slots after use; which we
can skip doing unless some of them were actually evaluated during the
previous evaluation call.  So this should frequently be a win as well,
while worst case is that it's similar cost to the previous approach.

Also, closer study suggests that the previous method of instantiating a
new ParamListInfo array per evaluation is actually probably optimal for
cursor-opening executor calls.  The reason is that whatever is visible in
the array is going to get copied into the cursor portal via copyParamList.
So if we used the function's main ParamListInfo for those calls, we'd end
up with all of its DTYPE_VAR vars getting copied, which might well include
large pass-by-reference values that the cursor actually has no need for.
To avoid a possible net degradation in cursor cases, go back to creating
and filling a private ParamListInfo in those cases (which therefore will be
exactly the same speed as before 21dcda27).  We still get some benefit
out of this though, because this approach means that we only have to defend
against copyParamList's try-to-fetch-every-slot behavior in the case of an
unshared ParamListInfo; so plpgsql_param_fetch() can skip testing
expr->paramnos in the common case.

To ensure that the main ParamListInfo's image of a DTYPE_VAR datum is
always valid, all assignments to such variables are now funneled through
assign_simple_var().  But this makes for cleaner and shorter code anyway.

Christoph Berg

Michael Paquier, reviewed by Fabien Coelho

These are basically just like the \ef and \sf commands for functions.

Petr Korobeinikov, reviewed by Jeevan Chalke, some changes by me

You can no longer use pgbench with multiple threads when compiled without
--enable-thread-safety. That's an acceptable limitation these days; it
still works fine with -j1, and all modern platforms support threads anyway.
This makes future maintenance and development of the code easier.

Fabien Coelho

There were two issues here. First, if a query got stuck so that it took
e.g. 5 seconds, and progress interval was 1 second, no progress reports were
printed until the query returned. Fix so that we wake up specifically to
print the progress report. Secondly, if pgbench got stuck so that it would
nevertheless not print a progress report on time, and enough time passes
that it's already time to print the next progress report, just skip the one
that was missed. Before this patch, it would print the missed one with 0 TPS
immediately after the previous one.

Fabien Coelho. Backpatch to 9.4, where progress reports were added.

Fabien Coelho

Commit de768844 changed an archive recovery so that the last WAL
segment with old timeline was renamed with suffix .partial. It should
have updated WAL-related utilities so that they can handle such
.paritial WAL files, but we forgot that.

This patch changes pg_archivecleanup so that it can clean up even
archived WAL files with .partial suffix. Also it allows us to specify
.partial WAL file name as the command-line argument "oldestkeptwalfile".

This patch also changes pg_resetxlog so that it can remove .partial
WAL files in pg_xlog directory.

pg_xlogdump cannot handle .partial WAL files. Per discussion,
we decided only to document that limitation instead of adding the fix.
Because a user can easily work around the limitation (i.e., just remove
.partial suffix from the file name) and the fix seems complicated for
very narrow use case.

Back-patch to 9.5 where the problem existed.

Review by Michael Paquier.
Discussion: http://www.postgresql.org/message-id/CAHGQGwGxMKnVHGgTfiig2Bt_2djec0in3-DLJmtg7+nEiidFdQ@mail.gmail.com

-t will now match views, foreign tables, materialized views, and sequences,
not only plain tables.  This is more useful, and also more consistent with
the behavior of pg_dump's -t switch, which has always matched all relation
types.

We're still not there on matching pg_dump's behavior entirely, so mention
that in the docs.

Craig Ringer, reviewed by Pavel Stehule

Expose PG_VERSION_NUM (e.g., "90600") as a Make variable; but for
consistency with the other Make variables holding similar info,
call the variable just VERSION_NUM not PG_VERSION_NUM.

There was some discussion of making this value available as a pg_config
value as well.  However, that would entail substantially more work than
this two-line patch.  Given that there was not exactly universal consensus
that we need this at all, let's just do a minimal amount of work for now.

Michael Paquier, reviewed by Pavel Stehule