We need to use a stamp file to record the runs of these scripts, as
is done on the Unix side.  I think I got it right, but can't test.

While at it, extend this handmade dependency logic to also check the
generating script files, as the makefiles do.

Discussion: https://postgr.es/m/16925.1525376229@sss.pgh.pa.us

If a particular output file already exists with the contents it should
have, leave it alone, so that its mod timestamp is not advanced.

In builds using --enable-depend, this can avoid the need to recompile .c
files whose included files didn't actually change.  It's not clear whether
it saves much of anything for users of ccache; but the cost of doing the
file comparisons seems to be negligible, so we might as well do it.

For developers using the MSVC toolchain, this will create a regression:
msvc/Solution.pm will sometimes run genbki.pl or Gen_fmgrtab.pl
unnecessarily.  I'll look into fixing that separately.

Discussion: https://postgr.es/m/16925.1525376229@sss.pgh.pa.us

Make these rules look more like the ones associated with genbki.pl,
to wit:

* Use a stamp file to record when we last ran the script, instead of
relying on the timestamps of the individual output files.

* Take the knowledge out of backend/Makefile and put it in utils/Makefile
where it belongs.  I moved down the handling of errcodes.h and probes.h
too, although those continue to be built by separate processes.

In itself, this is just much-needed cleanup with little practical effect.
However, by decoupling these makefile rules from the timestamps of the
generated header files, we open the door to not advancing those timestamps
unnecessarily, which will be taken advantage of by the next commit.

msvc/Solution.pm should be taught to do things similarly, but I'll leave
that for another commit.

Discussion: https://postgr.es/m/16925.1525376229@sss.pgh.pa.us

Python 3.7 removes the trailing comma in the repr() of
BaseException (see <https://bugs.python.org/issue30399>), leading to
test output differences.  Work around that by composing the equivalent
test output in a more manual way.

If an interrupt arrives in the middle of FinishPreparedTransaction
and any callback decide to call CHECK_FOR_INTERRUPTS (e.g.
RemoveTwoPhaseFile can write a warning with ereport, which checks for
interrupts) then it's possible to leave current GXact undeleted.

Backpatch to all supported branches

Stas Kelvich

Discussion: ihttps://www.postgresql.org/message-id/3AD85097-A3F3-4EBA-99BD-C38EDF8D2949@postgrespro.ru

autoprewarm.c mostly considered the number of blocks it might be dealing
with as being int64.  This is unnecessary, because NBuffers is declared
as int, and there's been no suggestion that we might widen it in the
foreseeable future.  Moreover, using int64 is problematic because the
code expected INT64_FORMAT to work with fscanf(), something we don't
guarantee, and which indeed fails on some older buildfarm members.

On top of that, the module randomly used uint32 rather than int64 variables
to hold block counters in several places, so it would fail anyway if we
ever did have NBuffers wider than that; and it also supposed that pg_qsort
could sort an int64 number of elements, which is wrong on 32-bit machines
(though no doubt a 32-bit machine couldn't actually have that many
buffers).

Hence, change all these variables to plain int.

In passing, avoid shadowing one variable named i with another,
and avoid casting away const in apw_compare_blockinfo.

Discussion: https://postgr.es/m/7773.1525288909@sss.pgh.pa.us

Unify indnkeys/indnatts/indnkeyatts usage  for all version of query to get
index information, remove indnkeys column  from query as unused.

Author: Marina Polyakova
Noticed by: Peter Eisentraut

Andrew Gierth pointed out that commit 1c72ec6f would yield the wrong
answer on big-endian ARM systems, because the data being CRC'd would be
different.  To fix that, and avoid the rather unsightly hard-wired
constant, simply compare the hardware and software implementations'
results.

While we're at it, also log the resulting decision at DEBUG1, and error
out if the hw and sw results unexpectedly differ.  Also, since this
file must compile for both frontend and backend, avoid incorrect
dependencies on backend-only headers.

In passing, add a comment to postmaster.c about when the CRC function
pointer will get initialized.

Thomas Munro, based on complaints from Andrew Gierth and Tom Lane

Discussion: https://postgr.es/m/HE1PR0801MB1323D171938EABC04FFE7FA9E3110@HE1PR0801MB1323.eurprd08.prod.outlook.com

Since the SPI stack has been moved from TopTransactionContext to
TopMemoryContext, setting _SPI_stack to NULL in AtEOXact_SPI() leaks
memory.  In fact, we don't need to do that anymore: We just leave the
allocated stack around for the next SPI use.

Also, refactor the SPI cleanup so that it is run both at transaction end
and when returning to the main loop on an exception.  The latter is
necessary when a procedure calls a COMMIT or ROLLBACK command that
itself causes an error.

Etsuro Fujita

Discussion: http://postgr.es/m/5AE99BA7.9060001@lab.ntt.co.jp

Instead of depending on glibc's getauxval() function, just try to execute
the CRC code, and trap SIGILL if that happens.

Thomas Munro

Discussion: https://postgr.es/m/HE1PR0801MB1323D171938EABC04FFE7FA9E3110@HE1PR0801MB1323.eurprd08.prod.outlook.com

As in e348e7ae for jsonb/plperl, prevent
putting a NaN into a jsonb numeric field.

Tests for this had been removed in
6278a2a2, but in case they are ever
resurrected: This would change the output of the test1nan() function to
an error.

Perl's XSUB.h header defines macros to replace libc functions.  Our header
port_win32.h does something similar earlier, so XSUB.h causes compiler
warnings about macro redefinition.  Undefine our macros before including
XSUB.h.

Thomas Munro

Discussion: https://postgr.es/m/CAEepm%3D3%3DTDYEXUEcHpEx%2BTwc31wo7PA0oBAiNt6sWmq93MW02A%40mail.gmail.com

Failure to use DatumGetFoo/FooGetDatum macros correctly, or at all,
causes some warnings about sign conversion.  This is just cosmetic
at the moment but in principle it's a type violation, so clean up
the instances I could find.

autoprewarm.c and sharedfileset.c contained code that unportably
assumed that pid_t is the same size as int.  We've variously dealt
with this by casting pid_t to int or to unsigned long for printing
purposes; I went with the latter.

Fix uninitialized-variable warning in RestoreGUCState.  This is
a live bug in some sense, but of no great significance given that
nobody is very likely to care what "line number" is associated with
a GUC that hasn't got a source file recorded.

statext_dependencies_load and statext_ndistinct_load were not up to snuff,
in addition to being randomly different from each other.  In detail:

* Deserialize the fetched bytea value before releasing the syscache
entry, not after.  This mistake causes visible regression test failures
when running with -DCATCACHE_FORCE_RELEASE.  Since it's not exposed by
-DCLOBBER_CACHE_ALWAYS, I think there may be no production hazard here
at present, but it's at least a latent bug.

* Use DatumGetByteaPP not DatumGetByteaP to save a detoasting cycle
for short stats values; the deserialize function has to be, and is,
prepared for short-header values since its other caller uses PP.

* Use a test-and-elog for null stats values in both functions, rather
than a test-and-elog in one case and an Assert in the other.  Perhaps
Asserts would be sufficient in both cases, but I don't see a good
argument for them being different.

* Minor cosmetic changes to make these functions more visibly alike.

Backpatch to v10 where this code came in.

Amit Langote, minor additional hacking by me

Discussion: https://postgr.es/m/1349aabb-3a1f-6675-9fc0-65e2ce7491dd@lab.ntt.co.jp

Amit Langote

Support for version-0 calling convention was removed in PostgreSQL v10.
Change the SPI example to use version 1 convention, so that it actually
works.

Author: John Naylor
Discussion: https://www.postgresql.org/message-id/CAJVSVGVydmhLBdm80Rw3G8Oq5TnA7eCxUv065yoZfNfLbF1tzA@mail.gmail.com

There were three related issues:

* BufFileAppend() incorrectly reset the seek position on the 'source' file.
  As a result, if you had called BufFileRead() on the file before calling
  BufFileAppend(), it got confused, and subsequent calls would read/write
  at wrong position.

* BufFileSize() did not work with files opened with BufFileOpenShared().

* FileGetSize() only worked on temporary files.

To fix, change the way BufFileSize() works so that it works on shared
files. Remove FileGetSize() altogether, as it's no longer needed. Remove
buffilesize from TapeShare struct, as the leader process can simply call
BufFileSize() to get the tape's size, there's no need to pass it through
shared memory anymore.

Discussion: https://www.postgresql.org/message-id/CAH2-WznEDYe_NZXxmnOfsoV54oFkTdMy7YLE2NPBLuttO96vTQ@mail.gmail.com

Commit 41c912ca caused MSVC to complain that not all control paths
return a value.

Thomas Munro

Discussion: https://postgr.es/m/CAEepm%3D3%3DTDYEXUEcHpEx%2BTwc31wo7PA0oBAiNt6sWmq93MW02A%40mail.gmail.com

For some reason it was previously defined as 0, which is silly.  The only
effect was to disable use of <stdbool.h>, which commit b2328bf6 intended
to make possible.

Thomas Munro

Discussion: https://postgr.es/m/CAEepm%3D3%3DTDYEXUEcHpEx%2BTwc31wo7PA0oBAiNt6sWmq93MW02A%40mail.gmail.com

Tom's earlier commit in 41c912ca didn't update a few cases that
are only encountered with the non-standard --with-llvm config
flag. Additionally there's also one case that appears to be a
deficiency in gcc's (up to trunk as of a few days ago) detection of
"fallthrough" comments - changing the placement slightly fixes that.

Author: Andres Freund
Discussion: https://postgr.es/m/20180502003239.wfnqu7ekz7j7imm4@alap3.anarazel.de

Don't overflow the result type of constant expressions.  Don't negate
unsigned types.  Define HAVE_STDBOOL_H for Visual C++ 2013 and later.

Thomas Munro
Reviewed-By: Michael Paquier and Tom Lane

Discussion: https://postgr.es/m/CAEepm%3D3%3DTDYEXUEcHpEx%2BTwc31wo7PA0oBAiNt6sWmq93MW02A%40mail.gmail.com

Recent gcc can warn about switch-case fall throughs that are not
explicitly labeled as intentional.  This seems like a good thing,
so clean up the warnings exposed thereby by labeling all such
cases with comments that gcc will recognize.

In files that already had one or more suitable comments, I generally
matched the existing style of those.  Otherwise I went with
/* FALLTHROUGH */, which is one of the spellings approved at the
more-restrictive-than-default level -Wimplicit-fallthrough=4.
(At the default level you can also spell it /* FALL ?THRU */,
and it's not picky about case.  What you can't do is include
additional text in the same comment, so some existing comments
containing versions of this aren't good enough.)

Testing with gcc 8.0.1 (Fedora 28's current version), I found that
I also had to put explicit "break"s after elog(ERROR) or ereport(ERROR);
apparently, for this purpose gcc doesn't recognize that those don't
return.  That seems like possibly a gcc bug, but it's fine because
in most places we did that anyway; so this amounts to a visit from the
style police.

Discussion: https://postgr.es/m/15083.1525207729@sss.pgh.pa.us

Previously a tuple that has been moved to a different partition (see
f16241be), set the block number on the old tuple to an invalid
value to indicate that fact. But the tuple offset was left
untouched. That turned out to trigger a wal_consistency_checking
failure as reported by Peter Geoghegan, as the offset wasn't
always overwritten during WAL replay.

Heikki observed that we're wasting valuable data by not putting
information also in the offset. Thus set that to
MovedPartitionsOffsetNumber when a tuple indicates it has moved.

We continue to set the block number to MovedPartitionsBlockNumber, as
that seems more likely to cause problems for code not updated to know
about moved tuples.

As t_ctid's offset number is now always set, this refinement also
fixes the wal_consistency_checking issue.

This technically is a minor disk format break, with previously created
moved tuples not being recognized anymore. But since there not even
has been a beta release since f16241be...

Reported-By: Peter Geoghegan
Author: Heikki Linnakangas, Amul Sul
Discussion: https://postgr.es/m/CAH2-Wzm9ty+1BX7-GMNJ=xPRg67oJTVeDNdA9LSyJJtMgRiCMA@mail.gmail.com

Without these fixes, changes to the inserted tuple made by remote
triggers are ignored when building local RETURNING tuples.

In the core code, call ExecInitRoutingInfo at a later point from
within ExecInitPartitionInfo so that the FDW callback gets invoked
after the returning list has been built.  But move CheckValidResultRel
out of ExecInitRoutingInfo so that it can happen at an earlier stage.

In postgres_fdw, refactor assorted deparsing functions to work with
the RTE rather than the PlannerInfo, which saves us having to
construct a fake PlannerInfo in cases where we don't have a real one.
Then, we can pass down a constructed RTE that yields the correct
deparse result when no real one exists.  Unfortunately, this
necessitates a hack that understands how the core code manages RT
indexes for update tuple routing, which is ugly, but we don't have a
better idea right now.

Original report, analysis, and patch by Etsuro Fujita.  Heavily
refactored by me.  Then worked over some more by Amit Langote.

Discussion: http://postgr.es/m/5AD4882B.10002@lab.ntt.co.jp

It turns out that old Perl versions (before about 5.10) don't have any
very reliable way to generate Inf or NaN numeric values.  Getting around
that would require way more work than is really justified to test the
code involved, so let's just drop these new test cases.

Discussion: https://postgr.es/m/28585.1525131438@sss.pgh.pa.us

Revert commits 23078689, 73042b8d, ce07aff4, f7df8043, 6ba0cc4b,
eb16011f, 68e7e973, 63ca350e.  We still have a problem here, but
somebody who's actually a Windows developer will need to spend time
on it.

Discussion: https://postgr.es/m/25495.1524517820@sss.pgh.pa.us

Back branches still are SGML.

David Rowley, Amit Langote

The previous coding here didn't actually produce Inf or NaN double
values in Perl versions 5.8.x.  Adopt a suggestion from stackoverflow.

Discussion: https://postgr.es/m/28585.1525131438@sss.pgh.pa.us

Revert the map/unmap dance I tried in commit 73042b8d; that helps
not at all.

Instead, speculate that the unwanted allocation is being done on
another thread, and thus timing variations explain the apparent
unpredictability.  Temporarily add a 1-second sleep before the
VirtualFree call, in hopes that any such other threads will
quiesce and not jog our elbow.

This is obviously not a desirable long-term fix, but as a means of
investigation it seems useful.

Discussion: https://postgr.es/m/25495.1524517820@sss.pgh.pa.us

The idea here is to get Windows' userspace infrastructure to allocate
whatever space it needs for MapViewOfFileEx() before we release the
locked-down space that we want to map the shared memory block into.

This is a fairly brute-force attempt, and would likely (for example)
fail with large shared memory on 32-bit Windows.  We could perhaps
ameliorate that by mapping only part of the shared memory block in
this way, but for the moment I just want to see if this approach
will fix dory's problem.

Discussion: https://postgr.es/m/25495.1524517820@sss.pgh.pa.us

Rather than elog'ing immediately, push the map data into a preallocated
StringInfo.  Perhaps this will prevent some of the mid-operation
allocations that are evidently happening now.

Discussion: https://postgr.es/m/25495.1524517820@sss.pgh.pa.us

The code previously undefined isnan because of a compiler warning on
MinGW.  Since we now need to use isnan, we can't do that anymore.

We might need a different solution if the compiler warning is too
annoying.

So by default, they don't output anything if everything is well.

Discussion: https://www.postgresql.org/message-id/867f8a1a-6cf0-d835-78d8-0844e4936241%402ndquadrant.com

When due to publication configuration, a TRUNCATE change ends up with
zero tables to be published, don't send the message out, just skip it.
It's not wrong, but obviously useless overhead.