This change makes this module act more like most of our other low-level
resource management modules.  It's a caller error if something is not
explicitly closed by the end of a successful transaction, so issue
a WARNING about it.  This would not actually have caught the file leak
bug fixed in commit 231bcd08, because that was in a transaction-abort
path; but it still seems like a good, and pretty cheap, cross-check.

Discussion: https://postgr.es/m/152056616579.4966.583293218357089052@wrigleys.postgresql.org

This field was a bool at one point, but now it's an int.
Spotted by Hari Babu; trivial patch is by Ashutosh Bapat.

Discussion: https://postgr.es/m/CAJrrPGedKiFE2fqntSauUfhapCksOJzam+QtHfSgx86LhXLeOQ@mail.gmail.com

Use the same code we already applied in duplicate_oids and unused_oids
to let this script find Catalog.pm without help.  This removes the need
to supply a -I switch in most cases.

Also, mark the script executable, again to follow the precedent of
duplicate_oids and unused_oids.  Now you can just do
"./reformat_dat_file.pl pg_proc.dat"
if you want to reformat only one or a few .dat files rather than all.

It'd be possible to remove the -I switches in the Makefile's convenience
targets, but I chose to leave them: they don't hurt anything, and it's
possible that in weird VPATH situations they might be of value.

I (tgl) originally coded the special case for pg_proc.pronargs as
though it were a kind of default value.  It seems better though to
treat computable columns as an independent concern: this makes the
code clearer, and probably a bit faster too since we needn't do
work inside the per-column loop.

Improve related comments, as well, in the expectation that there
might be more cases like this in future.

John Naylor, some additional comment-hacking by me

Discussion: https://postgr.es/m/CAJVSVGW-D7OobzU=dybVT2JqZAx-4X1yvBJdavBmqQL05Q6CLw@mail.gmail.com

Apparently $(foreach ... $(call install_llvm_module,...)) doesn't work
too well without a blank line ending the install_llvm_module macro.
The previous coding hackishly dodged this problem with some parens,
but that's not really a good solution because make misunderstands
where the command boundaries are that way.

Discussion: https://postgr.es/m/20180428073935.GB1736@paquier.xyz

Don't put comments inside the macros, per complaint from Michael Paquier.

Quote target directory path with single quotes, not double; that seems
to be our project standard.  Not quoting it at all definitely isn't
per standard.

Remove excess slash in one instance of path.

Remove useless semicolon.

Discussion: https://postgr.es/m/20180428073935.GB1736@paquier.xyz

Identify pg_replication_origin as a shared catalog in catalogs.sgml,
using the same boilerplate wording used for most other shared catalogs
(and tweak another place where someone had randomly deviated from
that boilerplate).

Make an example in mmgr/README more consistent with surrounding text.

Update an obsolete cross-reference in a comment in storage/block.h.

Zhuo Ql

Discussion: https://postgr.es/m/44296255.1819230.1524889719001@mail.yahoo.com

Add some debug printouts focused on the idea that MapViewOfFileEx might
be rounding its virtual memory allocation up more than we expect (and,
in particular, more than VirtualAllocEx does).

Once we've seen what this reports in one of the failures on buildfarm
members dory or jacana, we might revert this ... or perhaps just
decrease the log level.

Discussion: https://postgr.es/m/25495.1524517820@sss.pgh.pa.us

The core PLs have been extension-ified for seven years now, and we can
reasonably hope that all out-of-core PLs have been too.  So adjust a few
places that were still recommending CREATE LANGUAGE as the user-level
way to install a PL.

Discussion: https://postgr.es/m/CA+TgmoaJTUDMSuSCg4k08Dv8vhbrJq9nP3ZfPbmysVz_616qxw@mail.gmail.com

Newer perltidy versions can just avoid writing backup files, so we don't
need the old dance of deleting them afterwards.  Supported since 20120619.

https://metacpan.org/source/SHANCOCK/Perl-Tidy-20120619/CHANGES#L61

Commit b75f467b removed the LogicalTapeAssignReadBufferSize() function,
but forgot to update this comment. The read buffer size is an argument to
LogicalTapeRewindForRead() now. Doesn't seem worth going into the details
in the file header comment, so remove the outdated sentence altogether.

Reported-by: Amit Langote

Discussion: https://postgr.es/m/d2e6674c-c5c6-fe89-1d0b-3534b9db0476@lab.ntt.co.jp

Discussion: https://postgr.es/m/15719.1523984266@sss.pgh.pa.us

Update typedefs.list from current buildfarm results.  Adjust pgindent's
typedef blacklist to block some more unfortunate typedef names that have
snuck in since last time.  Manually tweak a few places where I didn't
like the initial results of pgindent'ing.

In the wake of commit 5602265f, we were doing duplicate-OID detection
quite inefficiently, by invoking duplicate_oids which does all the same
parsing of catalog headers and .dat files as genbki.pl does.  That adds
under half a second on modern machines, but quite a bit more on slow
buildfarm critters, so it seems worth avoiding.  Let's just extend
genbki.pl a little so it can also detect duplicate OIDs, and remove
the duplicate_oids call from the build process.

(This also means that duplicate OID detection will happen during
Windows builds, which AFAICS it didn't before.)

This makes the use-case for duplicate_oids a bit dubious, but it's
possible that people will still want to run that check without doing
a whole build run, so let's keep that script.

In passing, move down genbki.pl's creation of its temp output files
so that it doesn't happen until after we've done parsing and validation
of the input.  This avoids leaving a lot of clutter around after a
failure.

John Naylor and Tom Lane

Discussion: https://postgr.es/m/37D774E4-FE1F-437E-B3D2-593F314B7505@postgrespro.ru

Previously, you had to cd into src/include/catalog before running either
of these scripts.  That's a bit tedious, so let's make the scripts do it
for you.

In passing, improve the initial comments in both scripts.  Also remove
unused_oids' code to complain about duplicate oids.  That was added in
yesterday's commit 5602265f, but on second thought we shouldn't be
randomly redefining the script's behavior that way.

John Naylor and Tom Lane

Discussion: https://postgr.es/m/37D774E4-FE1F-437E-B3D2-593F314B7505@postgrespro.ru

Reported-by: vodevsh@gmail.com

Discussion: https://postgr.es/m/152404286919.19366.7988650271505173666@wrigleys.postgresql.org

Backpatch-through: 9.3

The predecessor test boiled down to "PQserverVersion(NULL) >= 100000",
which is always false.  No release includes that, so it could not have
reintroduced CVE-2018-1058.  Back-patch to 9.4, like the addition of the
predecessor in commit 8d2814f2.

Discussion: https://postgr.es/m/20180422215551.GB2676194@rfd.leadboat.com

unused_oids was previously a shell script, which of course didn't work at
all on Windows.  Also, commit 372728b0 introduced some other portability
problems, as complained of by Stas Kelvich.  We can improve matters by
converting it to Perl.

While we're at it, let's future-proof both this script and duplicate_oids
to use Catalog.pm rather than having a bunch of ad-hoc logic for parsing
catalog headers and .dat files.  These scripts are thereby a bit slower,
which doesn't seem like a problem for typical manual use.  It is a little
annoying for buildfarm purposes, but we should be able to fix that case
by having genbki.pl make the check instead of parsing the headers twice.
(That's not done in this commit, though.)

Stas Kelvich, adjusted a bit by me

Discussion: https://postgr.es/m/37D774E4-FE1F-437E-B3D2-593F314B7505@postgrespro.ru

Instead of immediately constructing the string we need to emit into the
.BKI file, preserve the items we extracted from the header file in a hash.
This eases using the info for other purposes.

John Naylor (with cosmetic adjustments by me)

Discussion: https://postgr.es/m/37D774E4-FE1F-437E-B3D2-593F314B7505@postgrespro.ru

Commit 0927d2f4 didn't check that
consider_parallel was set for the target relation or account for
the possibility that required_outer might be non-empty.

To prevent future bugs of this ilk, add some assertions to
add_partial_path and do a bit of future-proofing of the code
recently added to recurse_set_operations.

Report by Andreas Seltenreich.  Patch by Jeevan Chalke.  Review
by Amit Kapila and by me.

Discussion: http://postgr.es/m/CAM2+6=U+9otsyF2fYB8x_2TBeHTR90itarqW=qAEjN-kHaC7kw@mail.gmail.com

Also use palloc0() for main amcheck state, and adjust a few comments.

Somehow I pushed old version of patch in commit
4eaf7eac, so commit the difference.

Peter Geoghegan

Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com

Per discussion, this is now the project's standard version.
Update the obsolete URL, and explain how to install a specific version.

Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com

When bt_index_parent_check() is called with the heapallindexed option,
allocate a second Bloom filter to fingerprint block numbers that appear
in the downlinks of internal pages.  Use Bloom filter probes when
walking the B-Tree to detect missing downlinks.  This can detect subtle
problems with page deletion/VACUUM, such as corruption caused by the bug
just fixed in commit 6db4b499.

The downlink Bloom filter is bound in size by work_mem.  Its optimal
size is typically far smaller than that of the regular heapallindexed
Bloom filter, especially when the index has high fan-out.

Author: Peter Geoghegan
Reviewer: Teodor Sigaev
Discussion: https://postgr.es/m/CAH2-WznUzY4fWTjm1tBB3JpVz8cCfz7k_qVp5BhuPyhivmWJFg@mail.gmail.com

Author: Michael Paquier

Remove the words "if not already done."  This obsolete wording
corresponds to an early development version of what became edd44738.

Author: Etsuro Fujita
Reviewed-by: Amit Langote
Discussion: https://postgr.es/m/5ADF117B.5030606@lab.ntt.co.jp

Instead of doing ExecInitExpr every time a Param needs to be evaluated
in run-time partition pruning, do it once during run-time pruning
set-up and cache the exprstate in PartitionPruneContext, saving a lot of
work.

Author: David Rowley
Reviewed-by: Amit Langote, Álvaro Herrera
Discussion: https://postgr.es/m/CAKJS1f8-x+q-90QAPDu_okhQBV4DPEtPz8CJ=m0940GyT4DA4w@mail.gmail.com

This controls both plan-time and execution-time new-style partition
pruning.  While finer-grain control is possible (maybe using an enum GUC
instead of boolean), there doesn't seem to be much need for that.

This new parameter controls partition pruning for all queries:
trivially, SELECT queries that affect partitioned tables are naturally
under its control since they are using the new technology.  However,
while UPDATE/DELETE queries do not use the new code, we make the new GUC
control their behavior also (stealing control from
constraint_exclusion), because it is more natural, and it leads to a
more natural transition to the future in which those queries will also
use the new pruning code.

Constraint exclusion still controls pruning for regular inheritance
situations (those not involving partitioned tables).

Author: David Rowley
Review: Amit Langote, Ashutosh Bapat, Justin Pryzby, David G. Johnston
Discussion: https://postgr.es/m/CAKJS1f_0HwsxJG9m+nzU+CizxSdGtfe6iF_ykPYBiYft302DCw@mail.gmail.com

Previously, you could partition by a boolean column as long as you
spelled the bound values as string literals, for instance FOR VALUES
IN ('t').  The trouble with this is that ruleutils.c printed that as
FOR VALUES IN (TRUE), which is reasonable syntax but wasn't accepted by
the grammar.  That results in dump-and-reload failures for such cases.

Apply a minimal fix that just causes TRUE and FALSE to be converted to
strings 'true' and 'false'.  This is pretty grotty, but it's too late for
a more principled fix in v11 (to say nothing of v10).  We should revisit
the whole issue of how partition bound values are parsed for v12.

Amit Langote

Discussion: https://postgr.es/m/e05c5162-1103-7e37-d1ab-6de3e0afaf70@lab.ntt.co.jp

This could result in some misbehavior in a cascading replication setup.

Set Emacs's perl-continued-statement-offset to match perltidy's
--continuation-indentation, which is 2 (not overridden in PostgreSQL's
profile) rather than the 4 that Emacs uses by default.

Lifetime of the input string is not right, so create a separate copy.

Author: Amit Langote
Discussion: https://postgr.es/m/a2773420-50d1-0a42-3396-fe42b0921134@lab.ntt.co.jp

The switch covers all values of the enum driver variable, so having a
default: clause is useless, even if it's only to do Assert(false).

5c067521 erroneously had coded bms_prev_member assuming that a bitmapword
would always hold 32 bits and started it's search on what it thought was the
highest 8-bits of the word.  This was not the case if bitmapwords were 64
bits.

In passing add a test to exercise this function a little. Previously there was
no coverage at all.

David Rowly

Author: Liudmila Mantrova

After introducing usage of t_tid of inner or page high key for storing
number of attributes of tuple, validation of tuple's ItemPointer with
ItemPointerIsValid becomes incorrect, it's need to validate only blocknumber of
ItemPointer. Missing this causes a incorrect page deletion, fix that. Test is
added.

BTW, current contrib/amcheck doesn't fail on index corrupted by this way.

Also introduce BTreeTupleGetTopParent/BTreeTupleSetTopParent macroses to improve
code readability and to avoid possible confusion with page high key: high key
is used to store top-parent link for branch to remove.

Bug found by Michael Paquier, but bug doesn't exist in previous versions because
t_tid was set to P_HIKEY.

Author: Teodor Sigaev
Reviewer: Peter Geoghegan
Discussion: https://www.postgresql.org/message-id/flat/20180419052436.GA16000%40paquier.xyz

Author: David Rowley
Discussion: https://www.postgresql.org/message-id/CAKJS1f8Ge2y0sDs6RQEJFH-vjb-bWhs86rCX4Fp4FZ+TmxtRkw@mail.gmail.com