When compiling postgres using gcc -O3, there are false-positive
warnings about the now initialized variables. Silence them.

Author: Peter Eisentraut, Andres Freund
Discussion: https://postgr.es/m/15fb2350-b8b8-e188-278f-0b34fdee5210@2ndquadrant.com

If we don't do this, the rewind fails if the server wasn't cleanly shut
down, which seems unhelpful serving no purpose.

Also provide a new option --no-ensure-shutdown to suppress this
behavior, for alleged advanced usage that prefers to avoid the crash
recovery.

Authors: Paul Guo, Jimmy Yih, Ashwin Agrawal
Reviewed-by: Álvaro Herrera
Discussion: https://postgr.es/m/CAEET0ZEffUkXc48pg2iqARQgGRYDiiVxDu+yYek_bTwJF+q=Uw@mail.gmail.com

For some reason at least gcc-9 warns about the fallthrough, even
though it otherwise recognizes that elog(ERROR, ...) doesn't return.

Author: Andres Freund

We've noted certain EXPLAIN queries on these tables occasionally showing
unexpected plan choices.  This seems to happen because VACUUM sometimes
fails to update relpages/reltuples for one of these single-page tables,
due to bgwriter or checkpointer holding a pin on the lone page at just
the wrong time.  To ensure those values get set, insert explicit ANALYZE
operations on these tables after we finish populating them.  This
doesn't seem to affect any other test cases, so it's a usable fix.

Back-patch to v12.  In principle the issue exists further back, but
we have not seen it before v12, so I won't risk back-patching further.

Discussion: https://postgr.es/m/24480.1569518042@sss.pgh.pa.us

This removes the last of the temporary debugging queries added to the
regression tests by commit f03a9ca4.  We've pretty much convinced
ourselves that the plan instability we were seeing is due to VACUUM
sometimes failing to update relpages/reltuples for a single-page table,
due to bgwriter or checkpointer holding a pin on that page at just the
wrong time.  I'll push a workaround for that separately.

Discussion: https://postgr.es/m/CA+hUKG+0CxrKRWRMf5ymN3gm+BECHna2B-q1w8onKBep4HasUw@mail.gmail.com

The markup for optional parameters was neither correct nor consistent.
In passing, fix a spelling mistake.

Per report from Alex Macy.  Some of these mistakes are old, so
back-patch as appropriate.

Discussion: https://postgr.es/m/156953522258.1204.12736099368284950578@wrigleys.postgresql.org

The documentation states that no target settings will be used when
standby.signal is present, but this is not quite the case since
recovery_target_timeline is a valid recovery target for a standby.

Update the documentation with this exception.

Author: David Steele <david@pgmasters.net>
Discussion: https://www.postgresql.org/message-id/flat/e445616d-023e-a268-8aa1-67b8b335340c%40pgmasters.net

Author: Justin Pryzby
Reviewed-by: Tatsuro Yamada
Discussion: https://postgr.es/m/20190927022051.GC24334@telsasoft.com
Backpatch-through: 12

The test name and the following test cases suggest the index created
should be hash index, but it forgot to add 'using hash' in the test case.
This in itself won't improve code coverage as there were some other tests
which were covering the corresponding code.  However, it is better if the
added tests serve their actual purpose.

Reported-by: Paul A Jungwirth
Author: Paul A Jungwirth
Reviewed-by: Mahendra Singh
Backpatch-through: 9.4
Discussion: https://postgr.es/m/CA+renyV=Us-5XfMC25bNp-uWSj39XgHHmGE9Rh2cQKMegSj52g@mail.gmail.com

The code was enforcing AccessExclusiveLock for all custom relation
options, which is incorrect as the APIs allow a custom lock level to be
set.

While on it, fix a couple of inconsistencies in the tests and the README
of dummy_index_am.

Oversights in commit 773df883.

Discussion: https://postgr.es/m/20190925234152.GA2115@paquier.xyz

Author: Liudmila Mantrova <l.mantrova@postgrespro.ru>
Reported-by: Jeff Janes <jeff.janes@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAMkU%3D1wP-SO4KpiLxHJuPezTJCmK%3DJqefLXrr3eXFO7Qku%2BtMg%40mail.gmail.com

Update the note about why not to use // comments, even though it's now
technically supported.

The note about variable declarations was dropped here because it's
addressed more properly later in the chapter.

Discussion: https://www.postgresql.org/message-id/flat/156924954640.1117.6309209869705522549%40wrigleys.postgresql.org

Put the description of the SQL/JSON path language after the
description of the general JSON functions and operators, instead of
before.

Discussion: https://www.postgresql.org/message-id/16968.1569189812@sss.pgh.pa.us

This has been introduced by 709d003f, that has moved readSegNo, readOff
and readPageTLI into a new structure called WALOpenSegment initialized
separately.

Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/20190926.110809.248342687.horikyota.ntt@gmail.com

Discussion: https://postgr.es/m/CAPpHfdteFKW6MLpXM4md99m55YAuXs0n9_P2wiTq_EmG09doUA%40mail.gmail.com

LIKE INCLUDING DEFAULTS tried to copy the attrdef expression without
copying the state of the attgenerated column.  This is in fact wrong,
because GENERATED and DEFAULT expressions are not the same kind of animal;
one can contain Vars and the other not.  We *must* copy attgenerated
when we're copying the attrdef expression.  Rearrange the if-tests
so that the expression is copied only when the correct one of
INCLUDING DEFAULTS and INCLUDING GENERATED has been specified.

Per private report from Manuel Rigger.

Tom Lane and Peter Eisentraut

This commit implements jsonpath .datetime() method as it's specified in
SQL/JSON standard.  There are no-argument and single-argument versions of
this method.  No-argument version selects first of ISO datetime formats
matching input string.  Single-argument version accepts template string as
its argument.

Additionally to .datetime() method itself this commit also implements
comparison ability of resulting date and time values.  There is some difficulty
because exising jsonb_path_*() functions are immutable, while comparison of
timezoned and non-timezoned types involves current timezone.  At first, current
timezone could be changes in session.  Moreover, timezones themselves are not
immutable and could be updated.  This is why we let existing immutable functions
throw errors on such non-immutable comparison.  In the same time this commit
provides jsonb_path_*_tz() functions which are stable and support operations
involving timezones.  As new functions are added to the system catalog,
catversion is bumped.

Support of .datetime() method was the only blocker prevents T832 from being
marked as supported.  sql_features.txt is updated correspondingly.

Extracted from original patch by Nikita Glukhov, Teodor Sigaev, Oleg Bartunov.
Heavily revised by me.  Comments were adjusted by Liudmila Mantrova.

Discussion: https://postgr.es/m/fcc6fc6a-b497-f39a-923d-aa34d0c588e8%402ndQuadrant.com
Discussion: https://postgr.es/m/CAPpHfdsZgYEra_PeCLGNoXOWYx6iU-S3wF8aX0ObQUcZU%2B4XTw%40mail.gmail.com
Author: Alexander Korotkov, Nikita Glukhov, Teodor Sigaev, Oleg Bartunov, Liudmila Mantrova
Reviewed-by: Anastasia Lubennikova, Peter Eisentraut

SQL/JSON standard allows manipulation with datetime values.  So, it appears to
be convinient to allow datetime values to be represented in JsonbValue struct.
These datetime values are allowed for temporary representation only.  During
serialization datetime values are converted into strings.

SQL/JSON requires writing timestamps with timezone in the same timezone offset
as they were parsed.  This is why we allow storage of timezone offset in
JsonbValue struct.  For the same reason timezone offset argument is added to
JsonEncodeDateTime() function.

Extracted from original patch by Nikita Glukhov, Teodor Sigaev, Oleg Bartunov.
Revised by me.  Comments were adjusted by Liudmila Mantrova.

Discussion: https://postgr.es/m/fcc6fc6a-b497-f39a-923d-aa34d0c588e8%402ndQuadrant.com
Discussion: https://postgr.es/m/CAPpHfdsZgYEra_PeCLGNoXOWYx6iU-S3wF8aX0ObQUcZU%2B4XTw%40mail.gmail.com
Author: Nikita Glukhov, Teodor Sigaev, Oleg Bartunov, Alexander Korotkov, Liudmila Mantrova
Reviewed-by: Anastasia Lubennikova, Peter Eisentraut

Add support of error suppression in some date and time manipulation functions
as it's required for jsonpath .datetime() method support.  This commit doesn't
use PG_TRY()/PG_CATCH() in order to implement that.  Instead, it provides
internal versions of date and time functions used, which support error
suppression.

Discussion: https://postgr.es/m/CAPpHfdsZgYEra_PeCLGNoXOWYx6iU-S3wF8aX0ObQUcZU%2B4XTw%40mail.gmail.com
Author: Alexander Korotkov, Nikita Glukhov
Reviewed-by: Anastasia Lubennikova, Peter Eisentraut

This commit adds parse_datetime() function, which implements datetime
parsing with extended features demanded by upcoming jsonpath .datetime()
method:

 * Dynamic type identification based on template string,
 * Support for standard-conforming 'strict' mode,
 * Timezone offset is returned as separate value.

Extracted from original patch by Nikita Glukhov, Teodor Sigaev, Oleg Bartunov.
Revised by me.

Discussion: https://postgr.es/m/fcc6fc6a-b497-f39a-923d-aa34d0c588e8%402ndQuadrant.com
Discussion: https://postgr.es/m/CAPpHfdsZgYEra_PeCLGNoXOWYx6iU-S3wF8aX0ObQUcZU%2B4XTw%40mail.gmail.com
Author: Nikita Glukhov, Teodor Sigaev, Oleg Bartunov, Alexander Korotkov
Reviewed-by: Anastasia Lubennikova, Peter Eisentraut

SQL Standard 2016 defines rules for handling separators in datetime template
strings, which are different to to_date()/to_timestamp() rules.  Standard
allows only small set of separators and requires strict matching for them.

Standard applies to jsonpath .datetime() method and CAST (... FORMAT ...) SQL
clause.  We're not going to change handling of separators in existing
to_date()/to_timestamp() functions, because their current behavior is familiar
for users.  Standard behavior now available by special flag, which will be used
in upcoming .datetime() jsonpath method.

Discussion: https://postgr.es/m/CAPpHfdsZgYEra_PeCLGNoXOWYx6iU-S3wF8aX0ObQUcZU%2B4XTw%40mail.gmail.com
Author: Alexander Korotkov

Forgot to add the file in the previous commit.

All our current in core relation options of type string (not many,
admittedly) behave in reality like enums.  But after seeing an
implementation for enum reloptions, it's clear that strings are messier,
so introduce the new reloption type.  Switch all string options to be
enums instead.

Fortunately we have a recently introduced test module for reloptions, so
we don't lose coverage of string reloptions, which may still be used by
third-party modules.

Authors: Nikolay Shaplov, Álvaro Herrera
Reviewed-by: Nikita Glukhov, Aleksandr Parfenov
Discussion: https://postgr.es/m/43332102.S2V5pIjXRx@x200m

... into a new file, fe_utils/recovery_gen.c.

This can later be used by pg_rewind.

Authors: Paul Guo, Jimmy Yih, Ashwin Agrawal.  A few tweaks by Álvaro Herrera
Reviewed-by: Michaël Paquier
Discussion: https://postgr.es/m/CAEET0ZEffUkXc48pg2iqARQgGRYDiiVxDu+yYek_bTwJF+q=Uw@mail.gmail.com

Append node has been removed in v12 when there would be only one subnode
under it.

Author: Amit Langote
Discussion: https://postgr.es/m/CA+HiwqHhS62w8zUFXF4NBjvMboCXYnD-jWoWp-tfo2aHvP3Gxg@mail.gmail.com
Backpatch-through: 12

Several buildfarm members (crake, loach and spurfowl) are complaining
about two queries looking up at pg_class.reloptions which trigger the
validation routines for string reloptions with default values.  This
commit limits the routines to be triggered only when building an index
with all custom options set in CREATE INDEX, which is sufficient for the
coverage.

Introduced by 640c1986.

This includes more tests dedicated to relation options, bringing the
coverage of this code close to 100%, and the module can be used for
other purposes, like a base template for an index AM implementation.

Author: Nikolay Sharplov, Michael Paquier
Reviewed-by: Álvaro Herrera, Dent John
Discussion: https://postgr.es/m/17071942.m9zZutALE6@x200m

Relation options can define a lock mode other than AccessExclusiveMode
since 47167b79, but modules defining custom relation options did not
really have a way to enforce that.  Correct that by extending the
current API set so as modules can define a custom lock mode.

Author: Michael Paquier
Reviewed-by: Kuntal Ghosh
Discussion: https://postgr.es/m/20190920013831.GD1844@paquier.xyz

In-core relation options can use a custom lock mode since 47167b79, that
has lowered the lock available for some autovacuum parameters.  However
it forgot to consider custom relation options.  This causes failures
with ALTER TABLE SET when changing a custom relation option, as its lock
is not defined.  The existing APIs to define a custom reloption does not
allow to define a custom lock mode, so enforce its initialization to
AccessExclusiveMode which should be safe enough in all cases.  An
upcoming patch will extend the existing APIs to allow a custom lock mode
to be defined.

The problem can be reproduced with bloom indexes, so add a test there.

Reported-by: Nikolay Sharplov
Analyzed-by: Thomas Munro, Michael Paquier
Author: Michael Paquier
Reviewed-by: Kuntal Ghosh
Discussion: https://postgr.es/m/20190920013831.GD1844@paquier.xyz
Backpatch-through: 9.6

Our item contains only so->numberOfNonNullOrderBys of distances.  Reflect that
in the loop upper bound.

Discussion: https://postgr.es/m/53536807-784c-e029-6e92-6da802ab8d60%40postgrespro.ru
Author: Nikita Glukhov
Backpatch-through: 12

The state-tracking of WAL reading in various places was pretty messy,
mostly because the ancient physical-replication WAL reading code wasn't
using the XLogReader abstraction.  This led to some untidy code.  Make
it prettier by creating two additional supporting structs,
WALSegmentContext and WALOpenSegment which keep track of WAL-reading
state.  This makes code cleaner, as well as supports more future
cleanup.

Author: Antonin Houska
Reviewed-by: Álvaro Herrera and (older versions) Robert Haas
Discussion: https://postgr.es/m/14984.1554998742@spoje.net

Fix an oversight in commit 7266d099: as it stood, the code failed
when a function-in-FROM returns composite and can be simplified
to a composite constant.

For the moment, just test for composite result and abandon pullup
if we see one.  To make it actually work, we'd have to decompose
the composite constant into per-column constants; which is surely
do-able, but I'm not convinced it's worth the code space.

Per report from Raúl Marín Rodríguez.

Discussion: https://postgr.es/m/CAM6_UM4isP+buRA5sWodO_MUEgutms-KDfnkwGmryc5DGj9XuQ@mail.gmail.com

When a relation is truncated, shared_buffers needs to be scanned
so that any buffers for the relation forks are invalidated in it.
Previously, shared_buffers was scanned for each relation forks, i.e.,
MAIN, FSM and VM, when VACUUM truncated off any empty pages
at the end of relation or TRUNCATE truncated the relation in place.
Since shared_buffers needed to be scanned multiple times,
it could take a long time to finish those commands especially
when shared_buffers was large.

This commit changes the logic so that shared_buffers is scanned only
one time for those three relation forks.

Author: Kirk Jamison
Reviewed-by: Masahiko Sawada, Thomas Munro, Alvaro Herrera, Takayuki Tsunakawa and Fujii Masao
Discussion: https://postgr.es/m/D09B13F772D2274BB348A310EE3027C64E2067@g01jpexmbkw24

This was working around a bug in ccache that was fixed in ccache
3.2.2 (released 2015-05-10).  (Users of older ccache versions can
continue to set CCACHE_DISABLE themselves.)

Discussion: https://www.postgresql.org/message-id/20190530191130.GA24528@alvherre.pgsql

This appears to have been an oversight in b8d7f053. As it's
effectively harmless, though confusing, only fix in master.

Author: Andres Freund

Allow clients to require channel binding to enhance security against
untrusted servers.

Author: Jeff Davis
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/227015d8417f2b4fef03f8966dbfa5cbcc4f44da.camel%40j-davis.com

The array <@ and @> operators do not worry about duplicates: if every
member of array X matches some element of array Y, then X is contained
in Y, even if several members of X get matched to the same Y member.
This was not explicitly stated in the docs though, so improve matters.

Discussion: https://postgr.es/m/156614120484.1310.310161642239149585@wrigleys.postgresql.org