Skip to content

P
Postgres FD Implementation
Switch branch/tag
  1. 26 Oct, 2022 1 commit
    • Michael Paquier's avatar
      Fix ordering issue with WAL operations in GIN fast insert path · 5a30d43f
      Michael Paquier authored 
      Contrary to what is documented in src/backend/access/transam/README,
ginHeapTupleFastInsert() had a few ordering issues with the way it does
its WAL operations when inserting items in its fast path.

First, when using a separate list, XLogBeginInsert() was being always
called before START_CRIT_SECTION(), and in this case a second thing was
wrong when merging lists, as an exclusive lock was taken on the tail
page *before* calling XLogBeginInsert().  Finally, when inserting items
into a tail page, the order of XLogBeginInsert() and
START_CRIT_SECTION() was reversed.  This commit addresses all these
issues by moving the calls of XLogBeginInsert() after all the pages
logged are locked and pinned, within a critical section.

This has been applied first only on HEAD as of 56b6625, but as per
discussion with Tom Lane and Álvaro Herrera, a backpatch is preferred to
keep all the branches consistent and to respect the transam's README
where we can.

Author:  Matthias van de Meent, Zhang Mingli
Discussion: https://postgr.es/m/CAEze2WhL8uLMqynnnCu1LAPwxD5RKEo0nHV+eXGg_N6ELU88HQ@mail.gmail.com
Backpatch-through: 10
      5a30d43f
  3. 21 Oct, 2022 3 commits
  5. 20 Oct, 2022 2 commits
  7. 19 Oct, 2022 1 commit
  9. 17 Oct, 2022 4 commits
  11. 16 Oct, 2022 3 commits
    • Tom Lane's avatar
      Fix EXPLAIN of SEARCH BREADTH FIRST with a constant initial value. · 8122160f
      Tom Lane authored 
      If the non-recursive term of a SEARCH BREADTH FIRST recursive
query has only constants in its target list, the planner will
fold the starting RowExpr added by rewrite into a simple Const
of type RECORD.  The executor doesn't have any problem with
that --- but EXPLAIN VERBOSE will encounter the Const as the
ultimate source of truth about what the field names of the
SET column are, and it didn't know what to do with that.
Fortunately, we can pull the identifying typmod out of the
Const, in much the same way that record_out would.

For reasons that remain a bit obscure to me, this only fails
with SEARCH BREADTH FIRST, not SEARCH DEPTH FIRST or CYCLE.
But I added regression test cases for both of those options
too, just to make sure we don't break it in future.

Per bug #17644 from Matthijs van der Vleuten.  Back-patch
to v14 where these constructs were added.

Discussion: https://postgr.es/m/17644-3bd1f3036d6d7a16@postgresql.org
      8122160f
    • Tom Lane's avatar
      Rename parser token REF to REF_P to avoid a symbol conflict. · 18e60712
      Tom Lane authored 
      In the latest version of Apple's macOS SDK, <sys/socket.h>
fails to compile if "REF" is #define'd as something.
Apple may or may not agree that this is a bug, and even if
they do accept the bug report I filed, they probably won't
fix it very quickly.  In the meantime, our back branches will all
fail to compile gram.y.  v15 and HEAD currently escape the problem
thanks to the refactoring done in 98e93a1fc, but that's purely
accidental.  Moreover, since that patch removed a widely-visible
inclusion of <netdb.h>, back-patching it seems too likely to break
third-party code.

Instead, change the token's code name to REF_P, following our usual
convention for naming parser tokens that are likely to have symbol
conflicts.  The effects of that should be localized to the grammar
and immediately surrounding files, so it seems like a safer answer.

Per project policy that we want to keep recently-out-of-support
branches buildable on modern systems, back-patch all the way to 9.2.

Discussion: https://postgr.es/m/1803927.1665938411@sss.pgh.pa.us
      18e60712
    • Tom Lane's avatar
      Use libc's snprintf, not sprintf, for special cases in snprintf.c. · 6fa431d8
      Tom Lane authored 
      snprintf.c has always fallen back on libc's *printf implementation
when printing pointers (%p) and floats.  When this code originated,
we were still supporting some platforms that lacked native snprintf,
so we used sprintf for that.  That's not actually unsafe in our usage,
but nonetheless builds on macOS are starting to complain about sprintf
being unconditionally deprecated; and I wouldn't be surprised if other
platforms follow suit.  There seems little reason to believe that any
platform supporting C99 wouldn't have standards-compliant snprintf,
so let's just use that instead to suppress such warnings.

Back-patch to v12, which is where we started to require C99.  It's
also where we started to use our snprintf.c everywhere, so this
wouldn't be enough to suppress the warning in older branches anyway
--- that is, in older branches these aren't necessarily all our
usages of libc's sprintf.  It is enough in v12+ because any
deprecation annotation attached to libc's sprintf won't apply to
pg_sprintf.  (Whether all our usages of pg_sprintf are adequately
safe is not a matter I intend to address here, but perhaps it could
do with some review.)

Per report from Andres Freund and local testing.

Discussion: https://postgr.es/m/20221015211955.q4cwbsfkyk3c4ty3@awork3.anarazel.de
      6fa431d8
  13. 14 Oct, 2022 1 commit
  15. 13 Oct, 2022 1 commit
  17. 12 Oct, 2022 1 commit
  19. 11 Oct, 2022 3 commits
    • Tom Lane's avatar
      Harden pmsignal.c against clobbered shared memory. · b10546ec
      Tom Lane authored 
      The postmaster is not supposed to do anything that depends
fundamentally on shared memory contents, because that creates
the risk that a backend crash that trashes shared memory will
take the postmaster down with it, preventing automatic recovery.
In commit 969d7cd4 I lost sight of this principle and coded
AssignPostmasterChildSlot() in such a way that it could fail
or even crash if the shared PMSignalState structure became
corrupted.  Remarkably, we've not seen field reports of such
crashes; but I managed to induce one while testing the recent
changes around palloc chunk headers.

To fix, make a semi-duplicative state array inside the postmaster
so that we need consult only local state while choosing a "child
slot" for a new backend.  Ensure that other postmaster-executed
routines in pmsignal.c don't have critical dependencies on the
shared state, either.  Corruption of PMSignalState might now
lead ReleasePostmasterChildSlot() to conclude that backend X
failed, when actually backend Y was the one that trashed things.
But that doesn't matter, because we'll force a cluster-wide reset
regardless.

Back-patch to all supported branches, since this is an old bug.

Discussion: https://postgr.es/m/3436789.1665187055@sss.pgh.pa.us
      b10546ec
    • Tom Lane's avatar
      Yet further fixes for multi-row VALUES lists for updatable views. · 3162bd95
      Tom Lane authored 
      DEFAULT markers appearing in an INSERT on an updatable view
could be mis-processed if they were in a multi-row VALUES clause.
This would lead to strange errors such as "cache lookup failed
for type NNNN", or in older branches even to crashes.

The cause is that commit 41531e42 tried to re-use rewriteValuesRTE()
to remove any SetToDefault nodes (that hadn't previously been replaced
by the view's own default values) appearing in "product" queries,
that is DO ALSO queries.  That's fundamentally wrong because the
DO ALSO queries might not even be INSERTs; and even if they are,
their targetlists don't necessarily match the view's column list,
so that almost all the logic in rewriteValuesRTE() is inapplicable.

What we want is a narrow focus on replacing any such nodes with NULL
constants.  (That is, in this context we are interpreting the defaults
as being strictly those of the view itself; and we already replaced
any that aren't NULL.)  We could add still more !force_nulls tests
to further lobotomize rewriteValuesRTE(); but it seems cleaner to
split out this case to a new function, restoring rewriteValuesRTE()
to the charter it had before.

Per bug #17633 from jiye_sw.  Patch by me, but thanks to
Richard Guo and Japin Li for initial investigation.
Back-patch to all supported branches, as the previous fix was.

Discussion: https://postgr.es/m/17633-98cc85e1fa91e905@postgresql.org
      3162bd95
    • Alvaro Herrera's avatar
      Ensure all perl test modules are installed · 4f6d1cfd
      Alvaro Herrera authored 
      PostgreSQL::Test::Cluster and ::Utils were not being installed.  This is
very hard to notice, as it only seems to affect external modules that
want to run tests from 15 back in earlier versions.  Oversight in
b235d41d.

This applies only to branches 14 and back, because 15 had already been
made correct in commit b3b4d8e68ae8.

Discussion: https://postgr.es/m/20221010093415.poplkyn7pjeiv2y7@alvherre.pgsql
      4f6d1cfd
  21. 07 Oct, 2022 1 commit
    • Alvaro Herrera's avatar
      Fix self-referencing foreign keys with partitioned tables · 483d2693
      Alvaro Herrera authored 
      There are a number of bugs in this area.  Two of them are fixed here,
namely:
1. get_relation_idx_constraint_oid does not restrict the type of
   constraint that's returned, so with sufficient bad luck it can
   return the OID of a foreign key constraint.  This has the effect that
   a primary key in a partition can end up as a child of a foreign key,
   which makes no sense (it needs to be the child of the equivalent
   primary key.)
   Change the API contract so that only index-backed constraints are
   returned, mimicking get_constraint_index().

2. Both CloneFkReferenced and CloneFkReferencing clone a
   self-referencing foreign key, so the partition ends up with
   a duplicate foreign key.  Change the former function to ignore such
   constraints.

Add some tests to verify that things are better now.  (However, these
new tests show some additional misbehavior that will be fixed later --
namely that there's a constraint marked NOT VALID.)

Backpatch to 12, where these constraints are possible at all.

Author: Jehan-Guillaume de Rorthais <jgdr@dalibo.com>
Discussion: https://postgr.es/m/20220603154232.1715b14c@karst
      483d2693
  23. 30 Sep, 2022 2 commits
    • Tom Lane's avatar
      Avoid improbable PANIC during heap_update, redux. · b93d7e68
      Tom Lane authored 
      Commit 34f581c3 intended to ensure that RelationGetBufferForTuple
would acquire a visibility-map page pin in case the otherBuffer's
all-visible bit had become set since we last had lock on that page.
But I missed a case: when we're extending the relation, VM concerns
were dealt with only in the relatively-less-likely case that we
fail to conditionally lock the otherBuffer.  I think I'd believed
that we couldn't need to worry about it if the conditional lock
succeeds, which is true for the target buffer; but the otherBuffer
was unlocked for awhile so its bit might be set anyway.  So we need
to do the GetVisibilityMapPins dance, and then also recheck the
page's free space, in both cases.

Per report from Jaime Casanova.  Back-patch to v12 as the previous
patch was (although there's still no evidence that the bug is
reachable pre-v14).

Discussion: https://postgr.es/m/E1lWLjP-00006Y-Ml@gemulon.postgresql.org
      b93d7e68
    • Daniel Gustafsson's avatar
      doc: Fix PQsslAttribute docs for compression · 064e1c87
      Daniel Gustafsson authored 
      The compression parameter to PQsslAttribute has never returned the
compression method used, it has always returned "on" or "off since
it was added in commit 91fa7b47. Backpatch through v10.
Reviewed-by: default avatarTom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/B9EC60EC-F665-47E8-A221-398C76E382C9@yesql.se
Backpatch-through: v10
      064e1c87
  25. 29 Sep, 2022 1 commit
  27. 28 Sep, 2022 3 commits
  29. 25 Sep, 2022 2 commits
    • Tom Lane's avatar
      Fix tupdesc lifespan bug with AfterTriggersTableData.storeslot. · 99237646
      Tom Lane authored 
      Commit 25936fd4 adjusted things so that the "storeslot" we use
for remapping trigger tuples would have adequate lifespan, but it
neglected to consider the lifespan of the tuple descriptor that
the slot depends on.  It turns out that in at least some cases, the
tupdesc we are passing is a refcounted tupdesc, and the refcount for
the slot's reference can get assigned to a resource owner having
different lifespan than the slot does.  That leads to an error like
"tupdesc reference 0x7fdef236a1b8 is not owned by resource owner
SubTransaction".  Worse, because of a second oversight in the same
commit, we'd try to free the same tupdesc refcount again while
cleaning up after that error, leading to recursive errors and an
"ERRORDATA_STACK_SIZE exceeded" PANIC.

To fix the initial problem, let's just make a non-refcounted copy
of the tupdesc we're supposed to use.  That seems likely to guard
against additional problems, since there's no strong reason for
this code to assume that what it's given is a refcounted tupdesc;
in which case there's an independent hazard of the tupdesc having
shorter lifespan than the slot does.  (I didn't bother trying to
free said copy, since it should go away anyway when the (sub)
transaction context is cleaned up.)

The other issue can be fixed by making the code added to
AfterTriggerFreeQuery work like the rest of that function, ie be
sure that it doesn't try to free the same slot twice in the event
of recursive error cleanup.

While here, also clean up minor stylistic issues in the test case
added by 25936fd4: don't use "create or replace function", as any
name collision within the tests is likely to have ill effects
that that won't mask; and don't use function names as generic as
trigger_function1, especially if you're not going to drop them
at the end of the test stanza.

Per bug #17607 from Thomas Mc Kay.  Back-patch to v12, as the
previous fix was.

Discussion: https://postgr.es/m/17607-bd8ccc81226f7f80@postgresql.org
      99237646
    • Alvaro Herrera's avatar
      Add missing source files to pg_waldump/nls.mk · f2094c78
      Alvaro Herrera authored
      f2094c78
  31. 23 Sep, 2022 1 commit
  33. 22 Sep, 2022 3 commits
  35. 21 Sep, 2022 1 commit
    • Tom Lane's avatar
      Suppress more variable-set-but-not-used warnings from clang 15. · 88c947cb
      Tom Lane authored 
      Mop up assorted set-but-not-used warnings in the back branches.
This includes back-patching relevant fixes from commit 152c9f7b8
the rest of the way, but there are also several cases that did not
appear in HEAD.  Some of those we'd fixed in a retail way but not
back-patched, and others I think just got rewritten out of existence
during nearby refactoring.

While here, also back-patch b1980f6d (PL/Tcl: Fix compiler warnings
with Tcl 8.6) into 9.2, so that that branch compiles warning-free
with modern Tcl.

Per project policy, this is a candidate for back-patching into
out-of-support branches: it suppresses annoying compiler warnings
but changes no behavior.  Hence, back-patch all the way to 9.2.

Discussion: https://postgr.es/m/514615.1663615243@sss.pgh.pa.us
      88c947cb
  37. 20 Sep, 2022 4 commits
    • Tom Lane's avatar
      Disable -Wdeprecated-non-prototype in the back branches. · dcd7dbed
      Tom Lane authored 
      There doesn't seem to be any good ABI-preserving way to silence
clang 15's -Wdeprecated-non-prototype warnings about our tree-walk
APIs.  While we've fixed it properly in HEAD, the only way to not
see hundreds of these in the back branches is to disable the
warnings.  We're not going to do anything about them, so we might
as well disable them.

I noticed that we also get some of these warnings about fmgr.c's
support for V0 function call convention, in branches before v10
where we removed that.  That's another area we aren't going to
change, so turning off the warning seems fine for that too.

Per project policy, this is a candidate for back-patching into
out-of-support branches: it suppresses annoying compiler warnings
but changes no behavior.  Hence, back-patch all the way to 9.2.

Discussion: https://postgr.es/m/CA+hUKGKpHPDTv67Y+s6yiC8KH5OXeDg6a-twWo_xznKTcG0kSA@mail.gmail.com
      dcd7dbed
    • Tom Lane's avatar
      Suppress variable-set-but-not-used warnings from clang 15. · 2e124d85
      Tom Lane authored 
      clang 15+ will issue a set-but-not-used warning when the only
use of a variable is in autoincrements (e.g., "foo++;").
That's perfectly sensible, but it detects a few more cases that
we'd not noticed before.  Silence the warnings with our usual
methods, such as PG_USED_FOR_ASSERTS_ONLY, or in one case by
actually removing a useless variable.

One thing that we can't nicely get rid of is that with %pure-parser,
Bison emits "yynerrs" as a local variable that falls foul of this
warning.  To silence those, I inserted "(void) yynerrs;" in the
top-level productions of affected grammars.

Per recently-established project policy, this is a candidate
for back-patching into out-of-support branches: it suppresses
annoying compiler warnings but changes no behavior.  Hence,
back-patch to 9.5, which is as far as these patches go without
issues.  (A preliminary check shows that the prior branches
need some other set-but-not-used cleanups too, so I'll leave
them for another day.)

Discussion: https://postgr.es/m/514615.1663615243@sss.pgh.pa.us
      2e124d85
    • Michael Paquier's avatar
      doc: Fix parameter name for pg_create_logical_replication_slot() · 382cc680
      Michael Paquier authored 
      The parameter controlling if two-phase transactions can be decoded was
named "two_phase" in the documentation while its procedure defines
"twophase".

Author: Florin Irion
Discussion: https://postgr.es/m/5eeabd10-1aff-ea61-f92d-9fa0d9a7e207@gmail.com
Backpatch-through: 14
      382cc680
    • Michael Paquier's avatar
      Fix incorrect variable types for origin IDs in decode.c · e68fc64f
      Michael Paquier authored 
      These variables used XLogRecPtr instead of RepOriginId.

Author: Masahiko Sawada
Discussion: https://postgr.es/m/CAD21AoBm-vNyBSXGp4bmJGvhr=S-EGc5q1dtV70cFTcJvLhC=Q@mail.gmail.com
Backpatch-through: 14
      e68fc64f
  39. 19 Sep, 2022 1 commit
    • Tom Lane's avatar
      Future-proof the recursion inside ExecShutdownNode(). · 7394c763
      Tom Lane authored 
      The API contract for planstate_tree_walker() callbacks is that they
take a PlanState pointer and a context pointer.  Somebody figured
they could save a couple lines of code by ignoring that, and passing
ExecShutdownNode itself as the walker even though it has but one
argument.  Somewhat remarkably, we've gotten away with that so far.
However, it seems clear that the upcoming C2x standard means to
forbid such cases, and compilers that actively break such code
likely won't be far behind.  So spend the extra few lines of code
to do it honestly with a separate walker function.

In HEAD, we might as well go further and remove ExecShutdownNode's
useless return value.  I left that as-is in back branches though,
to forestall complaints about ABI breakage.

Back-patch, with the thought that this might become of practical
importance before our stable branches are all out of service.
It doesn't seem to be fixing any live bug on any currently known
platform, however.

Discussion: https://postgr.es/m/208054.1663534665@sss.pgh.pa.us
      7394c763
  41. 17 Sep, 2022 1 commit