Skip to content

P
Postgres FD Implementation
Switch branch/tag
  1. 26 Dec, 2020 6 commits
  3. 25 Dec, 2020 9 commits
  5. 24 Dec, 2020 5 commits
    • Bruce Momjian's avatar
      revert removal of hex_decode() from ecpg from commit c3826f83 · 558a6e8e
      Bruce Momjian authored 
      ecpglib on certain platforms can't handle the pg_log_fatal calls from
libraries.  This was reported by the buildfarm.  It needs a refactoring
and return value change if it is later removed.

Backpatch-through: master
      558a6e8e
    • Bruce Momjian's avatar
      move hex_decode() to /common so it can be called from frontend · c3826f83
      Bruce Momjian authored 
      This allows removal of a copy of hex_decode() from ecpg, and will be
used by the soon-to-be added pg_alterckey command.

Backpatch-through: master
      c3826f83
    • Tom Lane's avatar
      Fix race condition between shutdown and unstarted background workers. · 7519bd16
      Tom Lane authored 
      If a database shutdown (smart or fast) is commanded between the time
some process decides to request a new background worker and the time
that the postmaster can launch that worker, then nothing happens
because the postmaster won't launch any bgworkers once it's exited
PM_RUN state.  This is fine ... unless the requesting process is
waiting for that worker to finish (or even for it to start); in that
case the requestor is stuck, and only manual intervention will get us
to the point of being able to shut down.

To fix, cancel pending requests for workers when the postmaster sends
shutdown (SIGTERM) signals, and similarly cancel any new requests that
arrive after that point.  (We can optimize things slightly by only
doing the cancellation for workers that have waiters.)  To fit within
the existing bgworker APIs, the "cancel" is made to look like the
worker was started and immediately stopped, causing deregistration of
the bgworker entry.  Waiting processes would have to deal with
premature worker exit anyway, so this should introduce no bugs that
weren't there before.  We do have a side effect that registration
records for restartable bgworkers might disappear when theoretically
they should have remained in place; but since we're shutting down,
that shouldn't matter.

Back-patch to v10.  There might be value in putting this into 9.6
as well, but the management of bgworkers is a bit different there
(notably see 8ff51869) and I'm not convinced it's worth the effort
to validate the patch for that branch.

Discussion: https://postgr.es/m/661570.1608673226@sss.pgh.pa.us
      7519bd16
    • Tom Lane's avatar
      Improve client error messages for immediate-stop situations. · 7e784d1d
      Tom Lane authored 
      Up to now, if the DBA issued "pg_ctl stop -m immediate", the message
sent to clients was the same as for a crash-and-restart situation.
This is confusing, not least because the message claims that the
database will soon be up again, something we have no business
predicting.

Improve things so that we can generate distinct messages for the two
cases (and also recognize an ad-hoc SIGQUIT, should somebody try that).
To do that, add a field to pmsignal.c's shared memory data structure
that the postmaster sets just before broadcasting SIGQUIT to its
children.  No interlocking seems to be necessary; the intervening
signal-sending and signal-receipt should sufficiently serialize accesses
to the field.  Hence, this isn't any riskier than the existing usages
of pmsignal.c.

We might in future extend this idea to improve other
postmaster-to-children signal scenarios, although none of them
currently seem to be as badly overloaded as SIGQUIT.

Discussion: https://postgr.es/m/559291.1608587013@sss.pgh.pa.us
      7e784d1d
    • Michael Paquier's avatar
      Fix typos and grammar in docs and comments · 90fbf7c5
      Michael Paquier authored 
      This fixes several areas of the documentation and some comments in
matters of style, grammar, or even format.

Author: Justin Pryzby
Discussion: https://postgr.es/m/20201222041153.GK30237@telsasoft.com
      90fbf7c5
  7. 23 Dec, 2020 2 commits
    • Bruce Momjian's avatar
      dummy commit · 6ecf488d
      Bruce Momjian authored
      6ecf488d
    • Michael Paquier's avatar
      Fix portability issues with parsing of recovery_target_xid · 6db27037
      Michael Paquier authored 
      The parsing of this parameter has been using strtoul(), which is not
portable across platforms.  On most Unix platforms, unsigned long has a
size of 64 bits, while on Windows it is 32 bits.  It is common in
recovery scenarios to rely on the output of txid_current() or even the
newer pg_current_xact_id() to get a transaction ID for setting up
recovery_target_xid.  The value returned by those functions includes the
epoch in the computed result, which would cause strtoul() to fail where
unsigned long has a size of 32 bits once the epoch is incremented.

WAL records and 2PC data include only information about 32-bit XIDs and
it is not possible to have XIDs across more than one epoch, so
discarding the high bits from the transaction ID set has no impact on
recovery.  On the contrary, the use of strtoul() prevents a consistent
behavior across platforms depending on the size of unsigned long.

This commit changes the parsing of recovery_target_xid to use
pg_strtouint64() instead, available down to 9.6.  There is one TAP test
stressing recovery with recovery_target_xid, where a tweak based on
pg_reset{xlog,wal} is added to bump the XID epoch so as this change gets
tested, as per an idea from Alexander Lakhin.

Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/16780-107fd0c0385b1035@postgresql.org
Backpatch-through: 9.6
      6db27037
  9. 22 Dec, 2020 3 commits
    • Tom Lane's avatar
      Improve autoprewarm's handling of early-shutdown scenarios. · ff769831
      Tom Lane authored 
      Bad things happen if the DBA issues "pg_ctl stop -m fast" before
autoprewarm finishes loading its list of blocks to prewarm.
The current worker process successfully terminates early, but
(if this wasn't the last database with blocks to prewarm) the
leader process will just try to launch another worker for the
next database.  Since the postmaster is now in PM_WAIT_BACKENDS
state, it ignores the launch request, and the leader just sits
until it's killed manually.

This is mostly the fault of our half-baked design for launching
background workers, but a proper fix for that is likely to be
too invasive to be back-patchable.  To ameliorate the situation,
fix apw_load_buffers() to check whether SIGTERM has arrived
just before trying to launch another worker.  That leaves us with
only a very narrow window in each worker launch where SIGTERM
could occur between the launch request and successful worker start.

Another issue is that if the leader process does manage to exit,
it unconditionally rewrites autoprewarm.blocks with only the
blocks currently in shared buffers, thus forgetting any blocks
that we hadn't reached yet while prewarming.  This seems quite
unhelpful, since the next database start will then not have the
expected prewarming benefit.  Fix it to not modify the file if
we shut down before the initial load attempt is complete.

Per bug #16785 from John Thompson.  Back-patch to v11 where
the autoprewarm code was introduced.

Discussion: https://postgr.es/m/16785-c0207d8c67fb5f25@postgresql.org
      ff769831
    • Tom Lane's avatar
      Increase timeout in 021_row_visibility.pl. · 08dde1b3
      Tom Lane authored 
      Commit 7b28913b figured 30 seconds is long enough for anybody,
but in contexts like valgrind runs, it isn't necessarily.
      08dde1b3
    • Tomas Vondra's avatar
      Improve find_em_expr_usable_for_sorting_rel comment · 1ca2eb10
      Tomas Vondra authored 
      Clarify the relationship between find_em_expr_usable_for_sorting_rel and
prepare_sort_from_pathkeys, i.e. what restrictions need to be shared
between those two places.

Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs%3DhC0mSksZC%3DH5M8LSchj5e5OxpTAg%40mail.gmail.com
      1ca2eb10
  11. 21 Dec, 2020 7 commits
  13. 20 Dec, 2020 7 commits
  15. 19 Dec, 2020 1 commit
    • Tom Lane's avatar
      Remove now-useless ALWAYS_SUBDIRS entry in src/test/Makefile. · 08b01d4d
      Tom Lane authored 
      Commit 257836a7 added the "locale" subdirectory to SUBDIRS,
but neglected to remove it from ALWAYS_SUBDIRS.  This oversight
had no functional effect because the filter-out function would
remove it anyway.  Still, it's confusing to readers to list a
subdirectory in both places, especially because it makes the
associated comment into a partial lie.
      08b01d4d