1. 04 Aug, 2022 1 commit
  2. 03 Aug, 2022 2 commits
    • Tom Lane's avatar
      Fix incorrect tests for SRFs in relation_can_be_sorted_early(). · 445b9020
      Tom Lane authored
      Commit fac1b470 thought we could check for set-returning functions
      by testing only the top-level node in an expression tree.  This is
      wrong in itself, and to make matters worse it encouraged others
      to make the same mistake, by exporting tlist.c's special-purpose
      IS_SRF_CALL() as a widely-visible macro.  I can't find any evidence
      that anyone's taken the bait, but it was only a matter of time.
      
      Use expression_returns_set() instead, and stuff the IS_SRF_CALL()
      genie back in its bottle, this time with a warning label.  I also
      added a couple of cross-reference comments.
      
      After a fair amount of fooling around, I've despaired of making
      a robust test case that exposes the bug reliably, so no test case
      here.  (Note that the test case added by fac1b470 is itself
      broken, in that it doesn't notice if you remove the code change.
      The repro given by the bug submitter currently doesn't fail either
      in v15 or HEAD, though I suspect that may indicate an unrelated bug.)
      
      Per bug #17564 from Martijn van Oosterhout.  Back-patch to v13,
      as the faulty patch was.
      
      Discussion: https://postgr.es/m/17564-c7472c2f90ef2da3@postgresql.org
      445b9020
    • Tom Lane's avatar
      Reduce test runtime of src/test/modules/snapshot_too_old. · 8eaa4d0f
      Tom Lane authored
      The sto_using_cursor and sto_using_select tests were coded to exercise
      every permutation of their test steps, but AFAICS there is no value in
      exercising more than one.  This matters because each permutation costs
      about six seconds, thanks to the "pg_sleep(6)".  Perhaps we could
      reduce that, but the useless permutations seem worth getting rid of
      in any case.  (Note that sto_using_hash_index got it right already.)
      
      While here, clean up some other sloppiness such as an unused table.
      
      This doesn't make too much difference in interactive testing, since the
      wasted time is typically masked by parallelization with other tests.
      However, the buildfarm runs this as a serial step, which means we can
      expect to shave ~40 seconds from every buildfarm run.  That makes it
      worth back-patching.
      
      Discussion: https://postgr.es/m/2515192.1659454702@sss.pgh.pa.us
      8eaa4d0f
  3. 02 Aug, 2022 1 commit
    • Tom Lane's avatar
      Be more wary about 32-bit integer overflow in pg_stat_statements. · 17fd203b
      Tom Lane authored
      We've heard a couple of reports of people having trouble with
      multi-gigabyte-sized query-texts files.  It occurred to me that on
      32-bit platforms, there could be an issue with integer overflow
      of calculations associated with the total query text size.
      Address that with several changes:
      
      1. Limit pg_stat_statements.max to INT_MAX / 2 not INT_MAX.
      The hashtable code will bound it to that anyway unless "long"
      is 64 bits.  We still need overflow guards on its use, but
      this helps.
      
      2. Add a check to prevent extending the query-texts file to
      more than MaxAllocHugeSize.  If it got that big, qtext_load_file
      would certainly fail, so there's not much point in allowing it.
      Without this, we'd need to consider whether extent, query_offset,
      and related variables shouldn't be off_t not size_t.
      
      3. Adjust the comparisons in need_gc_qtexts() to be done in 64-bit
      arithmetic on all platforms.  It appears possible that under duress
      those multiplications could overflow 32 bits, yielding a false
      conclusion that we need to garbage-collect the texts file, which
      could lead to repeatedly garbage-collecting after every hash table
      insertion.
      
      Per report from Bruno da Silva.  I'm not convinced that these
      issues fully explain his problem; there may be some other bug that's
      contributing to the query-texts file becoming so large in the first
      place.  But it did get that big, so #2 is a reasonable defense,
      and #3 could explain the reported performance difficulties.
      
      (See also commit 8bbe4cbd, which addressed some related bugs.
      The second Discussion: link is the thread that led up to that.)
      
      This issue is old, and is primarily a problem for old platforms,
      so back-patch.
      
      Discussion: https://postgr.es/m/CAB+Nuk93fL1Q9eLOCotvLP07g7RAv4vbdrkm0cVQohDVMpAb9A@mail.gmail.com
      Discussion: https://postgr.es/m/5601D354.5000703@BlueTreble.com
      17fd203b
  4. 01 Aug, 2022 2 commits
  5. 31 Jul, 2022 1 commit
  6. 29 Jul, 2022 3 commits
  7. 28 Jul, 2022 2 commits
    • Alvaro Herrera's avatar
      Fix replay of create database records on standby · a3aacb7c
      Alvaro Herrera authored
      Crash recovery on standby may encounter missing directories
      when replaying database-creation WAL records.  Prior to this
      patch, the standby would fail to recover in such a case;
      however, the directories could be legitimately missing.
      Consider the following sequence of commands:
      
          CREATE DATABASE
          DROP DATABASE
          DROP TABLESPACE
      
      If, after replaying the last WAL record and removing the
      tablespace directory, the standby crashes and has to replay the
      create database record again, crash recovery must be able to continue.
      
      A fix for this problem was already attempted in 49d9cfc68bf4, but it
      was reverted because of design issues.  This new version is based
      on Robert Haas' proposal: any missing tablespaces are created
      during recovery before reaching consistency.  Tablespaces
      are created as real directories, and should be deleted
      by later replay.  CheckRecoveryConsistency ensures
      they have disappeared.
      
      The problems detected by this new code are reported as PANIC,
      except when allow_in_place_tablespaces is set to ON, in which
      case they are WARNING.  Apart from making tests possible, this
      gives users an escape hatch in case things don't go as planned.
      
      Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
      Author: Asim R Praveen <apraveen@pivotal.io>
      Author: Paul Guo <paulguo@gmail.com>
      Reviewed-by: Anastasia Lubennikova <lubennikovaav@gmail.com> (older versions)
      Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com> (older versions)
      Reviewed-by: default avatarMichaël Paquier <michael@paquier.xyz>
      Diagnosed-by: default avatarPaul Guo <paulguo@gmail.com>
      Discussion: https://postgr.es/m/CAEET0ZGx9AvioViLf7nbR_8tH9-=27DN5xWJ2P9-ROH16e4JUA@mail.gmail.com
      a3aacb7c
    • Thomas Munro's avatar
      Fix get_dirent_type() for symlinks on MinGW/MSYS. · 5ad478c9
      Thomas Munro authored
      On Windows with MSVC, get_dirent_type() was recently made to return
      DT_LNK for junction points by commit 9d3444dc, which fixed some
      defective dirent.c code.
      
      On Windows with Cygwin, get_dirent_type() already worked for symlinks,
      as it does on POSIX systems, because Cygwin has its own fake symlinks
      that behave like POSIX (on closer inspection, Cygwin's dirent has the
      BSD d_type extension but it's probably always DT_UNKNOWN, so we fall
      back to lstat(), which understands Cygwin symlinks with S_ISLNK()).
      
      On Windows with MinGW/MSYS, we need extra code, because the MinGW
      runtime has its own readdir() without d_type, and the lstat()-based
      fallback has no knowledge of our convention for treating junctions as
      symlinks.
      
      Back-patch to 14, where get_dirent_type() landed.
      Reported-by: default avatarAndrew Dunstan <andrew@dunslane.net>
      Discussion: https://postgr.es/m/b9ddf605-6b36-f90d-7c30-7b3e95c46276%40dunslane.net
      5ad478c9
  8. 27 Jul, 2022 1 commit
    • Alvaro Herrera's avatar
      Allow "in place" tablespaces. · 961cab0a
      Alvaro Herrera authored
      This is a backpatch to branches 10-14 of the following commits:
      
      7170f2159fb2 Allow "in place" tablespaces.
      c6f2f01611d4 Fix pg_basebackup with in-place tablespaces.
      f6f0db4d6240 Fix pg_tablespace_location() with in-place tablespaces
      7a7cd84893e0 doc: Remove mention to in-place tablespaces for pg_tablespace_location()
      5344723755bd Remove unnecessary Windows-specific basebackup code.
      
      In-place tablespaces were introduced as a testing helper mechanism, but
      they are going to be used for a bugfix in WAL replay to be backpatched
      to all stable branches.
      
      I (Álvaro) had to adjust some code to account for lack of
      get_dirent_type() in branches prior to 14.
      
      Author: Thomas Munro <thomas.munro@gmail.com>
      Author: Michaël Paquier <michael@paquier.xyz>
      Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
      Discussion: https://postgr.es/m/20220722081858.omhn2in5zt3g4nek@alvherre.pgsql
      961cab0a
  9. 26 Jul, 2022 1 commit
    • Tom Lane's avatar
      Force immediate commit after CREATE DATABASE etc in extended protocol. · 3e1297a6
      Tom Lane authored
      We have a few commands that "can't run in a transaction block",
      meaning that if they complete their processing but then we fail
      to COMMIT, we'll be left with inconsistent on-disk state.
      However, the existing defenses for this are only watertight for
      simple query protocol.  In extended protocol, we didn't commit
      until receiving a Sync message.  Since the client is allowed to
      issue another command instead of Sync, we're in trouble if that
      command fails or is an explicit ROLLBACK.  In any case, sitting
      in an inconsistent state while waiting for a client message
      that might not come seems pretty risky.
      
      This case wasn't reachable via libpq before we introduced pipeline
      mode, but it's always been an intended aspect of extended query
      protocol, and likely there are other clients that could reach it
      before.
      
      To fix, set a flag in PreventInTransactionBlock that tells
      exec_execute_message to force an immediate commit.  This seems
      to be the approach that does least damage to existing working
      cases while still preventing the undesirable outcomes.
      
      While here, add some documentation to protocol.sgml that explicitly
      says how to use pipelining.  That's latent in the existing docs if
      you know what to look for, but it's better to spell it out; and it
      provides a place to document this new behavior.
      
      Per bug #17434 from Yugo Nagata.  It's been wrong for ages,
      so back-patch to all supported branches.
      
      Discussion: https://postgr.es/m/17434-d9f7a064ce2a88a3@postgresql.org
      3e1297a6
  10. 25 Jul, 2022 1 commit
    • Heikki Linnakangas's avatar
      Fix ReadRecentBuffer for local buffers. · 3f968b94
      Heikki Linnakangas authored
      It incorrectly used GetBufferDescriptor instead of
      GetLocalBufferDescriptor, causing it to not find the correct buffer in
      most cases, and performing an out-of-bounds memory read in the corner
      case that temp_buffers > shared_buffers.
      
      It also bumped the usage-count on the buffer, even if it was
      previously pinned. That won't lead to crashes or incorrect results,
      but it's different from what the shared-buffer case does, and
      different from the usual code in LocalBufferAlloc. Fix that too, and
      make the code ordering match LocalBufferAlloc() more closely, so that
      it's easier to verify that it's doing the same thing.
      
      Currently, ReadRecentBuffer() is only used with non-temp relations, in
      WAL redo, so the broken code is currently dead code. However, it could
      be used by extensions.
      
      Backpatch-through: 14
      Discussion: https://www.postgresql.org/message-id/2d74b46f-27c9-fb31-7f99-327a87184cc0%40iki.fi
      Reviewed-by: Thomas Munro, Zhang Mingli, Richard Guo
      3f968b94
  11. 23 Jul, 2022 1 commit
    • Tom Lane's avatar
      Doc: improve documentation about random(). · 31d5354c
      Tom Lane authored
      We didn't explicitly say that random() uses a randomly-chosen seed
      if you haven't called setseed().  Do so.
      
      Also, remove ref/set.sgml's no-longer-accurate (and never very
      relevant) statement that the seed value is multiplied by 2^31-1.
      
      Back-patch to v12 where set.sgml's claim stopped being true.
      The claim that we use a source of random bits as seed was debatable
      before 4203842a, too, so v12 seems like a good place to stop.
      
      Per question from Carl Sopchak.
      
      Discussion: https://postgr.es/m/f37bb937-9d99-08f0-4de7-80c91a3cfc2e@sopchak.me
      31d5354c
  12. 22 Jul, 2022 2 commits
  13. 21 Jul, 2022 3 commits
    • Bruce Momjian's avatar
      doc: use wording "restore" instead of "reload" of dumps · e613466e
      Bruce Momjian authored
      Reported-by: axel.kluener@gmail.com
      
      Discussion: https://postgr.es/m/164736074430.660.3645615289283943146@wrigleys.postgresql.org
      
      Backpatch-through: 11
      e613466e
    • Bruce Momjian's avatar
      doc: clarify that auth. names are lower case and case-sensitive · 21640e98
      Bruce Momjian authored
      This is true even for acronyms that are usually upper case, like LDAP.
      
      Reported-by: Alvaro Herrera
      
      Discussion: https://postgr.es/m/202205141521.2nodjabmsour@alvherre.pgsql
      
      Backpatch-through: 10
      21640e98
    • Tom Lane's avatar
      Fix ruleutils issues with dropped cols in functions-returning-composite. · da9a28fd
      Tom Lane authored
      Due to lack of concern for the case in the dependency code, it's
      possible to drop a column of a composite type even though stored
      queries have references to the dropped column via functions-in-FROM
      that return the composite type.  There are "soft" references,
      namely FROM-clause aliases for such columns, and "hard" references,
      that is actual Vars referring to them.  The right fix for hard
      references is to add dependencies preventing the drop; something
      we've known for many years and not done (and this commit still doesn't
      address it).  A "soft" reference shouldn't prevent a drop though.
      We've been around on this before (cf. 9b35ddce, 2c4debbd), but
      nobody had noticed that the current behavior can result in dump/reload
      failures, because ruleutils.c can print more column aliases than the
      underlying composite type now has.  So we need to rejigger the
      column-alias-handling code to treat such columns as dropped and not
      print aliases for them.
      
      Rather than writing new code for this, I used expandRTE() which already
      knows how to figure out which function result columns are dropped.
      I'd initially thought maybe we could use expandRTE() in all cases, but
      that fails for EXPLAIN's purposes, because the planner strips a lot of
      RTE infrastructure that expandRTE() needs.  So this patch just uses it
      for unplanned function RTEs and otherwise does things the old way.
      
      If there is a hard reference (Var), then removing the column alias
      causes us to fail to print the Var, since there's no longer a name
      to print.  Failing seems less desirable than printing a made-up
      name, so I made it print "?dropped?column?" instead.
      
      Per report from Timo Stolz.  Back-patch to all supported branches.
      
      Discussion: https://postgr.es/m/5c91267e-3b6d-5795-189c-d15a55d61dbb@nullachtvierzehn.de
      da9a28fd
  14. 20 Jul, 2022 2 commits
    • Fujii Masao's avatar
      Fix assertion failure and segmentation fault in backup code. · be2e842c
      Fujii Masao authored
      When a non-exclusive backup is canceled, do_pg_abort_backup() is called
      and resets some variables set by pg_backup_start (pg_start_backup in v14
      or before). But previously it forgot to reset the session state indicating
      whether a non-exclusive backup is in progress or not in this session.
      
      This issue could cause an assertion failure when the session running
      BASE_BACKUP is terminated after it executed pg_backup_start and
      pg_backup_stop (pg_stop_backup in v14 or before). Also it could cause
      a segmentation fault when pg_backup_stop is called after BASE_BACKUP
      in the same session is canceled.
      
      This commit fixes the issue by making do_pg_abort_backup reset
      that session state.
      
      Back-patch to all supported branches.
      
      Author: Fujii Masao
      Reviewed-by: Kyotaro Horiguchi, Masahiko Sawada, Michael Paquier, Robert Haas
      Discussion: https://postgr.es/m/3374718f-9fbf-a950-6d66-d973e027f44c@oss.nttdata.com
      be2e842c
    • Fujii Masao's avatar
      Prevent BASE_BACKUP in the middle of another backup in the same session. · 2aedf25e
      Fujii Masao authored
      Multiple non-exclusive backups are able to be run conrrently in different
      sessions. But, in the same session, only one non-exclusive backup can be
      run at the same moment. If pg_backup_start (pg_start_backup in v14 or before)
      is called in the middle of another non-exclusive backup in the same session,
      an error is thrown.
      
      However, previously, in logical replication walsender mode, even if that
      walsender session had already called pg_backup_start and started
      a non-exclusive backup, it could execute BASE_BACKUP command and
      start another non-exclusive backup. Which caused subsequent pg_backup_stop
      to throw an error because BASE_BACKUP unexpectedly reset the session state
      marked by pg_backup_start.
      
      This commit prevents BASE_BACKUP command in the middle of another
      non-exclusive backup in the same session.
      
      Back-patch to all supported branches.
      
      Author: Fujii Masao
      Reviewed-by: Kyotaro Horiguchi, Masahiko Sawada, Michael Paquier, Robert Haas
      Discussion: https://postgr.es/m/3374718f-9fbf-a950-6d66-d973e027f44c@oss.nttdata.com
      2aedf25e
  15. 18 Jul, 2022 2 commits
  16. 17 Jul, 2022 2 commits
  17. 16 Jul, 2022 1 commit
  18. 15 Jul, 2022 2 commits
  19. 14 Jul, 2022 9 commits
  20. 13 Jul, 2022 1 commit
    • Alvaro Herrera's avatar
      Plug memory leak · 9e038d69
      Alvaro Herrera authored
      Commit 054325c5eeb3 created a memory leak in PQsendQueryInternal in case
      an error occurs while sending the message.  Repair.
      
      Backpatch to 14, like that commit.  Reported by Coverity.
      9e038d69