1. 08 Jul, 2020 8 commits
  2. 07 Jul, 2020 5 commits
    • Tom Lane's avatar
      Un-break pg_upgrade from pre-v12 servers. · 3f96af46
      Tom Lane authored
      I neglected to test this scenario while preparing commit f3faf35f,
      so of course it was broken, thanks to some very obscure and undocumented
      code in pg_dump.  Pre-v12 databases might have toast tables attached to
      partitioned tables, which we need to ignore since newer servers never
      create such useless toast tables.  There was a filter for this case in
      binary_upgrade_set_type_oids_by_rel_oid(), which appeared to just
      prevent the pg_type OID from being copied.  But actually it managed to
      prevent the toast table from being created at all --- or it did before
      I took out that logic.  But that was a fundamentally bizarre place to be
      making the test in the first place.  The place where the filter should
      have been, one would think, is binary_upgrade_set_pg_class_oids(), so
      add it there.
      
      While at it, reorganize binary_upgrade_set_pg_class_oids() so that it
      doesn't make a completely useless query when it knows it's being
      invoked for an index.  And correct a comment that mis-described the
      scenario where we need to force creation of a TOAST table.
      
      Per buildfarm.
      3f96af46
    • Tom Lane's avatar
      Don't create pg_type entries for sequences or toast tables. · f3faf35f
      Tom Lane authored
      Commit f7f70d5e left one inconsistency behind: we're still creating
      pg_type entries for the composite types of sequences and toast tables,
      but not arrays over those composites.  But there seems precious little
      reason to have named composite types for toast tables, and not much more
      to have them for sequences (especially given the thought that sequences
      may someday not be standalone relations at all).
      
      So, let's close that inconsistency by removing these composite types,
      rather than adding arrays for them.  This buys back a little bit of
      the initial pg_type bloat added by the previous patch, and could be
      a significant savings in a large database with many toast tables.
      
      Aside from a small logic rearrangement in heap_create_with_catalog,
      this patch mostly needs to clean up some places that were assuming that
      pg_class.reltype always has a valid value.  Those are really pre-existing
      bugs, given that it's documented otherwise; notably, the plpgsql changes
      fix code that gives "cache lookup failed for type 0" on indexes today.
      But none of these seem interesting enough to back-patch.
      
      Also, remove the pg_dump/pg_upgrade infrastructure for propagating
      a toast table's pg_type OID into the new database, since we no longer
      need that.
      
      Discussion: https://postgr.es/m/761F1389-C6A8-4C15-80CE-950C961F5341@gmail.com
      f3faf35f
    • Alvaro Herrera's avatar
      Morph pg_replication_slots.min_safe_lsn to safe_wal_size · a8aaa0c7
      Alvaro Herrera authored
      The previous definition of the column was almost universally disliked,
      so provide this updated definition which is more useful for monitoring
      purposes: a large positive value is good, while zero or a negative value
      means danger.  This should be operationally more convenient.
      
      Backpatch to 13, where the new column to pg_replication_slots (and the
      feature it represents) were added.
      
      Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
      Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
      Reported-by: default avatarFujii Masao <masao.fujii@oss.nttdata.com>
      Discussion: https://postgr.es/m/9ddfbf8c-2f67-904d-44ed-cf8bc5916228@oss.nttdata.com
      a8aaa0c7
    • Magnus Hagander's avatar
      Check ssl_in_use flag when reporting statistics · 6a5c750f
      Magnus Hagander authored
      Previously we checked that the ssl pointer was not null, but this puts a
      requirement on there being such a pointer which may not be true in
      future multi-ssl-library supporting times. This seems to have been an
      oversight in 9029f4b3, but hasn't really had any effect since we only
      have one library.
      
      Author: Daniel Gustafsson
      6a5c750f
    • Peter Eisentraut's avatar
      71ec58e7
  3. 06 Jul, 2020 9 commits
    • Peter Geoghegan's avatar
      Remove unnecessary PageIsEmpty() nbtree build check. · 28c16f49
      Peter Geoghegan authored
      nbtree index builds cannot write out an empty page.  That would mean
      that there was no way to create a pivot tuple pointing to the page one
      level up, since _bt_truncate() generates one based on page's firstright
      tuple.
      
      Replace the unnecessary PageIsEmpty() check with an assertion that
      checks that the page has space for at least two line pointers (the
      would-be high key line pointer, plus at least one valid "data item"
      tuple line pointer).
      
      The PageIsEmpty() check was added by commit 5d9f146c over 20 years ago.
      It looks like it has always been unnecessary.
      28c16f49
    • Tom Lane's avatar
      Create composite array types for initdb-created relations. · f7f70d5e
      Tom Lane authored
      When we invented arrays of composite types (commit bc8036fc),
      we excluded system catalogs, basically just on the grounds of not
      wanting to bloat pg_type.  However, it's definitely inconsistent that
      catalogs' composite types can't be put into arrays when others can.
      Another problem is that the exclusion is done by checking
      IsUnderPostmaster in heap_create_with_catalog, which means that
      
      (1) If a user tries to create a table in single-user mode, it doesn't
      get an array type.  That's bad in itself, plus it breaks pg_upgrade.
      
      (2) If someone drops and recreates a system view or information_schema
      view (as we occasionally recommend doing), it will now have an array
      type where it did not before, making for still more inconsistency.
      
      So this is all pretty messy.  Let's just get rid of the inconsistency
      and decree that system-created relations should have array types if
      similar user-created ones would, i.e. it only depends on the relkind.
      As of HEAD, that means that the initial contents of pg_type grow from
      411 rows to 605, which is a lot of growth percentage-wise, but it's
      still quite a small catalog compared to others.
      
      Wenjing Zeng, reviewed by Shawn Wang, further hacking by me
      
      Discussion: https://postgr.es/m/761F1389-C6A8-4C15-80CE-950C961F5341@gmail.com
      f7f70d5e
    • Peter Eisentraut's avatar
      Fix typo in test · bae9e8a5
      Peter Eisentraut authored
      The test was supposed to error but didn't.  Apparently, a copy and
      paste and string replace mistake from a nearby similar test.
      bae9e8a5
    • Fujii Masao's avatar
      doc: Add note about possible performance overhead by enabling track_planning. · 321fa6a4
      Fujii Masao authored
      Enabling pg_stat_statements.track_plaanning may incur a noticeable
      performance penalty, especially when a fewer kinds of queries are executed
      on many concurrent connections. This commit documents this note.
      
      Back-patch to v13 where pg_stat_statements.track_plaanning was added.
      
      Suggested-by: Pavel Stehule
      Author: Fujii Masao
      Reviewed-by: Pavel Stehule
      Discussion: https://postgr.es/m/CAFj8pRC9Jxa8r5i0TNBWLb8mzuaYzEoLq3QOvip0jVpHPOLbVA@mail.gmail.com
      321fa6a4
    • Michael Paquier's avatar
      Refactor routines for name lookups of procedures and operators · aa384348
      Michael Paquier authored
      This introduces a new set of extended routines for procedure and
      operator name lookups, with a flag bitmask argument that can modify the
      result.  The following options are available:
      - Force schema qualification, ignoring search_path.  This is similar to
      the existing option for format_{operator|procedure}_qualified().
      - Force NULL as result instead of a numeric OID for an undefined
      object.  This option is new.
      
      This is a refactoring similar to 1185c782, that will be used for a future
      patch to improve the SQL functions providing information using object
      addresses for undefined objects.
      
      Author: Michael Paquier
      Reviewed-by: Aleksander Alekseev, Dmitry Dolgov, Daniel Gustafsson,
      Álvaro Herrera
      Discussion: https://postgr.es/m/CAB7nPqSZxrSmdHK-rny7z8mi=EAFXJ5J-0RbzDw6aus=wB5azQ@mail.gmail.com
      aa384348
    • Amit Kapila's avatar
      Remove extra whitespace in comments atop ReorderBufferCheckMemoryLimit. · 04c7f414
      Amit Kapila authored
      Backpatch-through: 13, where it was introduced
      04c7f414
    • Michael Paquier's avatar
      Add new flag to format_type_extended() to get NULL for undefined type · 1185c782
      Michael Paquier authored
      If a type scanned is undefined, type format routines have two behaviors
      depending on if FORMAT_TYPE_ALLOW_INVALID is used by the caller or not:
      - Issue a cache lookup error
      - Return an undefined type name "???", "???[]" or "-"
      
      The current interface is not really helpful for callers willing to
      format properly a type name, but still make sure that the type is
      defined as there could be types matching the strings generated when
      looking for an undefined type, even if that should not be a problem in
      practice.  In order to counter that, add a new flag called
      FORMAT_TYPE_INVALID_AS_NULL that returns a NULL result instead of "???
      or "-" which does not generate an error.  This flag will be used in a
      follow-up patch improving the set of SQL functions showing information
      for object addresses when it comes to undefined objects.
      
      Author: Michael Paquier
      Reviewed-by: Aleksander Alekseev, Dmitry Dolgov, Daniel Gustafsson,
      Álvaro Herrera
      Discussion: https://postgr.es/m/CAB7nPqSZxrSmdHK-rny7z8mi=EAFXJ5J-0RbzDw6aus=wB5azQ@mail.gmail.com
      1185c782
    • Amit Kapila's avatar
      Remove unused function parameter in end_parallel_vacuum. · 231ef5b9
      Amit Kapila authored
      Author: Vignesh C
      Reviewed-by: Sawada Masahiko
      Backpatch-through: 13, where it was introduced
      Discussion: https://postgr.es/m/CALDaNm3Ppt71NafGY5mk3V2i3Q+mm93pVibDq-0NpW7WU67Jcg@mail.gmail.com
      231ef5b9
    • Michael Paquier's avatar
      Improve perl script in MSVC to build binaries · 404b912c
      Michael Paquier authored
      This commit includes two improvements for build.pl in src/tools/msvc/:
      - Fix two warnings related to $ARGV[0] being uninitialized, something
      that happens when calling build.pl (or build.bat) without arguments to
      compile all the components with a release quality.
      - If calling the script with more than two arguments, exit immediately
      and show a new help output.  build.pl was not failing in this case
      before this commit, and the extra arguments were just ignored, but the
      new behavior helps in understanding how to run the script without
      looking at the documentation for the Windows builds.
      
      Reported-by: Ranier Vilela
      Author: Michael Paquier
      Reviewed-by: Juan José Santamaría Flecha, David Zhang
      Discussion: https://postgr.es/m/CAEudQAo38dfR_0Ugt2OHy4mq-6Hz93XPSBAGEUV67RhKdgp8Zg@mail.gmail.com
      404b912c
  4. 05 Jul, 2020 4 commits
  5. 04 Jul, 2020 2 commits
    • Joe Conway's avatar
      Fix "ignoring return value" complaints from commit 96d1f423 · 1d05627f
      Joe Conway authored
      The cfbot and some BF animals are complaining about the previous
      read_binary_file commit because of ignoring return value of ‘fread’.
      So let's make everyone happy by testing the return value even though
      not strictly needed.
      
      Reported by Justin Pryzby, and suggested patch by Tom Lane. Backpatched
      to v11 same as the previous commit.
      
      Reported-By: Justin Pryzby
      Reviewed-By: Tom Lane
      Discussion: https://postgr.es/m/flat/969b8d82-5bb2-5fa8-4eb1-f0e685c5d736%40joeconway.com
      Backpatch-through: 11
      1d05627f
    • Joe Conway's avatar
      Read until EOF vice stat-reported size in read_binary_file · 96d1f423
      Joe Conway authored
      read_binary_file(), used by SQL functions pg_read_file() and friends,
      uses stat to determine file length to read, when not passed an explicit
      length as an argument. This is problematic, for example, if the file
      being read is a virtual file with a stat-reported length of zero.
      Arrange to read until EOF, or StringInfo data string lenth limit, is
      reached instead.
      
      Original complaint and patch by me, with significant review, corrections,
      advice, and code optimizations by Tom Lane. Backpatched to v11. Prior to
      that only paths relative to the data and log dirs were allowed for files,
      so no "zero length" files were reachable anyway.
      
      Reviewed-By: Tom Lane
      Discussion: https://postgr.es/m/flat/969b8d82-5bb2-5fa8-4eb1-f0e685c5d736%40joeconway.com
      Backpatch-through: 11
      96d1f423
  6. 03 Jul, 2020 6 commits
    • Tom Lane's avatar
      Clamp total-tuples estimates for foreign tables to ensure planner sanity. · ca5e93f7
      Tom Lane authored
      After running GetForeignRelSize for a foreign table, adjust rel->tuples
      to be at least as large as rel->rows.  This prevents bizarre behavior
      in estimate_num_groups() and perhaps other places, especially in the
      scenario where rel->tuples is zero because pg_class.reltuples is
      (suggesting that ANALYZE has never been run for the table).  As things
      stood, we'd end up estimating one group out of any GROUP BY on such a
      table, whereas the default group-count estimate is more likely to result
      in a sane plan.
      
      Also, clarify in the documentation that GetForeignRelSize has the option
      to override the rel->tuples value if it has a better idea of what to use
      than what is in pg_class.reltuples.
      
      Per report from Jeff Janes.  Back-patch to all supported branches.
      
      Patch by me; thanks to Etsuro Fujita for review
      
      Discussion: https://postgr.es/m/CAMkU=1xNo9cnan+Npxgz0eK7394xmjmKg-QEm8wYG9P5-CcaqQ@mail.gmail.com
      ca5e93f7
    • Tom Lane's avatar
      Fix temporary tablespaces for shared filesets some more. · f7b5988c
      Tom Lane authored
      Commit ecd9e9f0 fixed the problem in the wrong place, causing unwanted
      side-effects on the behavior of GetNextTempTableSpace().  Instead,
      let's make SharedFileSetInit() responsible for subbing in the value
      of MyDatabaseTableSpace when the default tablespace is called for.
      
      The convention about what is in the tempTableSpaces[] array is
      evidently insufficiently documented, so try to improve that.
      
      It also looks like SharedFileSetInit() is doing the wrong thing in the
      case where temp_tablespaces is empty.  It was hard-wiring use of the
      pg_default tablespace, but it seems like using MyDatabaseTableSpace
      is more consistent with what happens for other temp files.
      
      Back-patch the reversion of PrepareTempTablespaces()'s behavior to
      9.5, as ecd9e9f0 was.  The changes in SharedFileSetInit() go back
      to v11 where that was introduced.  (Note there is net zero code change
      before v11 from these two patch sets, so nothing to release-note.)
      
      Magnus Hagander and Tom Lane
      
      Discussion: https://postgr.es/m/CABUevExg5YEsOvqMxrjoNvb3ApVyH+9jggWGKwTDFyFCVWczGQ@mail.gmail.com
      f7b5988c
    • Tom Lane's avatar
      Inline plpgsql's exec_stmt() into exec_stmts(). · 1f902d49
      Tom Lane authored
      This saves one level of C function call per plpgsql statement executed,
      and permits a tiny additional optimization of not saving and restoring
      estate->err_stmt for each statement in a block.  The net effect seems
      nearly un-measurable on x86_64, but there's a clear win on aarch64,
      amounting to two or three percent in a loop over a few simple plpgsql
      statements.
      
      To do this, we have to get rid of the other existing call sites for
      exec_stmt().  Replace them with exec_toplevel_block(), which is just
      defined to do what exec_stmts() does, but for a single
      PLpgSQL_stmt_block statement.  Hard-wiring the expectation of which
      statement type applies here allows us to skip the dispatch switch,
      making this not much uglier than the previous factorization.
      
      Amit Khandekar, tweaked a bit by me
      
      Discussion: https://postgr.es/m/CAJ3gD9eBNrmUD7WBBLG8ohaZ485H9y+4eihQTgr+K8Lhka3vcQ@mail.gmail.com
      1f902d49
    • Magnus Hagander's avatar
      Fix temporary tablespaces for shared filesets · ecd9e9f0
      Magnus Hagander authored
      A likely copy/paste error in 98e8b480 from  back in 2004 would
      cause temp tablespace to be reset to InvalidOid if temp_tablespaces
      was set to the same value as the primary tablespace in the database.
      This would cause shared filesets (such as for parallel hash joins)
      to ignore them, putting the temporary files in the default tablespace
      instead of the configured one. The bug is in the old code, but it
      appears to have been exposed only once we had shared filesets.
      
      Reviewed-By: Daniel Gustafsson
      Discussion: https://postgr.es/m/CABUevExg5YEsOvqMxrjoNvb3ApVyH+9jggWGKwTDFyFCVWczGQ@mail.gmail.com
      Backpatch-through: 9.5
      ecd9e9f0
    • Fujii Masao's avatar
      doc: Correct description of restart_lsn in pg_replication_slots · 8f9b6d40
      Fujii Masao authored
      Previously the document explained that restart_lsn indicates the LSN of
      oldest WAL won't be automatically removed during checkpoints. But
      since v13 this was no longer true thanks to max_slot_wal_keep_size.
      
      Back-patch to v13 where max_slot_wal_keep_size was added.
      
      Author: Fujii Masao
      Discussion: https://postgr.es/m/6497f1e9-3148-c5da-7e49-b2fddad9a42f@oss.nttdata.com
      8f9b6d40
    • Fujii Masao's avatar
      Change default of pg_stat_statements.track_planning to off. · d1763ea8
      Fujii Masao authored
      Since v13 pg_stat_statements is allowed to track the planning time of
      statements when track_planning option is enabled. Its default was on.
      
      But this feature could cause more terrible spinlock contentions in
      pg_stat_statements. As a result of this, Robins Tharakan reported that
      v13 beta1 showed ~45% performance drop at high DB connection counts
      (when compared with v12.3) during fully-cached SELECT-only test using
      pgbench.
      
      To avoid this performance regression by the default setting,
      this commit changes default of pg_stat_statements.track_planning to off.
      
      Back-patch to v13 where pg_stat_statements.track_planning was introduced.
      
      Reported-by: Robins Tharakan
      Author: Fujii Masao
      Reviewed-by: Julien Rouhaud
      Discussion: https://postgr.es/m/2895b53b033c47ccb22972b589050dd9@EX13D05UWC001.ant.amazon.com
      d1763ea8
  7. 02 Jul, 2020 3 commits
    • Peter Geoghegan's avatar
      Initialize work_mem using current guc.c default. · 947456a8
      Peter Geoghegan authored
      Do the same for the maintenance_work_mem global variable.
      
      Oversight in commit 848ae330, which increased the previous defaults
      for work_mem and maintenance_work_mem by 4X.
      947456a8
    • Peter Geoghegan's avatar
      nbtree: Rename _bt_search() variables. · e25d462a
      Peter Geoghegan authored
      Make some of the variable names in _bt_search() consistent with
      corresponding variables within _bt_getstackbuf().  This naming scheme is
      clearer because the variable names always express a relationship between
      the currently locked buffer/page and some other page.
      e25d462a
    • Michael Paquier's avatar
      Move description of libpqwalreceiver hooks out of the replication's README · 641dd167
      Michael Paquier authored
      src/backend/replication/README includes since 32bc08b1 a basic
      description of the WAL receiver hooks available in walreceiver.h for a
      module like libpqwalreceiver, but the README has never been updated to
      reflect changes done to the hooks, so the contents of the README have
      rotten with the time.  This commit moves the description from the README
      to walreceiver.h, where it will be hard to miss that a description
      update or addition is needed depending on the modifications done to the
      hooks.
      
      Each hook now includes a description of what it does in walreceiver.h,
      and the replication's README mentions walreceiver.h.
      
      Thanks also to Amit Kapila for the discussion.
      
      Author: Michael Paquier
      Reviewed-by: Peter Eisentraut
      Discussion: https://postgr.es/m/20200502024606.GA471944@paquier.xyz
      641dd167
  8. 01 Jul, 2020 3 commits