1. 09 Mar, 2016 18 commits
    • Alvaro Herrera's avatar
      PostgresNode: add backup_fs_hot and backup_fs_cold · 28f6df3c
      Alvaro Herrera authored
      These simple methods rely on RecursiveCopy to create a filesystem-level
      backup of a server.  They aren't currently used anywhere yet,but will be
      useful for future tests.
      
      Author: Craig Ringer
      Reviewed-By: Michael Paquier, Salvador Fandino, Álvaro Herrera
      Commitfest-URL: https://commitfest.postgresql.org/9/569/
      28f6df3c
    • Alvaro Herrera's avatar
      Add filter capability to RecursiveCopy::copypath · a31aaec4
      Alvaro Herrera authored
      This allows skipping copying certain files and subdirectories in tests.
      This is useful in some circumstances such as copying a data directory;
      future tests want this feature.
      
      Also POD-ify the module.
      
      Authors: Craig Ringer, Pallavi Sontakke
      Reviewed-By: Álvaro Herrera
      a31aaec4
    • Tom Lane's avatar
      Fix incorrect handling of NULL index entries in indexed ROW() comparisons. · a298a1e0
      Tom Lane authored
      An index search using a row comparison such as ROW(a, b) > ROW('x', 'y')
      would stop upon reaching a NULL entry in the "b" column, ignoring the
      fact that there might be non-NULL "b" values associated with later values
      of "a".  This happens because _bt_mark_scankey_required() marks the
      subsidiary scankey for "b" as required, which is just wrong: it's for
      a column after the one with the first inequality key (namely "a"), and
      thus can't be considered a required match.
      
      This bit of brain fade dates back to the very beginnings of our support
      for indexed ROW() comparisons, in 2006.  Kind of astonishing that no one
      came across it before Glen Takahashi, in bug #14010.
      
      Back-patch to all supported versions.
      
      Note: the given test case doesn't actually fail in unpatched 9.1, evidently
      because the fix for bug #6278 (i.e., stopping at nulls in either scan
      direction) is required to make it fail.  I'm sure I could devise a case
      that fails in 9.1 as well, perhaps with something involving making a cursor
      back up; but it doesn't seem worth the trouble.
      a298a1e0
    • Robert Haas's avatar
      Re-pgindent vacuumlazy.c. · be060cbc
      Robert Haas authored
      be060cbc
    • Robert Haas's avatar
      pgbench: When -T is used, don't wait for transactions beyond end of run. · accf7616
      Robert Haas authored
      At low rates, this can lead to pgbench taking significantly longer to
      terminate than the user might expect.  Repair.
      
      Fabien Coelho, reviewed by Aleksander Alekseev, Álvaro Herrera, and me.
      accf7616
    • Alvaro Herrera's avatar
      pgcrypto: support changing S2K iteration count · 188f359d
      Alvaro Herrera authored
      pgcrypto already supports key-stretching during symmetric encryption,
      including the salted-and-iterated method; but the number of iterations
      was not configurable.  This commit implements a new s2k-count parameter
      to pgp_sym_encrypt() which permits selecting a larger number of
      iterations.
      
      Author: Jeff Janes
      188f359d
    • Robert Haas's avatar
      Add a generic command progress reporting facility. · b6fb6471
      Robert Haas authored
      Using this facility, any utility command can report the target relation
      upon which it is operating, if there is one, and up to 10 64-bit
      counters; the intent of this is that users should be able to figure out
      what a utility command is doing without having to resort to ugly hacks
      like attaching strace to a backend.
      
      As a demonstration, this adds very crude reporting to lazy vacuum; we
      just report the target relation and nothing else.  A forthcoming patch
      will make VACUUM report a bunch of additional data that will make this
      much more interesting.  But this gets the basic framework in place.
      
      Vinayak Pokale, Rahila Syed, Amit Langote, Robert Haas, reviewed by
      Kyotaro Horiguchi, Jim Nasby, Thom Brown, Masahiko Sawada, Fujii Masao,
      and Masanori Oyama.
      b6fb6471
    • Tom Lane's avatar
      Fix incorrect tlist generation in create_gather_plan(). · 8776c15c
      Tom Lane authored
      This function is written as though Gather doesn't project; but it does.
      Even if it did not project, though, we must use build_path_tlist to ensure
      that the output columns receive correct sortgroupref labeling.
      
      Per report from Amit Kapila.
      8776c15c
    • Robert Haas's avatar
      postgres_fdw: Consider foreign joining and foreign sorting together. · aa09cd24
      Robert Haas authored
      Commit ccd8f979 gave us the ability to
      request that the remote side sort the data, and, later, commit
      e4106b25 gave us the ability to
      request that the remote side perform the join for us rather than doing
      it locally.  But we could not do both things at the same time: a
      remote SQL query that had an ORDER BY clause would never be a join.
      This commit adds that capability.
      
      Ashutosh Bapat, reviewed by me.
      aa09cd24
    • Tom Lane's avatar
      Fix copy-and-pasteo in comment. · d31f20e2
      Tom Lane authored
      Wensheng Zhang
      d31f20e2
    • Tom Lane's avatar
      Improve handling of pathtargets in planner.c. · 51c0f63e
      Tom Lane authored
      Refactor so that the internal APIs in planner.c deal in PathTargets not
      targetlists, and establish a more regular structure for deriving the
      targets needed for successive steps.
      
      There is more that could be done here; calculating the eval costs of each
      successive target independently is both inefficient and wrong in detail,
      since we won't actually recompute values available from the input node's
      tlist.  But it's no worse than what happened before the pathification
      rewrite.  In any case this seems like a good starting point for considering
      how to handle Konstantin Knizhnik's function-evaluation-postponement patch.
      51c0f63e
    • Andres Freund's avatar
      Add valgrind suppressions for python code. · 2f1f4439
      Andres Freund authored
      Python's allocator does some low-level tricks for efficiency;
      unfortunately they trigger valgrind errors. Those tricks can be disabled
      making instrumentation easier; but few people testing postgres will have
      such a build of python. So add broad suppressions of the resulting
      errors.
      
      See also https://svn.python.org/projects/python/trunk/Misc/README.valgrind
      
      This possibly will suppress valid errors, but without it it's basically
      impossible to use valgrind with plpython code.
      
      Author: Andres Freund
      Backpatch: 9.4, where we started to maintain valgrind suppressions
      2f1f4439
    • Andres Freund's avatar
      Add valgrind suppressions for bootstrap related code. · 5e43bee8
      Andres Freund authored
      Author: Andres Freund
      Backpatch: 9.4, where we started to maintain valgrind suppressions
      5e43bee8
    • Tom Lane's avatar
      Improve handling of group-column indexes in GroupingSetsPath. · 9e8b9942
      Tom Lane authored
      Instead of having planner.c compute a groupColIdx array and store it in
      GroupingSetsPaths, make create_groupingsets_plan() find the grouping
      columns by searching in the child plan node's tlist.  Although that's
      probably a bit slower for create_groupingsets_plan(), it's more like
      the way every other plan node type does this, and it provides positive
      confirmation that we know which child output columns we're supposed to be
      grouping on.  (Indeed, looking at this now, I'm not at all sure that it
      wasn't broken before, because create_groupingsets_plan() isn't demanding
      an exact tlist match from its child node.)  Also, this allows substantial
      simplification in planner.c, because it no longer needs to compute the
      groupColIdx array at all; no other cases were using it.
      
      I'd intended to put off this refactoring until later (like 9.7), but
      in view of the likely bug fix and the need to rationalize planner.c's
      tlist handling so we can do something sane with Konstantin Knizhnik's
      function-evaluation-postponement patch, I think it can't wait.
      9e8b9942
    • Peter Eisentraut's avatar
      Handle invalid libpq sockets in more places · a40814d7
      Peter Eisentraut authored
      Also, make error messages consistent.
      
      From: Michael Paquier <michael.paquier@gmail.com>
      a40814d7
    • Peter Eisentraut's avatar
    • Peter Eisentraut's avatar
      psql: Fix some strange code in SQL help creation · 92d4294d
      Peter Eisentraut authored
      Struct QL_HELP used to be defined as static in the sql_help.h header
      file, which is included in sql_help.c and help.c, thus creating two
      separate instances of the struct.  This causes a warning from GCC 6,
      because the struct is not used in sql_help.c.
      
      Instead, declare the struct as extern in the header file and define it
      in sql_help.c.  This also allows making a bunch of functions static
      because they are no longer needed outside of sql_help.c.
      Reviewed-by: default avatarThomas Munro <thomas.munro@enterprisedb.com>
      92d4294d
    • Peter Eisentraut's avatar
      ecpg: Fix typo · 0d0644dc
      Peter Eisentraut authored
      GCC 6 points out the redundant conditions, which were apparently typos.
      Reviewed-by: default avatarThomas Munro <thomas.munro@enterprisedb.com>
      0d0644dc
  2. 08 Mar, 2016 17 commits
    • Andres Freund's avatar
      ltree: Zero padding bytes when allocating memory for externally visible data. · 7a1d4a24
      Andres Freund authored
      ltree/ltree_gist/ltxtquery's headers stores data at MAXALIGN alignment,
      requiring some padding bytes. So far we left these uninitialized. Zero
      those by using palloc0.
      
      Author: Andres Freund
      Reported-By: Andres Freund / valgrind / buildarm animal skink
      Backpatch: 9.1-
      7a1d4a24
    • Tom Lane's avatar
      Fix minor thinko in pathification code. · 61fd2189
      Tom Lane authored
      I passed the wrong "root" struct to create_pathtarget in build_minmax_path.
      Since the subroot is a clone of the outer root, this would not cause any
      serious problems, but it would waste some cycles because
      set_pathtarget_cost_width would not have access to Var width estimates
      set up while running query_planner on the subroot.
      61fd2189
    • Andres Freund's avatar
      plperl: Correctly handle empty arrays in plperl_ref_from_pg_array. · e66197fa
      Andres Freund authored
      plperl_ref_from_pg_array() didn't consider the case that postgrs arrays
      can have 0 dimensions (when they're empty) and accessed the first
      dimension without a check. Fix that by special casing the empty array
      case.
      
      Author: Alex Hunsaker
      Reported-By: Andres Freund / valgrind / buildfarm animal skink
      Discussion: 20160308063240.usnzg6bsbjrne667@alap3.anarazel.de
      Backpatch: 9.1-
      e66197fa
    • Tom Lane's avatar
      Finish refactoring make_foo() functions in createplan.c. · 8c314b98
      Tom Lane authored
      This patch removes some redundant cost calculations that I left for later
      cleanup in commit 3fc6e2d7.  There's now a uniform policy that the
      make_foo() convenience functions don't do any cost calculations.  Most of
      their callers copy costs from the source Path node, and for those that
      don't, the calculation in the make_foo() function wasn't necessarily right
      anyhow.  (make_result() was particularly a mess, as it was serving multiple
      callers using cost calcs designed for only the first one or two that had
      ever existed.)  Aside from saving a few cycles, this ensures that what
      EXPLAIN prints matches the costs we used for planning purposes.  It does
      not change any planner decisions, since the decisions are already made.
      8c314b98
    • Robert Haas's avatar
      Comment update for fdw_recheck_quals. · 7400559a
      Robert Haas authored
      Commit 5fc4c26d could've done a better
      job updating these comments.
      
      Etsuro Fujita
      7400559a
    • Robert Haas's avatar
      Update GetForeignPlan documentation. · dff7ad3c
      Robert Haas authored
      Commit 385f337c added a new argument
      to the FDW GetForeignPlan method, but failed to update the documentation
      to match.
      
      Etsuro Fujita
      dff7ad3c
    • Robert Haas's avatar
      Fix reversed argument to bms_is_subset. · d29b153f
      Robert Haas authored
      Ashutosh Bapat
      d29b153f
    • Robert Haas's avatar
      Add new flags argument for xl_heap_visible to heap2_desc. · 734f86d5
      Robert Haas authored
      Masahiko Sawada
      734f86d5
    • Robert Haas's avatar
      Fix typo. · 272baaa5
      Robert Haas authored
      Masahiko Sawada
      272baaa5
    • Robert Haas's avatar
      Fix parallel query on standby servers. · dcfecaae
      Robert Haas authored
      Without this fix, it inevitably bombs out with "ERROR:  failed to
      initialize transaction_read_only to 0".  Repair.
      
      Ashutosh Sharma; comments adjusted by me.
      dcfecaae
    • Robert Haas's avatar
      Add some functions to fd.c for the convenience of extensions. · 070140ee
      Robert Haas authored
      For example, if you want to perform an ioctl() on a file descriptor
      opened through the fd.c routines, there's no way to do that without
      being able to get at the underlying fd.
      
      KaiGai Kohei
      070140ee
    • Robert Haas's avatar
      Department of second thoughts: remove PD_ALL_FROZEN. · 77a1d1e7
      Robert Haas authored
      Commit a892234f added a second bit per
      page to the visibility map, which still seems like a good idea, but it
      also added a second page-level bit alongside PD_ALL_VISIBLE to track
      whether the visibility map bit was set.  That no longer seems like a
      clever plan, because we don't really need that bit for anything.  We
      always clear both bits when the page is modified anyway.
      
      Patch by me, reviewed by Kyotaro Horiguchi and Masahiko Sawada.
      77a1d1e7
    • Robert Haas's avatar
      Add pg_visibility contrib module. · ba0a198f
      Robert Haas authored
      This lets you examine the visibility map as well as page-level
      visibility information.  I initially wrote it as a debugging aid,
      but was encouraged to polish it for commit.
      
      Patch by me, reviewed by Masahiko Sawada.
      
      Discussion: 56D77803.6080503@BlueTreble.com
      ba0a198f
    • Robert Haas's avatar
      pg_upgrade: Remove converter plugin facility. · 6f56b41a
      Robert Haas authored
      We've not found a use for this so far, and the current need, which
      is to convert the visibility map to a new format, does not suit the
      existing design anyway.  So just rip it out.
      
      Author: Masahiko Sawada, slightly revised by me.
      Discussion: 20160215211313.GB31273@momjian.us
      6f56b41a
    • Tom Lane's avatar
      Fix minor typo in logical-decoding docs. · a93aec4e
      Tom Lane authored
      David Rowley
      a93aec4e
    • Tom Lane's avatar
      Spell "parallel" correctly. · cf8e7b16
      Tom Lane authored
      Per David Rowley.
      cf8e7b16
    • Peter Eisentraut's avatar
      Fix uninstall target in tsearch Makefile · 1c2db8c3
      Peter Eisentraut authored
      Artur Zakirov
      1c2db8c3
  3. 07 Mar, 2016 5 commits
    • Joe Conway's avatar
      Make get_controlfile() error logging consistent with src/common · 7b077af5
      Joe Conway authored
      As originally committed, get_controlfile() used a non-standard approach
      to error logging. Make it consistent with the majority of error logging
      done in src/common.
      
      Applies to master only.
      7b077af5
    • Andres Freund's avatar
      Further improvements to c8f621c4. · b63bea5f
      Andres Freund authored
      Coverity and inspection for the issue addressed in fd45d16f found some
      questionable code.
      
      Specifically coverity noticed that the wrong length was added in
      ReorderBufferSerializeChange() - without immediate negative consequences
      as the variable isn't used afterwards.  During code-review and testing I
      noticed that a bit of space was wasted when allocating tuple bufs in
      several places.  Thirdly, the debug memset()s in
      ReorderBufferGetTupleBuf() reduce the error checking valgrind can do.
      
      Backpatch: 9.4, like c8f621c4.
      b63bea5f
    • Tom Lane's avatar
      Make the upper part of the planner work by generating and comparing Paths. · 3fc6e2d7
      Tom Lane authored
      I've been saying we needed to do this for more than five years, and here it
      finally is.  This patch removes the ever-growing tangle of spaghetti logic
      that grouping_planner() used to use to try to identify the best plan for
      post-scan/join query steps.  Now, there is (nearly) independent
      consideration of each execution step, and entirely separate construction of
      Paths to represent each of the possible ways to do that step.  We choose
      the best Path or set of Paths using the same add_path() logic that's been
      used inside query_planner() for years.
      
      In addition, this patch removes the old restriction that subquery_planner()
      could return only a single Plan.  It now returns a RelOptInfo containing a
      set of Paths, just as query_planner() does, and the parent query level can
      use each of those Paths as the basis of a SubqueryScanPath at its level.
      This allows finding some optimizations that we missed before, wherein a
      subquery was capable of returning presorted data and thereby avoiding a
      sort in the parent level, making the overall cost cheaper even though
      delivering sorted output was not the cheapest plan for the subquery in
      isolation.  (A couple of regression test outputs change in consequence of
      that.  However, there is very little change in visible planner behavior
      overall, because the point of this patch is not to get immediate planning
      benefits but to create the infrastructure for future improvements.)
      
      There is a great deal left to do here.  This patch unblocks a lot of
      planner work that was basically impractical in the old code structure,
      such as allowing FDWs to implement remote aggregation, or rewriting
      plan_set_operations() to allow consideration of multiple implementation
      orders for set operations.  (The latter will likely require a full
      rewrite of plan_set_operations(); what I've done here is only to fix it
      to return Paths not Plans.)  I have also left unfinished some localized
      refactoring in createplan.c and planner.c, because it was not necessary
      to get this patch to a working state.
      
      Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
      3fc6e2d7
    • Tom Lane's avatar
      Fix backwards test for Windows service-ness in pg_ctl. · b642e50a
      Tom Lane authored
      A thinko in a9676139 caused pg_ctl to get it exactly backwards when
      deciding whether to report problems to the Windows eventlog or to stderr.
      Per bug #14001 from Manuel Mathar, who also identified the fix.
      Like the previous patch, back-patch to all supported branches.
      b642e50a
    • Tom Lane's avatar
      Re-fix broken definition for function name in pgbench's exprscan.l. · 94f1adcc
      Tom Lane authored
      Wups, my first try wasn't quite right either.  Too focused on fixing
      the existing bug, not enough on not introducing new ones.
      94f1adcc