Skip to content

P
Postgres FD Implementation
Switch branch/tag
  1. 22 Mar, 2018 12 commits
    • Tom Lane's avatar
      Fix errors in contrib/bloom index build. · c35b4728
      Tom Lane authored 
      Count the number of tuples in the index honestly, instead of assuming
that it's the same as the number of tuples in the heap.  (It might be
different if the index is partial.)

Fix counting of tuples in current index page, too.  This error would
have led to failing to write out the final page of the index if it
contained exactly one tuple, so that the last tuple of the relation
would not get indexed.

Back-patch to 9.6 where contrib/bloom was added.

Tomas Vondra and Tom Lane

Discussion: https://postgr.es/m/3b3d8eac-c709-0d25-088e-b98339a1b28a@2ndquadrant.com
      c35b4728
    • Robert Haas's avatar
      Implement partition-wise grouping/aggregation. · e2f1eb0e
      Robert Haas authored 
      If the partition keys of input relation are part of the GROUP BY
clause, all the rows belonging to a given group come from a single
partition.  This allows aggregation/grouping over a partitioned
relation to be broken down * into aggregation/grouping on each
partition.  This should be no worse, and often better, than the normal
approach.

If the GROUP BY clause does not contain all the partition keys, we can
still perform partial aggregation for each partition and then finalize
aggregation after appending the partial results.  This is less certain
to be a win, but it's still useful.

Jeevan Chalke, Ashutosh Bapat, Robert Haas.  The larger patch series
of which this patch is a part was also reviewed and tested by Antonin
Houska, Rajkumar Raghuwanshi, David Rowley, Dilip Kumar, Konstantin
Knizhnik, Pascal Legrand, and Rafia Sabih.

Discussion: http://postgr.es/m/CAM2+6=V64_xhstVHie0Rz=KPEQnLJMZt_e314P0jaT_oJ9MR8A@mail.gmail.com
      e2f1eb0e
    • Teodor Sigaev's avatar
      Add conditional.c to libpgfeutils for MSVC build · 2058d6a2
      Teodor Sigaev authored 
      conditional.c was moved in f67b113a commit
but forgotten to add to Windows build system.

I don't have a Windows box, so blind attempt.
      2058d6a2
    • Teodor Sigaev's avatar
      UINT64CONST'fy long constants in pgbench · 2216fded
      Teodor Sigaev authored 
      In commit e51a0484 it was missed 64-bit
constants, wrap them with UINT64CONST().

Per buildfarm member dromedary and gripe from Tom Lane
      2216fded
    • Teodor Sigaev's avatar
      Add \if support to pgbench · f67b113a
      Teodor Sigaev authored 
      Patch adds \if to pgbench as it done for psql. Implementation shares condition
stack code with psql, so, this code is moved to fe_utils directory.

Author: Fabien COELHO with minor editorization by me
Review by: Vik Fearing, Fedor Sigaev
Discussion: https://www.postgresql.org/message-id/flat/alpine.DEB.2.20.1711252200190.28523@lancre
      f67b113a
    • Dean Rasheed's avatar
      Improve ANALYZE's strategy for finding MCVs. · b5db1d93
      Dean Rasheed authored 
      Previously, a value was included in the MCV list if its frequency was
25% larger than the estimated average frequency of all nonnull values
in the table.  For uniform distributions, that can lead to values
being included in the MCV list and significantly overestimated on the
basis of relatively few (sometimes just 2) instances being seen in the
sample.  For non-uniform distributions, it can lead to too few values
being included in the MCV list, since the overall average frequency
may be dominated by a small number of very common values, while the
remaining values may still have a large spread of frequencies, causing
both substantial overestimation and underestimation of the remaining
values.  Furthermore, increasing the statistics target may have little
effect because the overall average frequency will remain relatively
unchanged.

Instead, populate the MCV list with the largest set of common values
that are statistically significantly more common than the average
frequency of the remaining values.  This takes into account the
variance of the sample counts, which depends on the counts themselves
and on the proportion of the table that was sampled.  As a result, it
constrains the relative standard error of estimates based on the
frequencies of values in the list, reducing the chances of too many
values being included.  At the same time, it allows more values to be
included, since the MCVs need only be more common than the remaining
non-MCVs, rather than the overall average.  Thus it tends to produce
fewer MCVs than the previous code for uniform distributions, and more
for non-uniform distributions, reducing estimation errors in both
cases.  In addition, the algorithm responds better to increasing the
statistics target, allowing more values to be included in the MCV list
when more of the table is sampled.

Jeff Janes, substantially modified by me. Reviewed by John Naylor and
Tomas Vondra.

Discussion: https://postgr.es/m/CAMkU=1yvdGvW9TmiLAhz2erFnvnPFYHbOZuO+a=4DVkzpuQ2tw@mail.gmail.com
      b5db1d93
    • Andres Freund's avatar
      Add file containing extensions of the LLVM C API. · 31bc604e
      Andres Freund authored 
      Author: Andres Freund
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
      31bc604e
    • Andres Freund's avatar
      Basic JIT provider and error handling infrastructure. · 432bb9e0
      Andres Freund authored 
      This commit introduces:

1) JIT provider abstraction, which allows JIT functionality to be
   implemented in separate shared libraries. That's desirable because
   it allows to install JIT support as a separate package, and because
   it allows experimentation with different forms of JITing.
2) JITContexts which can be, using functions introduced in follow up
   commits, used to emit JITed functions, and have them be cleaned up
   on error.
3) The outline of a LLVM JIT provider, which will be fleshed out in
   subsequent commits.

Documentation for GUCs added, and for JIT in general, will be added in
later commits.

Author: Andres Freund, with architectural input from Jeff Davis
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
      432bb9e0
    • Andres Freund's avatar
      Fix typo in BITCODE_CXXFLAGS assignment. · 4317cc68
      Andres Freund authored 
      Typoed-In: 5b2526c8
Reported-By: Catalin Iacob
      4317cc68
    • Andres Freund's avatar
      Empty CXXFLAGS inherited from autoconf. · a02671cf
      Andres Freund authored 
      We do the same for CFLAGS.  This was an omission in 6869b4f2.

Reported-By: Catalin Iacob
      a02671cf
    • Tom Lane's avatar
      Prevent extensions from creating custom GUCs that are GUC_LIST_QUOTE. · 846b5a52
      Tom Lane authored 
      Pending some solution for the problems noted in commit 74286994,
disallow dynamic creation of GUC_LIST_QUOTE variables.

If there are any extensions out there using this feature, they'd not
be happy for us to start enforcing this rule in minor releases, so
this is a HEAD-only change.  The previous commit didn't make things
any worse than they already were for such cases.

Discussion: https://postgr.es/m/20180111064900.GA51030@paquier.xyz
      846b5a52
    • Tom Lane's avatar
      Fix mishandling of quoted-list GUC values in pg_dump and ruleutils.c. · 74286994
      Tom Lane authored 
      Code that prints out the contents of setconfig or proconfig arrays in
SQL format needs to handle GUC_LIST_QUOTE variables differently from
other ones, because for those variables, flatten_set_variable_args()
already applied a layer of quoting.  The value can therefore safely
be printed as-is, and indeed must be, or flatten_set_variable_args()
will muck it up completely on reload.  For all other GUC variables,
it's necessary and sufficient to quote the value as a SQL literal.

We'd recognized the need for this long ago, but mis-analyzed the
need slightly, thinking that all GUC_LIST_INPUT variables needed
the special treatment.  That's actually wrong, since a valid value
of a LIST variable might include characters that need quoting,
although no existing variables accept such values.

More to the point, we hadn't made any particular effort to keep the
various places that deal with this up-to-date with the set of variables
that actually need special treatment, meaning that we'd do the wrong
thing with, for example, temp_tablespaces values.  This affects dumping
of SET clauses attached to functions, as well as ALTER DATABASE/ROLE SET
commands.

In ruleutils.c we can fix it reasonably honestly by exporting a guc.c
function that allows discovering the flags for a given GUC variable.
But pg_dump doesn't have easy access to that, so continue the old method
of having a hard-wired list of affected variable names.  At least we can
fix it to have just one list not two, and update the list to match
current reality.

A remaining problem with this is that it only works for built-in
GUC variables.  pg_dump's list obvious knows nothing of third-party
extensions, and even the "ask guc.c" method isn't bulletproof since
the relevant extension might not be loaded.  There's no obvious
solution to that, so for now, we'll just have to discourage extension
authors from inventing custom GUCs that need GUC_LIST_QUOTE.

This has been busted for a long time, so back-patch to all supported
branches.

Michael Paquier and Tom Lane, reviewed by Kyotaro Horiguchi and
Pavel Stehule

Discussion: https://postgr.es/m/20180111064900.GA51030@paquier.xyz
      74286994
  3. 21 Mar, 2018 14 commits
  5. 20 Mar, 2018 10 commits
  7. 19 Mar, 2018 4 commits
    • Peter Eisentraut's avatar
      Add missing break · 13c7c65e
      Peter Eisentraut authored
      13c7c65e
    • Tom Lane's avatar
      Fix some corner-case issues in REFRESH MATERIALIZED VIEW CONCURRENTLY. · 6497a18e
      Tom Lane authored 
      refresh_by_match_merge() has some issues in the way it builds a SQL
query to construct the "diff" table:

1. It doesn't require the selected unique index(es) to be indimmediate.
2. It doesn't pay attention to the particular equality semantics enforced
by a given index, but just assumes that they must be those of the column
datatype's default btree opclass.
3. It doesn't check that the indexes are btrees.
4. It's insufficiently careful to ensure that the parser will pick the
intended operator when parsing the query.  (This would have been a
security bug before CVE-2018-1058.)
5. It's not careful about indexes on system columns.

The way to fix #4 is to make use of the existing code in ri_triggers.c
for generating an arbitrary binary operator clause.  I chose to move
that to ruleutils.c, since that seems a more reasonable place to be
exporting such functionality from than ri_triggers.c.

While #1, #3, and #5 are just latent given existing feature restrictions,
and #2 doesn't arise in the core system for lack of alternate opclasses
with different equality behaviors, #4 seems like an issue worth
back-patching.  That's the bulk of the change anyway, so just back-patch
the whole thing to 9.4 where this code was introduced.

Discussion: https://postgr.es/m/13836.1521413227@sss.pgh.pa.us
      6497a18e
    • Andrew Dunstan's avatar
      Don't use an Msys virtual path to create a tablespace · 9ad21a69
      Andrew Dunstan authored 
      The new unlogged_reinit recovery tests create a new tablespace using
TestLib.pm's tempdir. However, on msys that function returns a virtual
path that isn't understood by Postgres. Here we add a new function to
TestLib.pm to turn such a path into a real path on the underlying file
system, and use it in the new test to create the tablespace. The new
function is essentially a NOOP everywhere but msys.
      9ad21a69
    • Tom Lane's avatar
      Fix performance hazard in REFRESH MATERIALIZED VIEW CONCURRENTLY. · 6fbd5cce
      Tom Lane authored 
      Jeff Janes discovered that commit 7ca25b7d made one of the queries run by
REFRESH MATERIALIZED VIEW CONCURRENTLY perform badly.  The root cause is
bad cardinality estimation for correlated quals, but a principled solution
to that problem is some way off, especially since the planner lacks any
statistics about whole-row variables.  Moreover, in non-error cases this
query produces no rows, meaning it must be run to completion; but use of
LIMIT 1 encourages the planner to pick a fast-start, slow-completion plan,
exactly not what we want.  Remove the LIMIT clause, and instead rely on
the count parameter we pass to SPI_execute() to prevent excess work if the
query does return some rows.

While we've heard no field reports of planner misbehavior with this query,
it could be that people are having performance issues that haven't reached
the level of pain needed to cause a bug report.  In any case, that LIMIT
clause can't possibly do anything helpful with any existing version of the
planner, and it demonstrably can cause bad choices in some cases, so
back-patch to 9.4 where the code was introduced.

Thomas Munro

Discussion: https://postgr.es/m/CAMkU=1z-JoGymHneGHar1cru4F1XDfHqJDzxP_CtK5cL3DOfmg@mail.gmail.com
      6fbd5cce