Skip to content

P
Postgres FD Implementation
Switch branch/tag
  1. 24 Jun, 2019 4 commits
    • Tom Lane's avatar
      Further fix ALTER COLUMN TYPE's handling of indexes and index constraints. · f946a409
      Tom Lane authored 
      This patch reverts all the code changes of commit e76de886, which turns
out to have been seriously misguided.  We can't wait till later to compute
the definition string for an index; we must capture that before applying
the data type change for any column it depends on, else ruleutils.c will
deliverr wrong/misleading results.  (This fine point was documented
nowhere, of course.)

I'd also managed to forget that ATExecAlterColumnType executes once per
ALTER COLUMN TYPE clause, not once per statement; which resulted in the
code being basically completely broken for any case in which multiple ALTER
COLUMN TYPE clauses are applied to a table having non-constraint indexes
that must be rebuilt.  Through very bad luck, none of the existing test
cases nor the ones added by e76de886 caught that, but of course it was
soon found in the field.

The previous patch also had an implicit assumption that if a constraint's
index had a dependency on a table column, so would the constraint --- but
that isn't actually true, so it didn't fix such cases.

Instead of trying to delete unneeded index dependencies later, do the
is-there-a-constraint lookup immediately on seeing an index dependency,
and switch to remembering the constraint if so.  In the unusual case of
multiple column dependencies for a constraint index, this will result in
duplicate constraint lookups, but that's not that horrible compared to all
the other work that happens here.  Besides, such cases did not work at all
before, so it's hard to argue that they're performance-critical for anyone.

Per bug #15865 from Keith Fiske.  As before, back-patch to all supported
branches.

Discussion: https://postgr.es/m/15865-17940eacc8f8b081@postgresql.org
      f946a409
    • Peter Geoghegan's avatar
      Correct obsolete amcheck comments. · b00326df
      Peter Geoghegan authored 
      Oversight in commit dd299df8.
      b00326df
    • Tom Lane's avatar
      Drop test user when done with it. · f31111bb
      Tom Lane authored 
      Commit d7f8d26d added a test case that created a user, but forgot
to drop it again.  This is no good; for one thing, it causes repeated
"make installcheck" runs to fail.
      f31111bb
    • Peter Eisentraut's avatar
      Upgrade internal error message to external · 12e037e2
      Peter Eisentraut authored 
      As part of REINDEX CONCURRENTLY, this formerly internal-only error
message becomes potentially user-visible (see regression tests), so
change from errmsg_internal() to errmsg(), and update comment.
      12e037e2
  3. 23 Jun, 2019 4 commits
    • Noah Misch's avatar
      Don't call PG_RETURN_BOOL() in a function not returning Datum. · 9a81c9fa
      Noah Misch authored 
      This code is new in v12, and the defect probably was not user-visible.
      9a81c9fa
    • Dean Rasheed's avatar
      Add security checks to the multivariate MCV estimation code. · d7f8d26d
      Dean Rasheed authored 
      The multivariate MCV estimation code may run user-defined operators on
the values in the MCV list, which means that those operators may
potentially leak the values from the MCV list. Guard against leaking
data to unprivileged users by checking that the user has SELECT
privileges on the table or all of the columns referred to by the
statistics.

Additionally, if there are any securityQuals on the RTE (either due to
RLS policies on the table, or accessing the table via a security
barrier view), not all rows may be visible to the current user, even
if they have table or column privileges. Thus we further insist that
the operator be leakproof in this case.

Dean Rasheed, reviewed by Tomas Vondra.

Discussion: https://postgr.es/m/CAEZATCUhT9rt7Ui=Vdx4N==VV5XOK5dsXfnGgVOz_JhAicB=ZA@mail.gmail.com
      d7f8d26d
    • Thomas Munro's avatar
      Remove unnecessary comment. · 89ff7c08
      Thomas Munro authored 
      Author: Vik Fearing
Discussion: https://postgr.es/m/150d3e9f-c7ec-3fb3-4fdb-def47c4144af%402ndquadrant.com
      89ff7c08
    • Tom Lane's avatar
      Fix spinlock assembly code for MIPS so it works on MIPS r6. · 1323bfce
      Tom Lane authored 
      Original MIPS-I processors didn't have the LL/SC instructions (nor any
other userland synchronization primitive).  If the build toolchain
targets that ISA variant by default, as an astonishingly large fraction
of MIPS platforms still do, the assembler won't take LL/SC without
coercion in the form of a ".set mips2" instruction.  But we issued that
unconditionally, making it an ISA downgrade for chips later than MIPS2.
That breaks things for the latest MIPS r6 ISA, which encodes these
instructions differently.  Adjust the code so we don't change ISA level
if it's >= 2.

Note that this patch doesn't change what happens on an actual MIPS-I
processor: either the kernel will emulate these instructions
transparently, or you'll get a SIGILL failure.  That tradeoff seemed
fine in 2002 when this code was added (cf 3cbe6b24), and it's even
more so today when MIPS-I is basically extinct.  But let's add a
comment about that.

YunQiang Su (with cosmetic adjustments by me).  Back-patch to all
supported branches.

Discussion: https://postgr.es/m/15844-8f62fe7e163939b3@postgresql.org
      1323bfce
  5. 22 Jun, 2019 1 commit
  7. 20 Jun, 2019 3 commits
  9. 19 Jun, 2019 11 commits
  11. 18 Jun, 2019 3 commits
    • Andres Freund's avatar
      Fix memory corruption/crash in ANALYZE. · 23224563
      Andres Freund authored 
      This fixes an embarrassing oversight I (Andres) made in 737a292b,
namely missing two place where liverows/deadrows were used when
converting those variables to pointers, leading to incrementing the
pointer, rather than the value.

It's not that actually that easy to trigger a crash: One needs tuples
deleted by the current transaction, followed by a tuple deleted in
another session, all in one page. Which is presumably why this hasn't
been noticed before.

Reported-By: Steve Singer
Author: Steve Singer
Discussion: https://postgr.es/m/c7988239-d42c-ddc4-41db-171b23b35e4f@ssinger.info
      23224563
    • Alvaro Herrera's avatar
      Avoid spurious deadlocks when upgrading a tuple lock · 8b21b416
      Alvaro Herrera authored 
      This puts back reverted commit de87a084, with some bug fixes.

When two (or more) transactions are waiting for transaction T1 to release a
tuple-level lock, and transaction T1 upgrades its lock to a higher level, a
spurious deadlock can be reported among the waiting transactions when T1
finishes.  The simplest example case seems to be:

T1: select id from job where name = 'a' for key share;
Y: select id from job where name = 'a' for update; -- starts waiting for T1
Z: select id from job where name = 'a' for key share;
T1: update job set name = 'b' where id = 1;
Z: update job set name = 'c' where id = 1; -- starts waiting for T1
T1: rollback;

At this point, transaction Y is rolled back on account of a deadlock: Y
holds the heavyweight tuple lock and is waiting for the Xmax to be released,
while Z holds part of the multixact and tries to acquire the heavyweight
lock (per protocol) and goes to sleep; once T1 releases its part of the
multixact, Z is awakened only to be put back to sleep on the heavyweight
lock that Y is holding while sleeping.  Kaboom.

This can be avoided by having Z skip the heavyweight lock acquisition.  As
far as I can see, the biggest downside is that if there are multiple Z
transactions, the order in which they resume after T1 finishes is not
guaranteed.

Backpatch to 9.6.  The patch applies cleanly on 9.5, but the new tests don't
work there (because isolationtester is not smart enough), so I'm not going
to risk it.

Author: Oleksii Kliukin
Discussion: https://postgr.es/m/B9C9D7CD-EB94-4635-91B6-E558ACEC0EC3@hintbits.com
Discussion: https://postgr.es/m/2815.1560521451@sss.pgh.pa.us
      8b21b416
    • Thomas Munro's avatar
      Prevent Parallel Hash Join for JOIN_UNIQUE_INNER. · aca127c1
      Thomas Munro authored 
      WHERE EXISTS (...) queries cannot be executed by Parallel Hash Join
with jointype JOIN_UNIQUE_INNER, because there is no way to make a
partial plan totally unique.  The consequence of allowing such plans
was duplicate results from some EXISTS queries.

Back-patch to 11.  Bug #15857.

Author: Thomas Munro
Reviewed-by: Tom Lane
Reported-by: Vladimir Kriukov
Discussion: https://postgr.es/m/15857-d1ba2a64bce0795e%40postgresql.org
      aca127c1
  13. 17 Jun, 2019 6 commits
  15. 16 Jun, 2019 3 commits
  17. 15 Jun, 2019 4 commits
    • Tomas Vondra's avatar
      Fix incorrect CREATE STATISTICS example in docs · 7f44efa1
      Tomas Vondra authored 
      The example was incorrectly using parantheses around the list of columns, so
just drop them.

Reported-By: Robert Haas
Discussion: https://postgr.es/m/CA%2BTgmoZZEMAqWMAfvLHZnK57SoxOutgvE-ALO94WsRA7zZ7wyQ%40mail.gmail.com
      7f44efa1
    • Tomas Vondra's avatar
      Add pg_stats_ext view for extended statistics · aa087ec6
      Tomas Vondra authored 
      Regular per-column statistics are stored in pg_statistics catalog, which
is however rather difficult to read, so we also have pg_stats view with
a human-reablable version of the data.

For extended statistic the catalog was fairly easy to read, so we did
not have such human-readable view so far.  Commit 9b6babfa2d however did
split the catalog into two, which makes querying harder.  Furthermore,
we want to show the multi-column MCV list in a way similar to per-column
stats (and not as a bytea value).

This commit introduces pg_stats_ext view, joining the two catalogs and
massaging the data to produce human-readable output similar to pg_stats.
It also considers RLS and access privileges - the data is shown only when
the user has access to all columns the extended statistic is defined on.

Bumped CATVERSION due to adding new system view.

Author: Dean Rasheed, with improvements by me
Reviewed-by: Dean Rasheed, John Naylor
Discussion: https://postgr.es/m/CAEZATCUhT9rt7Ui%3DVdx4N%3D%3DVV5XOK5dsXfnGgVOz_JhAicB%3DZA%40mail.gmail.com
      aa087ec6
    • Tomas Vondra's avatar
      Rework the pg_statistic_ext catalog · 6cbfb784
      Tomas Vondra authored 
      Since extended statistic got introduced in PostgreSQL 10, there was a
single catalog pg_statistic_ext storing both the definitions and built
statistic.  That's however problematic when a user is supposed to have
access only to the definitions, but not to user data.

Consider for example pg_dump on a database with RLS enabled - if the
pg_statistic_ext catalog respects RLS (which it should, if it contains
user data), pg_dump would not see any records and the result would not
define any extended statistics.  That would be a surprising behavior.

Until now this was not a pressing issue, because the existing types of
extended statistic (functional dependencies and ndistinct coefficients)
do not include any user data directly.  This changed with introduction
of MCV lists, which do include most common combinations of values.

The easiest way to fix this is to split the pg_statistic_ext catalog
into two - one for definitions, one for the built statistic values.
The new catalog is called pg_statistic_ext_data, and we're maintaining
a 1:1 relationship with the old catalog - either there are matching
records in both catalogs, or neither of them.

Bumped CATVERSION due to changing system catalog definitions.

Author: Dean Rasheed, with improvements by me
Reviewed-by: Dean Rasheed, John Naylor
Discussion: https://postgr.es/m/CAEZATCUhT9rt7Ui%3DVdx4N%3D%3DVV5XOK5dsXfnGgVOz_JhAicB%3DZA%40mail.gmail.com
      6cbfb784
    • Andrew Gierth's avatar
      Prefer timezone name "UTC" over alternative spellings. · e3846a00
      Andrew Gierth authored 
      tzdb 2019a made "UCT" a link to the "UTC" zone rather than a separate
zone with its own abbreviation. Unfortunately, our code for choosing a
timezone in initdb has an arbitrary preference for names earlier in
the alphabet, and so it would choose the spelling "UCT" over "UTC"
when the system is running on a UTC zone.

Commit 23bd3cec was backpatched in order to address this issue, but
that code helps only when /etc/localtime exists as a symlink, and does
nothing to help on systems where /etc/localtime is a copy of a zone
file (as is the standard setup on FreeBSD and probably some other
platforms too) or when /etc/localtime is simply absent (giving UTC as
the default).

Accordingly, add a preference for the spelling "UTC", such that if
multiple zone names have equally good content matches, we prefer that
name before applying the existing arbitrary rules. Also add a slightly
lower preference for "Etc/UTC"; lower because that preserves the
previous behaviour of choosing the shorter name, but letting us still
choose "Etc/UTC" over "Etc/UCT" when both exist but "UTC" does
not (not common, but I've seen it happen).

Backpatch all the way, because the tzdb change that sparked this issue
is in those branches too.
      e3846a00
  19. 14 Jun, 2019 1 commit