- 06 Mar, 2019 6 commits
-
-
Andres Freund authored
This adds pg_dump support for table AMs in a similar manner to how tablespaces are handled. That is, instead of specifying the AM for every CREATE TABLE etc, emit SET default_table_access_method statements. That makes it easier to change the AM for all/most tables in a dump, and allows restore to succeed even if some AM is not available. This increases the dump archive version, as a tables/matview's AM needs to be tracked therein. Author: Dimitri Dolgov, Andres Freund Discussion: https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de https://postgr.es/m/20190304234700.w5tmhducs5wxgzls@alap3.anarazel.de
-
Andres Freund authored
This introduces the concept of table access methods, i.e. CREATE ACCESS METHOD ... TYPE TABLE and CREATE TABLE ... USING (storage-engine). No table access functionality is delegated to table AMs as of this commit, that'll be done in following commits. Subsequent commits will incrementally abstract table access functionality to be routed through table access methods. That change is too large to be reviewed & committed at once, so it'll be done incrementally. Docs will be updated at the end, as adding them incrementally would likely make them less coherent, and definitely is a lot more work, without a lot of benefit. Table access methods are specified similar to index access methods, i.e. pg_am.amhandler returns, as INTERNAL, a pointer to a struct with callbacks. In contrast to index AMs that struct needs to live as long as a backend, typically that's achieved by just returning a pointer to a constant struct. Psql's \d+ now displays a table's access method. That can be disabled with HIDE_TABLEAM=true, which is mainly useful so regression tests can be run against different AMs. It's quite possible that this behaviour still needs to be fine tuned. For now it's not allowed to set a table AM for a partitioned table, as we've not resolved how partitions would inherit that. Disallowing allows us to introduce, if we decide that's the way forward, such a behaviour without a compatibility break. Catversion bumped, to add the heap table AM and references to it. Author: Haribabu Kommi, Andres Freund, Alvaro Herrera, Dimitri Golgov and others Discussion: https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql https://postgr.es/m/20190107235616.6lur25ph22u5u5av@alap3.anarazel.de https://postgr.es/m/20190304234700.w5tmhducs5wxgzls@alap3.anarazel.de
-
Andres Freund authored
I broke/typoed this in 4da597ed. Astonishingly this mostly doesn't cause breakage, except when trying to change the tuple descriptor of a slot (because TTS_FLAG_FIXED is assumed to be set). Author: Andres Freund
-
Robert Haas authored
Antonin Houska Discussion: http://postgr.es/m/30413.1551870730@localhost
-
Andrew Dunstan authored
The original 200 default value was set back in f425b605 when the cost delay settings were first added. Hardware has improved quite a bit since then and we've also made improvements such as sorting buffers during checkpoints (9cd00c45) which should result in less random writes. This low default value was reportedly causing problems with badly configured servers and in the absence of a native method to remove excessive bloat from tables without incurring an AccessExclusiveLock, this often made cleaning up the damage caused by badly configured auto-vacuums difficult. It seems more likely that someone will notice that auto-vacuum is running too quickly than too slowly, so let's go all out and multiple the default value for the setting by 10. With the default vacuum_cost_page_dirty and autovacuum_vacuum_cost_delay (assuming a page size of 8192 bytes), this allows autovacuum a theoretical maximum dirty write rate of around 39MB/s instead of just 3.9MB/s. Author: David Rowley Discussion: https://postgr.es/m/CAKJS1f_YbXC2qTMPyCbmsPiKvZYwpuQNQMohiRXLj1r=8_rYvw@mail.gmail.com
-
Michael Paquier authored
This was missing since 803b1301, which has introduced the option for the user-facing VACUUM and ANALYZE. Author: Masahiko Sawada Discussion: https://postgr.es/m/CAD21AoD2TMdTxRhZ7WSp940V82_OAyPmgHnbi25UbbArLgA92Q@mail.gmail.com
-
- 05 Mar, 2019 4 commits
-
-
Andrew Dunstan authored
This test fails if the containing directory contains a funny character such as a space or some perl metacharacter. To avoid that, we check for files names using readdir and a regex, rather than using a glob pattern. Discussion: https://postgr.es/m/CAM6_UM6dGdU39PKAC24T+HD9ouy0jLN9vH6163K8QEEzr__iZw@mail.gmail.com Author: Fabien COELHO Reviewed-by: Raúl Marín Rodríguez
-
Peter Eisentraut authored
The original commit appears to have accidentally introduced a duplicate definition. Keep only one of them.
-
Heikki Linnakangas authored
Scanning an index in physical order is faster than walking it in logical order, because sequential I/O is faster than random I/O. The idea and code structure is borrowed from B-tree vacuum code. Patch by Andrey Borodin, with changes by me. Based on early work by Konstantin Kuznetsov, although the patch has been rewritten multiple times since his original version. Discussion: https://www.postgresql.org/message-id/1B9FAC6F-FA19-4A24-8C1B-F4F574844892%40yandex-team.ru
-
Peter Geoghegan authored
The nbtree README claims that VACUUM can never finish interrupted page splits by design. That isn't entirely accurate, though. Note an exception to the general rule. Discussion: https://postgr.es/m/CAH2-Wz=_Xvv8byzK_LvY4ci76OgsHCQzoKF7We8yG9waO7j6rA@mail.gmail.com
-
- 04 Mar, 2019 9 commits
-
-
Andrew Dunstan authored
For some reason the dump test with names with high bits set fails on Msys2 (although not Msys1). Disable the tests for now, so that other tests can run.
-
Andrew Dunstan authored
This is the only test that fails when run as an admin user. The reason is that when Postgres is started via pg_ctl its admin privileges are lowered. However, this test called 'postgres -D datadir' directly, resulting in a failure. Replace that by calling pg_ctl and then checking the result for the expected failure, and the logfile for the expected error message.
-
Peter Geoghegan authored
Commit 2c03216d, which revamped the WAL record format, failed to update a comment referencing the old API. Update the comment.
-
Andrew Dunstan authored
Currently only frogmouth in the buildfarm uses the 32bit params, and it's not able to build past release 10, so put those last, saving substantial configure time on more modern systems. Even if we get a modern 32 bit Windows system at some stage we should probably prefer the 64 bit interface here these days.
-
Alvaro Herrera authored
Adds another introspection feature for partitioning, necessary for further psql patches. Reviewed-by: Michaël Paquier Discussion: https://postgr.es/m/20190226222757.GA31622@alvherre.pgsql
-
Alvaro Herrera authored
It's worth immortalizing this behavior, per discussion. Discussion: https://postgr.es/m/20190228193203.GA26151@alvherre.pgsql
-
Peter Eisentraut authored
This was no longer useful since the latch functions use memory barriers already, which are also compiler barriers, and volatile does not help with cross-process access. Discussion: https://www.postgresql.org/message-id/flat/20190218202511.qsfpuj5sy4dbezcw%40alap3.anarazel.de#18783c27d73e9e40009c82f6e0df0974
-
Michael Paquier authored
The implementation of readdir() in src/port/ which gets used by MSVC has been added in 399a36a7, and since the beginning it considers all errors on the first file lookup as ENOENT, setting errno accordingly and letting the routine caller think that the directory is empty. While this is normally enough for the case of the backend, this can confuse callers of this routine on Windows as all errors would map to the same behavior. So, for example, even permission errors would be thought as having an empty directory, while there could be contents in it. This commit changes the error handling so as readdir() gets a behavior similar to native implementations: force errno=0 when seeing ERROR_FILE_NOT_FOUND as error and consider other errors as plain failures. While looking at the patch, I noticed that MinGW does not enforce errno=0 when looking at the first file, but it gets enforced on the next file lookups. A comment related to that was incorrect in the code. Reported-by: Yuri Kurenkov Diagnosed-by: Yuri Kurenkov, Grigory Smolkin Author: Konstantin Knizhnik Reviewed-by: Andrew Dunstan, Michael Paquier Discussion: https://postgr.es/m/2cad7829-8d66-e39c-b937-ac825db5203d@postgrespro.ru Backpatch-through: 9.4
-
Andrew Dunstan authored
-
- 03 Mar, 2019 4 commits
-
-
Andrew Dunstan authored
The test crashes and burns quite badly, for some reason, but even if it didn't it wouldn't work, since Windows doesn't let you rename a file held by a running process.
-
Tom Lane authored
StoreIndexTuple was a loop over index_getattr, which is O(N^2) if the index columns are variable-width, and the performance impact is already quite visible at ten columns. The obvious move is to replace that with a call to index_deform_tuple ... but that's *also* a loop over index_getattr. Improve it to be essentially a clone of heap_deform_tuple. (There are a few other places that loop over all index columns with index_getattr, and perhaps should be changed likewise, but most of them don't seem performance-critical. Anyway, the rest would mostly only be interested in the index key columns, which there aren't likely to be so many of. Wide index tuples are a new thing with INCLUDE.) Konstantin Knizhnik Discussion: https://postgr.es/m/e06b2d27-04fc-5c0e-bb8c-ecd72aa24959@postgrespro.ru
-
Andrew Dunstan authored
Commit f092de05 added a test for pg_dumpall --exclude-database including the wildcard pattern '*dump*' which matches some files in the source directory. The test library on msys uses the shell which expands this and thus the program gets incorrect arguments. This doesn't happen if the pattern doesn't match any files, so here the pattern is set to '*dump_test*' which is such a pattern. Per buildfarm animal jacana.
-
Dean Rasheed authored
Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE, and to be in the same order. That was fine when the target relation was a table, but for an updatable view it could be broken in at least three different ways --- rewriteTargetListIU() could insert additional targetlist entries for view columns with defaults, the view columns could be in a different order from the columns of the underlying base relation, and targetlist entries could be merged together when assigning to elements of an array or composite type. As a result, when recursing to the base relation, the list of attribute numbers generated from the rewritten targetlist could no longer be relied upon to match the columns of the VALUES RTE. We got away with that prior to 41531e42 because it used to always be the case that rewriteValuesRTE() did nothing for the underlying base relation, since all DEFAULTS had already been replaced when it was initially invoked for the view, but that was incorrect because it failed to apply defaults from the base relation. Fix this by examining the targetlist entries more carefully and picking out just those that are simple Vars referencing the VALUES RTE. That's sufficient for the purposes of rewriteValuesRTE(), which is only responsible for dealing with DEFAULT items in the VALUES RTE. Any DEFAULT item in the VALUES RTE that doesn't have a matching simple-Var-assignment in the targetlist is an error which we complain about, but in theory that ought to be impossible. Additionally, move this code into rewriteValuesRTE() to give a clearer separation of concerns between the 2 functions. There is no need for rewriteTargetListIU() to know about the details of the VALUES RTE. While at it, fix the comment for rewriteValuesRTE() which claimed that it doesn't support array element and field assignments --- that hasn't been true since a3c7a993 (9.6 and later). Back-patch to all supported versions, with minor differences for the pre-9.6 branches, which don't support array element and field assignments to the same column in multi-row VALUES lists. Reviewed by Amit Langote. Discussion: https://postgr.es/m/15623-5d67a46788ec8b7f@postgresql.org
-
- 02 Mar, 2019 2 commits
-
-
Michael Paquier authored
This changes the partition functions so as tables and indexes which are not part of partition trees are handled the same way as what is done for undefined objects and unsupported relkinds: pg_partition_tree() returns no rows and pg_partition_root() returns a NULL result. Hence, partitioned tables, partitioned indexes and relations whose flag pg_class.relispartition is set are considered as valid objects to process. Previously, tables and indexes not included in a partition tree were processed the same way as a partition or a partitioned table, which caused the functions to return inconsistent results for inherited tables, especially when inheriting from multiple tables. Reported-by: Álvaro Herrera Author: Amit Langote, Michael Paquier Reviewed-by: Tom Lane Discussion: https://postgr.es/m/20190228193203.GA26151@alvherre.pgsql
-
Andres Freund authored
Author: Andres Freund Discussion: https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
-
- 01 Mar, 2019 9 commits
-
-
Tom Lane authored
Future-proofing against a common mistake in attempts to optimize NOT IN. We don't have such an optimization right now, but attempts to do so are in the works, and some of 'em are buggy. Add a regression test case covering the point. David Rowley Discussion: https://postgr.es/m/CAKJS1f90E9agVZryVyUpbHQbjTt5ExqS2Fsodmt5_A7E_cEyVA@mail.gmail.com
-
Tom Lane authored
In particular, make it possible to prove/refute "x IS NULL" and "x IS NOT NULL" predicates from a clause involving a ScalarArrayOpExpr even when we are unable or unwilling to deconstruct the expression into an AND/OR tree. This avoids a former unexpected degradation of plan quality when the size of an ARRAY[] expression or array constant exceeded the arbitrary MAX_SAOP_ARRAY_SIZE limit. For IS-NULL proofs, we don't really care about the values of the individual array elements; at most, we care whether there are any, and for some common cases we needn't even know that. The main user-visible effect of this is to let the optimizer recognize applicability of partial indexes with "x IS NOT NULL" predicates to queries with "x IN (array)" clauses in some cases where it previously failed to recognize that. The structure of predtest.c is such that a bunch of related proofs will now also succeed, but they're probably much less useful in the wild. James Coleman, reviewed by David Rowley Discussion: https://postgr.es/m/CAAaqYe8yKSvzbyu8w-dThRs9aTFMwrFxn_BkTYeXgjqe3CbNjg@mail.gmail.com
-
Peter Eisentraut authored
-
Andrew Dunstan authored
It turns out that different getopt implementations spell the error for missing arguments different ways. This test is of fairly marginal value, so instead of trying to keep up with the different error messages just remove the test.
-
Andres Freund authored
For the upcoming pluggable table access methods it's quite inconvenient to store tuples as HeapTuples, as that'd require converting tuples from a their native format into HeapTuples. Instead use slots to manage epq tuples. To fit into that scheme, change the foreign data wrapper callback RefetchForeignRow, to store the tuple in a slot. Insist on using the caller provided slot, so it conveniently can be stored in the corresponding EPQ slot. As there is no in core user of RefetchForeignRow, that change was done blindly, but we plan to test that soon. To avoid duplicating that work for row locks, move row locks to just directly use the EPQ slots - it previously temporarily stored tuples in LockRowsState.lr_curtuples, but that doesn't seem beneficial, given we'd possibly end up with a significant number of additional slots. The behaviour of es_epqTupleSet[rti -1] is now checked by es_epqTupleSlot[rti -1] != NULL, as that is distinguishable from a slot containing an empty tuple. Author: Andres Freund, Haribabu Kommi, Ashutosh Bapat Discussion: https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
-
Andrew Dunstan authored
Headings are added for the User Configurations and Databases sections, and for each user configuration and database in the output. Author: Fabien Coelho Discussion: https://postgr.es/m/alpine.DEB.2.21.1812272222130.32444@lancre
-
Andrew Dunstan authored
This option functions similarly to pg_dump's --exclude-table option, but for database names. The option can be given once, and the argument can be a pattern including wildcard characters. Author: Andrew Dunstan. Reviewd-by: Fabien Coelho and Michael Paquier Discussion: https://postgr.es/m/43a54a47-4aa7-c70e-9ca6-648f436dd6e6@2ndQuadrant.com
-
Amit Kapila authored
After commit b0eaa4c5, we use a local map of pages to find the required space for small relations. We do clear this map when we have found a block with enough free space, when we extend the relation, or on transaction abort so that it can be used next time. However, we miss to clear it when we didn't find any pages to try from the map which leads to an assertion failure when we later tried to use it after relation extension. In the passing, I have improved some comments in this area. Reported-by: Tom Lane based on buildfarm results Author: Amit Kapila Reviewed-by: John Naylor Tested-by: Kuntal Ghosh Discussion: https://postgr.es/m/32368.1551114120@sss.pgh.pa.us
-
Michael Paquier authored
The function was tweaked so as it returned one row full of NULLs when working on an unsupported relkind or an undefined object as of cc53123b, and after discussion with Amit and Álvaro it looks more natural to make it return no rows. Author: Michael Paquier Reviewed-by: Álvaro Herrera, Amit Langote Discussion: https://postgr.es/m/20190227184808.GA17357@alvherre.pgsql
-
- 28 Feb, 2019 6 commits
-
-
Andres Freund authored
Previously that was needed to safely store the table oid, but after b8d71745 that's not necessary anymore. Author: Andres Freund
-
Andres Freund authored
After 5408e233 it's not necessary to force materializing the target slot when copying from one buffer slot to another. Previously that was required because the HeapTupleData portion of the source slot wasn't guaranteed to stay valid long enough, but now we can simply copy that part into the destination slot's tupdata. Author: Andres Freund
-
Alvaro Herrera authored
Discussion: https://postgr.es/m/20190220173815.GA7959@alvherre.pgsql Reviewed-by: Robert Haas
-
Joe Conway authored
When backend functions were added to expose controldata via SQL, reading of pg_control was consolidated under src/common so that both frontend and backend could share the same code. That move from frontend-only to shared frontend-backend failed to recognize the risk (and coding standards violation) of using a bare open(). In particular, it risked leaking file descriptors if transient errors occurred while reading the file. Fix that by using OpenTransientFile() instead in the backend case, which is purpose-built for this type of usage. Since there have been no complaints from the field, and an intermittent failure low risk, no backpatch. Hard failure would of course be bad, but in that case these functions are probably the least of your worries. Author: Joe Conway Reviewed-By: Michael Paquier Reported by: Michael Paquier Discussion: https://postgr.es/m/20190227074728.GA15710@paquier.xyz
-
Andres Freund authored
While not common, it can be useful to store a virtual tuple into a buffer tuple table slot, and then materialize that slot. So far we've asserted out, which surprisingly wasn't a problem for anything in core. But that seems fragile, and it also breaks redis_fdw after ff11e7f4. Thus, allow materializing a virtual tuple stored in a buffer tuple table slot. Author: Andres Freund Discussion: https://postgr.es/m/20190227181621.xholonj7ff7ohxsg@alap3.anarazel.de https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
-
Alvaro Herrera authored
Commit f831d4ac changed what pg_dump emits for some empty fields: they were output as empty strings before, NULL pointer afterwards. That makes old pg_restore unable to work (crash) with such files, which is unacceptable. Return to the original representation by explicitly setting those struct members to "" where needed; remove some no longer needed checks for NULL input. We can declutter the code a little by returning to NULLs when we next update the archive version, so add a note to remind us later. Discussion: https://postgr.es/m/20190225074539.az6j3u464cvsoxh6@depesz.com Reported-by: hubert depesz lubaczewski Author: Dmitry Dolgov
-