Commit fd31cd26 authored by Robert Haas's avatar Robert Haas

Don't vacuum all-frozen pages.

Commit a892234f gave us enough
infrastructure to avoid vacuuming pages where every tuple on the
page is already frozen.  So, replace the notion of a scan_all or
whole-table vacuum with the less onerous notion of an "aggressive"
vacuum, which will pages that are all-visible, but still skip those
that are all-frozen.

This should greatly reduce the cost of anti-wraparound vacuuming
on large clusters where the majority of data is never touched
between one cycle and the next, because we'll no longer have to
read all of those pages only to find out that we don't need to
do anything with them.

Patch by me, reviewed by Masahiko Sawada.
parent 364a9f47
...@@ -5984,12 +5984,15 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv; ...@@ -5984,12 +5984,15 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</term> </term>
<listitem> <listitem>
<para> <para>
<command>VACUUM</> performs a whole-table scan if the table's <command>VACUUM</> performs an aggressive scan if the table's
<structname>pg_class</>.<structfield>relfrozenxid</> field has reached <structname>pg_class</>.<structfield>relfrozenxid</> field has reached
the age specified by this setting. The default is 150 million the age specified by this setting. An aggressive scan differs from
transactions. Although users can set this value anywhere from zero to a regular <command>VACUUM</> in that it visits every page that might
two billions, <command>VACUUM</> will silently limit the effective value contain unfrozen XIDs or MXIDs, not just those that might contain dead
to 95% of <xref linkend="guc-autovacuum-freeze-max-age">, so that a tuples. The default is 150 million transactions. Although users can
set this value anywhere from zero to two billions, <command>VACUUM</>
will silently limit the effective value to 95% of
<xref linkend="guc-autovacuum-freeze-max-age">, so that a
periodical manual <command>VACUUM</> has a chance to run before an periodical manual <command>VACUUM</> has a chance to run before an
anti-wraparound autovacuum is launched for the table. For more anti-wraparound autovacuum is launched for the table. For more
information see information see
...@@ -6028,9 +6031,12 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv; ...@@ -6028,9 +6031,12 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</term> </term>
<listitem> <listitem>
<para> <para>
<command>VACUUM</> performs a whole-table scan if the table's <command>VACUUM</> performs an aggressive scan if the table's
<structname>pg_class</>.<structfield>relminmxid</> field has reached <structname>pg_class</>.<structfield>relminmxid</> field has reached
the age specified by this setting. The default is 150 million multixacts. the age specified by this setting. An aggressive scan differs from
a regular <command>VACUUM</> in that it visits every page that might
contain unfrozen XIDs or MXIDs, not just those that might contain dead
tuples. The default is 150 million multixacts.
Although users can set this value anywhere from zero to two billions, Although users can set this value anywhere from zero to two billions,
<command>VACUUM</> will silently limit the effective value to 95% of <command>VACUUM</> will silently limit the effective value to 95% of
<xref linkend="guc-autovacuum-multixact-freeze-max-age">, so that a <xref linkend="guc-autovacuum-multixact-freeze-max-age">, so that a
......
...@@ -438,22 +438,27 @@ ...@@ -438,22 +438,27 @@
</para> </para>
<para> <para>
<command>VACUUM</> normally skips pages that don't have any dead row <command>VACUUM</> uses the <link linkend="storage-vm">visibility map</>
versions, but those pages might still have row versions with old XID to determine which pages of a relation must be scanned. Normally, it
values. To ensure all old row versions have been frozen, a will skips pages that don't have any dead row versions even if those pages
scan of the whole table is needed. might still have row versions with old XID values. Therefore, normal
<xref linkend="guc-vacuum-freeze-table-age"> controls when scans won't succeed in freezing every row version in the table.
<command>VACUUM</> does that: a whole table sweep is forced if Periodically, <command>VACUUM</> will perform an <firstterm>aggressive
the table hasn't been fully scanned for <varname>vacuum_freeze_table_age</> vacuum</>, skipping only those pages which contain neither dead rows nor
minus <varname>vacuum_freeze_min_age</> transactions. Setting it to 0 any unfrozen XID or MXID values.
forces <command>VACUUM</> to always scan all pages, effectively ignoring <xref linkend="guc-vacuum-freeze-table-age">
the visibility map. controls when <command>VACUUM</> does that: all-visible but not all-frozen
pages are scanned if the number of transactions that have passed since the
last such scan is greater than <varname>vacuum_freeze_table_age</> minus
<varname>vacuum_freeze_min_age</>. Setting
<varname>vacuum_freeze_table_age</> to 0 forces <command>VACUUM</> to
use this more aggressive strategy for all scans.
</para> </para>
<para> <para>
The maximum time that a table can go unvacuumed is two billion The maximum time that a table can go unvacuumed is two billion
transactions minus the <varname>vacuum_freeze_min_age</> value at transactions minus the <varname>vacuum_freeze_min_age</> value at
the time <command>VACUUM</> last scanned the whole table. If it were to go the time of the last aggressive vacuum. If it were to go
unvacuumed for longer than unvacuumed for longer than
that, data loss could result. To ensure that this does not happen, that, data loss could result. To ensure that this does not happen,
autovacuum is invoked on any table that might contain unfrozen rows with autovacuum is invoked on any table that might contain unfrozen rows with
...@@ -491,7 +496,7 @@ ...@@ -491,7 +496,7 @@
normal delete and update activity is run in that window. Setting it too normal delete and update activity is run in that window. Setting it too
close could lead to anti-wraparound autovacuums, even though the table close could lead to anti-wraparound autovacuums, even though the table
was recently vacuumed to reclaim space, whereas lower values lead to more was recently vacuumed to reclaim space, whereas lower values lead to more
frequent whole-table scans. frequent aggressive vacuuming.
</para> </para>
<para> <para>
...@@ -527,7 +532,7 @@ ...@@ -527,7 +532,7 @@
<structname>pg_database</>. In particular, <structname>pg_database</>. In particular,
the <structfield>relfrozenxid</> column of a table's the <structfield>relfrozenxid</> column of a table's
<structname>pg_class</> row contains the freeze cutoff XID that was used <structname>pg_class</> row contains the freeze cutoff XID that was used
by the last whole-table <command>VACUUM</> for that table. All rows by the last aggressive <command>VACUUM</> for that table. All rows
inserted by transactions with XIDs older than this cutoff XID are inserted by transactions with XIDs older than this cutoff XID are
guaranteed to have been frozen. Similarly, guaranteed to have been frozen. Similarly,
the <structfield>datfrozenxid</> column of a database's the <structfield>datfrozenxid</> column of a database's
...@@ -552,20 +557,23 @@ SELECT datname, age(datfrozenxid) FROM pg_database; ...@@ -552,20 +557,23 @@ SELECT datname, age(datfrozenxid) FROM pg_database;
</para> </para>
<para> <para>
<command>VACUUM</> normally <command>VACUUM</> normally only scans pages that have been modified
only scans pages that have been modified since the last vacuum, but since the last vacuum, but <structfield>relfrozenxid</> can only be
<structfield>relfrozenxid</> can only be advanced when the whole table is advanced when every page of the table
scanned. The whole table is scanned when <structfield>relfrozenxid</> is that might contain unfrozen XIDs is scanned. This happens when
more than <varname>vacuum_freeze_table_age</> transactions old, when <structfield>relfrozenxid</> is more than
<command>VACUUM</>'s <literal>FREEZE</> option is used, or when all pages <varname>vacuum_freeze_table_age</> transactions old, when
happen to <command>VACUUM</>'s <literal>FREEZE</> option is used, or when all
pages that are not already all-frozen happen to
require vacuuming to remove dead row versions. When <command>VACUUM</> require vacuuming to remove dead row versions. When <command>VACUUM</>
scans the whole table, after it's finished <literal>age(relfrozenxid)</> scans every page in the table that is not already all-frozen, it should
should be a little more than the <varname>vacuum_freeze_min_age</> setting set <literal>age(relfrozenxid)</> to a value just a little more than the
that was used (more by the number of transactions started since the <varname>vacuum_freeze_min_age</> setting
<command>VACUUM</> started). If no whole-table-scanning <command>VACUUM</> that was used (more by the number of transcations started since the
is issued on the table until <varname>autovacuum_freeze_max_age</> is <command>VACUUM</> started). If no <structfield>relfrozenxid</>-advancing
reached, an autovacuum will soon be forced for the table. <command>VACUUM</> is issued on the table until
<varname>autovacuum_freeze_max_age</> is reached, an autovacuum will soon
be forced for the table.
</para> </para>
<para> <para>
...@@ -634,21 +642,23 @@ HINT: Stop the postmaster and vacuum that database in single-user mode. ...@@ -634,21 +642,23 @@ HINT: Stop the postmaster and vacuum that database in single-user mode.
</para> </para>
<para> <para>
During a <command>VACUUM</> table scan, either partial or of the whole Whenever <command>VACUUM</> scans any part of a table, it will replace
table, any multixact ID older than any multixact ID it encounters which is older than
<xref linkend="guc-vacuum-multixact-freeze-min-age"> <xref linkend="guc-vacuum-multixact-freeze-min-age">
is replaced by a different value, which can be the zero value, a single by a different value, which can be the zero value, a single
transaction ID, or a newer multixact ID. For each table, transaction ID, or a newer multixact ID. For each table,
<structname>pg_class</>.<structfield>relminmxid</> stores the oldest <structname>pg_class</>.<structfield>relminmxid</> stores the oldest
possible multixact ID still appearing in any tuple of that table. possible multixact ID still appearing in any tuple of that table.
If this value is older than If this value is older than
<xref linkend="guc-vacuum-multixact-freeze-table-age">, a whole-table <xref linkend="guc-vacuum-multixact-freeze-table-age">, an aggressive
scan is forced. <function>mxid_age()</> can be used on vacuum is forced. As discussed in the previous section, an aggressive
vacuum means that only those pages which are known to be all-frozen will
be skipped. <function>mxid_age()</> can be used on
<structname>pg_class</>.<structfield>relminmxid</> to find its age. <structname>pg_class</>.<structfield>relminmxid</> to find its age.
</para> </para>
<para> <para>
Whole-table <command>VACUUM</> scans, regardless of Aggressive <command>VACUUM</> scans, regardless of
what causes them, enable advancing the value for that table. what causes them, enable advancing the value for that table.
Eventually, as all tables in all databases are scanned and their Eventually, as all tables in all databases are scanned and their
oldest multixact values are advanced, on-disk storage for older oldest multixact values are advanced, on-disk storage for older
...@@ -656,13 +666,13 @@ HINT: Stop the postmaster and vacuum that database in single-user mode. ...@@ -656,13 +666,13 @@ HINT: Stop the postmaster and vacuum that database in single-user mode.
</para> </para>
<para> <para>
As a safety device, a whole-table vacuum scan will occur for any table As a safety device, an aggressive vacuum scan will occur for any table
whose multixact-age is greater than whose multixact-age is greater than
<xref linkend="guc-autovacuum-multixact-freeze-max-age">. Whole-table <xref linkend="guc-autovacuum-multixact-freeze-max-age">. Aggressive
vacuum scans will also occur progressively for all tables, starting with vacuum scans will also occur progressively for all tables, starting with
those that have the oldest multixact-age, if the amount of used member those that have the oldest multixact-age, if the amount of used member
storage space exceeds the amount 50% of the addressable storage space. storage space exceeds the amount 50% of the addressable storage space.
Both of these kinds of whole-table scans will occur even if autovacuum is Both of these kinds of aggressive scans will occur even if autovacuum is
nominally disabled. nominally disabled.
</para> </para>
</sect3> </sect3>
...@@ -743,9 +753,9 @@ vacuum threshold = vacuum base threshold + vacuum scale factor * number of tuple ...@@ -743,9 +753,9 @@ vacuum threshold = vacuum base threshold + vacuum scale factor * number of tuple
<command>UPDATE</command> and <command>DELETE</command> operation. (It <command>UPDATE</command> and <command>DELETE</command> operation. (It
is only semi-accurate because some information might be lost under heavy is only semi-accurate because some information might be lost under heavy
load.) If the <structfield>relfrozenxid</> value of the table is more load.) If the <structfield>relfrozenxid</> value of the table is more
than <varname>vacuum_freeze_table_age</> transactions old, the whole than <varname>vacuum_freeze_table_age</> transactions old, an aggressive
table is scanned to freeze old tuples and advance vacuum is performed to freeze old tuples and advance
<structfield>relfrozenxid</>, otherwise only pages that have been modified <structfield>relfrozenxid</>; otherwise, only pages that have been modified
since the last vacuum are scanned. since the last vacuum are scanned.
</para> </para>
......
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment