Commit cd5e8225 authored by Noah Misch's avatar Noah Misch

Change XID and mxact limits to warn at 40M and stop at 3M.

We have edge-case bugs when assigning values in the last few dozen pages
before the wrap limit.  We may introduce similar bugs in the future.  At
default BLCKSZ, this makes such bugs unreachable outside of single-user
mode.  Also, when VACUUM began to consume mxacts, multiStopLimit did not
change to compensate.

pg_upgrade may fail on a cluster that was already printing "must be
vacuumed" warnings.  Follow the warning's instructions to clear the
warning, then run pg_upgrade again.  One can still, peacefully consume
98% of XIDs or mxacts, so DBAs need not change routine VACUUM settings.

Discussion: https://postgr.es/m/20200621083513.GA3074645@rfd.leadboat.com
parent 9f968278
...@@ -608,10 +608,10 @@ SELECT datname, age(datfrozenxid) FROM pg_database; ...@@ -608,10 +608,10 @@ SELECT datname, age(datfrozenxid) FROM pg_database;
<para> <para>
If for some reason autovacuum fails to clear old XIDs from a table, the If for some reason autovacuum fails to clear old XIDs from a table, the
system will begin to emit warning messages like this when the database's system will begin to emit warning messages like this when the database's
oldest XIDs reach eleven million transactions from the wraparound point: oldest XIDs reach forty million transactions from the wraparound point:
<programlisting> <programlisting>
WARNING: database "mydb" must be vacuumed within 10985967 transactions WARNING: database "mydb" must be vacuumed within 39985967 transactions
HINT: To avoid a database shutdown, execute a database-wide VACUUM in that database. HINT: To avoid a database shutdown, execute a database-wide VACUUM in that database.
</programlisting> </programlisting>
...@@ -621,7 +621,7 @@ HINT: To avoid a database shutdown, execute a database-wide VACUUM in that data ...@@ -621,7 +621,7 @@ HINT: To avoid a database shutdown, execute a database-wide VACUUM in that data
be able to advance the database's <structfield>datfrozenxid</structfield>.) be able to advance the database's <structfield>datfrozenxid</structfield>.)
If these warnings are If these warnings are
ignored, the system will shut down and refuse to start any new ignored, the system will shut down and refuse to start any new
transactions once there are fewer than 1 million transactions left transactions once there are fewer than three million transactions left
until wraparound: until wraparound:
<programlisting> <programlisting>
...@@ -629,7 +629,7 @@ ERROR: database is not accepting commands to avoid wraparound data loss in data ...@@ -629,7 +629,7 @@ ERROR: database is not accepting commands to avoid wraparound data loss in data
HINT: Stop the postmaster and vacuum that database in single-user mode. HINT: Stop the postmaster and vacuum that database in single-user mode.
</programlisting> </programlisting>
The 1-million-transaction safety margin exists to let the The three-million-transaction safety margin exists to let the
administrator recover without data loss, by manually executing the administrator recover without data loss, by manually executing the
required <command>VACUUM</command> commands. However, since the system will not required <command>VACUUM</command> commands. However, since the system will not
execute commands once it has gone into the safety shutdown mode, execute commands once it has gone into the safety shutdown mode,
......
...@@ -2217,28 +2217,24 @@ SetMultiXactIdLimit(MultiXactId oldest_datminmxid, Oid oldest_datoid, ...@@ -2217,28 +2217,24 @@ SetMultiXactIdLimit(MultiXactId oldest_datminmxid, Oid oldest_datoid,
multiWrapLimit += FirstMultiXactId; multiWrapLimit += FirstMultiXactId;
/* /*
* We'll refuse to continue assigning MultiXactIds once we get within 100 * We'll refuse to continue assigning MultiXactIds once we get within 3M
* multi of data loss. * multi of data loss. See SetTransactionIdLimit.
*
* Note: This differs from the magic number used in
* SetTransactionIdLimit() since vacuum itself will never generate new
* multis. XXX actually it does, if it needs to freeze old multis.
*/ */
multiStopLimit = multiWrapLimit - 100; multiStopLimit = multiWrapLimit - 3000000;
if (multiStopLimit < FirstMultiXactId) if (multiStopLimit < FirstMultiXactId)
multiStopLimit -= FirstMultiXactId; multiStopLimit -= FirstMultiXactId;
/* /*
* We'll start complaining loudly when we get within 10M multis of the * We'll start complaining loudly when we get within 40M multis of data
* stop point. This is kind of arbitrary, but if you let your gas gauge * loss. This is kind of arbitrary, but if you let your gas gauge get
* get down to 1% of full, would you be looking for the next gas station? * down to 2% of full, would you be looking for the next gas station? We
* We need to be fairly liberal about this number because there are lots * need to be fairly liberal about this number because there are lots of
* of scenarios where most transactions are done by automatic clients that * scenarios where most transactions are done by automatic clients that
* won't pay attention to warnings. (No, we're not gonna make this * won't pay attention to warnings. (No, we're not gonna make this
* configurable. If you know enough to configure it, you know enough to * configurable. If you know enough to configure it, you know enough to
* not get in this kind of trouble in the first place.) * not get in this kind of trouble in the first place.)
*/ */
multiWarnLimit = multiStopLimit - 10000000; multiWarnLimit = multiWrapLimit - 40000000;
if (multiWarnLimit < FirstMultiXactId) if (multiWarnLimit < FirstMultiXactId)
multiWarnLimit -= FirstMultiXactId; multiWarnLimit -= FirstMultiXactId;
......
...@@ -350,27 +350,30 @@ SetTransactionIdLimit(TransactionId oldest_datfrozenxid, Oid oldest_datoid) ...@@ -350,27 +350,30 @@ SetTransactionIdLimit(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
/* /*
* We'll refuse to continue assigning XIDs in interactive mode once we get * We'll refuse to continue assigning XIDs in interactive mode once we get
* within 1M transactions of data loss. This leaves lots of room for the * within 3M transactions of data loss. This leaves lots of room for the
* DBA to fool around fixing things in a standalone backend, while not * DBA to fool around fixing things in a standalone backend, while not
* being significant compared to total XID space. (Note that since * being significant compared to total XID space. (Note that since
* vacuuming requires one transaction per table cleaned, we had better be * vacuuming requires one transaction per table cleaned, we had better be
* sure there's lots of XIDs left...) * sure there's lots of XIDs left...) Also, at default BLCKSZ, this
* leaves two completely-idle segments. In the event of edge-case bugs
* involving page or segment arithmetic, idle segments render the bugs
* unreachable outside of single-user mode.
*/ */
xidStopLimit = xidWrapLimit - 1000000; xidStopLimit = xidWrapLimit - 3000000;
if (xidStopLimit < FirstNormalTransactionId) if (xidStopLimit < FirstNormalTransactionId)
xidStopLimit -= FirstNormalTransactionId; xidStopLimit -= FirstNormalTransactionId;
/* /*
* We'll start complaining loudly when we get within 10M transactions of * We'll start complaining loudly when we get within 40M transactions of
* the stop point. This is kind of arbitrary, but if you let your gas * data loss. This is kind of arbitrary, but if you let your gas gauge
* gauge get down to 1% of full, would you be looking for the next gas * get down to 2% of full, would you be looking for the next gas station?
* station? We need to be fairly liberal about this number because there * We need to be fairly liberal about this number because there are lots
* are lots of scenarios where most transactions are done by automatic * of scenarios where most transactions are done by automatic clients that
* clients that won't pay attention to warnings. (No, we're not gonna make * won't pay attention to warnings. (No, we're not gonna make this
* this configurable. If you know enough to configure it, you know enough * configurable. If you know enough to configure it, you know enough to
* to not get in this kind of trouble in the first place.) * not get in this kind of trouble in the first place.)
*/ */
xidWarnLimit = xidStopLimit - 10000000; xidWarnLimit = xidWrapLimit - 40000000;
if (xidWarnLimit < FirstNormalTransactionId) if (xidWarnLimit < FirstNormalTransactionId)
xidWarnLimit -= FirstNormalTransactionId; xidWarnLimit -= FirstNormalTransactionId;
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment