Commit 5374d097 authored by Tom Lane's avatar Tom Lane

Change planner to use the current true disk file size as its estimate of

a relation's number of blocks, rather than the possibly-obsolete value
in pg_class.relpages.  Scale the value in pg_class.reltuples correspondingly
to arrive at a hopefully more accurate number of rows.  When pg_class
contains 0/0, estimate a tuple width from the column datatypes and divide
that into current file size to estimate number of rows.  This improved
methodology allows us to jettison the ancient hacks that put bogus default
values into pg_class when a table is first created.  Also, per a suggestion
from Simon, make VACUUM (but not VACUUM FULL or ANALYZE) adjust the value
it puts into pg_class.reltuples to try to represent the mean tuple density
instead of the minimal density that actually prevails just after VACUUM.
These changes alter the plans selected for certain regression tests, so
update the expected files accordingly.  (I removed join_1.out because
it's not clear if it still applies; we can add back any variant versions
as they are shown to be needed.)
parent b7bcea64
<!--
Documentation of the system catalogs, directed toward PostgreSQL developers
$PostgreSQL: pgsql/doc/src/sgml/catalogs.sgml,v 2.92 2004/11/15 06:32:13 neilc Exp $
$PostgreSQL: pgsql/doc/src/sgml/catalogs.sgml,v 2.93 2004/12/01 19:00:27 tgl Exp $
-->
<chapter id="catalogs">
......@@ -1079,11 +1079,12 @@
<entry><type>int4</type></entry>
<entry></entry>
<entry>
Size of the on-disk representation of this table in pages (size
Size of the on-disk representation of this table in pages (of size
<symbol>BLCKSZ</symbol>).
This is only an estimate used by the planner.
It is updated by <command>VACUUM</command>,
<command>ANALYZE</command>, and <command>CREATE INDEX</command>.
<command>ANALYZE</command>, and a few DDL commands
such as <command>CREATE INDEX</command>.
</entry>
</row>
......@@ -1095,7 +1096,8 @@
Number of rows in the table.
This is only an estimate used by the planner.
It is updated by <command>VACUUM</command>,
<command>ANALYZE</command>, and <command>CREATE INDEX</command>.
<command>ANALYZE</command>, and a few DDL commands
such as <command>CREATE INDEX</command>.
</entry>
</row>
......
<!--
$PostgreSQL: pgsql/doc/src/sgml/diskusage.sgml,v 1.11 2004/06/21 04:06:03 tgl Exp $
$PostgreSQL: pgsql/doc/src/sgml/diskusage.sgml,v 1.12 2004/12/01 19:00:27 tgl Exp $
-->
<chapter id="diskusage">
......@@ -7,11 +7,7 @@ $PostgreSQL: pgsql/doc/src/sgml/diskusage.sgml,v 1.11 2004/06/21 04:06:03 tgl Ex
<para>
This chapter discusses how to monitor the disk usage of a
<productname>PostgreSQL</> database system. In the current
release, the database administrator does not have much control over
the on-disk storage layout, so this chapter is mostly informative
and can give you some ideas how to manage the disk usage with
operating system tools.
<productname>PostgreSQL</> database system.
</para>
<sect1 id="disk-usage">
......@@ -23,11 +19,12 @@ $PostgreSQL: pgsql/doc/src/sgml/diskusage.sgml,v 1.11 2004/06/21 04:06:03 tgl Ex
<para>
Each table has a primary heap disk file where most of the data is
stored. To store long column values, there is also a
<acronym>TOAST</> file associated with the table, named based on the
table's OID (actually <literal>pg_class.relfilenode</>), and an index on the
<acronym>TOAST</> table. There also may be indexes associated with
the base table.
stored. If the table has any columns with potentially-wide values,
there is also a <acronym>TOAST</> file associated with the table,
which is used to store values too wide to fit comfortably in the main
table. There will be one index on the
<acronym>TOAST</> table, if present. There may also be indexes associated
with the base table.
</para>
<para>
......@@ -45,18 +42,24 @@ SELECT relfilenode, relpages FROM pg_class WHERE relname = 'customer';
16806 | 60
(1 row)
</programlisting>
Each page is typically 8 kilobytes. (Remember, <literal>relpages</>
is only updated by <command>VACUUM</> and <command>ANALYZE</>.)
Each page is typically 8 kilobytes. (Remember, <structfield>relpages</>
is only updated by <command>VACUUM</>, <command>ANALYZE</>, and
a few DDL commands such as <command>CREATE INDEX</>.) The
<structfield>relfilenode</> value is of interest if you want to examine
the table's disk file directly.
</para>
<para>
To show the space used by <acronym>TOAST</> tables, use a query
like the following, substituting the <literal>relfilenode</literal>
number of the heap (determined by the query above):
like the following:
<programlisting>
SELECT relname, relpages
FROM pg_class
WHERE relname = 'pg_toast_16806' OR relname = 'pg_toast_16806_index'
FROM pg_class,
(SELECT reltoastrelid FROM pg_class
WHERE relname = 'customer') ss
WHERE oid = ss.reltoastrelid
OR oid = (SELECT reltoastidxid FROM pg_class
WHERE oid = ss.reltoastrelid)
ORDER BY relname;
relname | relpages
......@@ -74,7 +77,7 @@ SELECT c2.relname, c2.relpages
WHERE c.relname = 'customer'
AND c.oid = i.indrelid
AND c2.oid = i.indexrelid
ORDER BY c2.relname;
ORDER BY c2.relname;
relname | relpages
----------------------+----------
......@@ -113,13 +116,10 @@ SELECT relname, relpages FROM pg_class ORDER BY relpages DESC;
<para>
The most important disk monitoring task of a database administrator
is to make sure the disk doesn't grow full. A filled data disk may
result in subsequent corruption of database indexes, but not of the
tables themselves. If the WAL files are on the same disk (as
is the case for a default configuration) then a filled disk during
database initialization may result in corrupted or incomplete WAL
files. This failure condition is detected and the database server
will refuse to start up.
is to make sure the disk doesn't grow full. A filled data disk will
not result in data corruption, but it may well prevent useful activity
from occurring. If the disk holding the WAL files grows full, database
server panic and consequent shutdown may occur.
</para>
<para>
......
<!--
$PostgreSQL: pgsql/doc/src/sgml/perform.sgml,v 1.47 2004/11/15 06:32:14 neilc Exp $
$PostgreSQL: pgsql/doc/src/sgml/perform.sgml,v 1.48 2004/12/01 19:00:27 tgl Exp $
-->
<chapter id="performance-tips">
......@@ -389,14 +389,15 @@ SELECT relname, relkind, reltuples, relpages FROM pg_class WHERE relname LIKE 't
<para>
For efficiency reasons, <structfield>reltuples</structfield>
and <structfield>relpages</structfield> are not updated on-the-fly,
and so they usually contain only approximate values (which is good
enough for the planner's purposes). They are initialized with dummy
values (presently 1000 and 10 respectively) when a table is created.
They are updated by certain commands, presently <command>VACUUM</>,
<command>ANALYZE</>, and <command>CREATE INDEX</>. A stand-alone
and so they usually contain somewhat out-of-date values.
They are updated by <command>VACUUM</>, <command>ANALYZE</>, and a
few DDL commands such as <command>CREATE INDEX</>. A stand-alone
<command>ANALYZE</>, that is one not part of <command>VACUUM</>,
generates an approximate <structfield>reltuples</structfield> value
since it does not read every row of the table.
since it does not read every row of the table. The planner
will scale the values it finds in <structname>pg_class</structname>
to match the current physical table size, thus obtaining a closer
approximation.
</para>
<indexterm>
......
<!--
$PostgreSQL: pgsql/doc/src/sgml/release.sgml,v 1.314 2004/11/27 21:56:04 petere Exp $
$PostgreSQL: pgsql/doc/src/sgml/release.sgml,v 1.315 2004/12/01 19:00:27 tgl Exp $
-->
<appendix id="release">
......@@ -527,6 +527,24 @@ $PostgreSQL: pgsql/doc/src/sgml/release.sgml,v 1.314 2004/11/27 21:56:04 petere
</para>
</listitem>
<listitem>
<para>
Use dynamically-generated table size estimates while planning (Tom)
</para>
<para>
The optimizer now uses a table's current actual size on disk as its
estimate of the number of blocks in the table, and it makes an estimate
of the number of rows in the table based on the current size on disk.
Formerly, the
<structname>pg_class</structname>.<structfield>relpages</structfield>
and
<structname>pg_class</structname>.<structfield>reltuples</structfield>
fields were used as-is, but these values might be quite out-of-date,
leading to poor choices of plans. They are now treated only as an
indication of the table's density (rows per page).
</para>
</listitem>
<listitem>
<para>
Improved index usage with <literal>OR</> clauses (Tom)
......
......@@ -12,7 +12,7 @@
* Portions Copyright (c) 1994, Regents of the University of California
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/access/nbtree/nbtree.c,v 1.122 2004/11/17 03:13:38 neilc Exp $
* $PostgreSQL: pgsql/src/backend/access/nbtree/nbtree.c,v 1.123 2004/12/01 19:00:37 tgl Exp $
*
*-------------------------------------------------------------------------
*/
......@@ -785,6 +785,10 @@ btvacuumcleanup(PG_FUNCTION_ARGS)
* Do the physical truncation.
*/
RelationTruncate(rel, new_pages);
/* update statistics */
stats->pages_removed = num_pages - new_pages;
num_pages = new_pages;
}
}
......
......@@ -8,7 +8,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/catalog/heap.c,v 1.276 2004/08/31 17:10:36 tgl Exp $
* $PostgreSQL: pgsql/src/backend/catalog/heap.c,v 1.277 2004/12/01 19:00:39 tgl Exp $
*
*
* INTERFACE ROUTINES
......@@ -607,37 +607,22 @@ AddNewRelationTuple(Relation pg_class_desc,
*/
new_rel_reltup = new_rel_desc->rd_rel;
/*
* Here we insert bogus estimates of the size of the new relation. In
* reality, of course, the new relation has 0 tuples and pages, and if
* we were tracking these statistics accurately then we'd set the
* fields that way. But at present the stats will be updated only by
* VACUUM or CREATE INDEX, and the user might insert a lot of tuples
* before he gets around to doing either of those. So, instead of
* saying the relation is empty, we insert guesstimates. The point is
* to keep the optimizer from making really stupid choices on
* never-yet-vacuumed tables; so the estimates need only be large
* enough to discourage the optimizer from using nested-loop plans.
* With this hack, nested-loop plans will be preferred only after the
* table has been proven to be small by VACUUM or CREATE INDEX.
* Maintaining the stats on-the-fly would solve the problem more
* cleanly, but the overhead of that would likely cost more than it'd
* save. (NOTE: CREATE INDEX inserts the same bogus estimates if it
* finds the relation has 0 rows and pages. See index.c.)
*/
switch (relkind)
{
case RELKIND_RELATION:
case RELKIND_INDEX:
case RELKIND_TOASTVALUE:
new_rel_reltup->relpages = 10; /* bogus estimates */
new_rel_reltup->reltuples = 1000;
/* The relation is real, but as yet empty */
new_rel_reltup->relpages = 0;
new_rel_reltup->reltuples = 0;
break;
case RELKIND_SEQUENCE:
/* Sequences always have a known size */
new_rel_reltup->relpages = 1;
new_rel_reltup->reltuples = 1;
break;
default: /* views, etc */
default:
/* Views, etc, have no disk storage */
new_rel_reltup->relpages = 0;
new_rel_reltup->reltuples = 0;
break;
......
......@@ -8,7 +8,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/catalog/index.c,v 1.241 2004/10/15 22:39:53 tgl Exp $
* $PostgreSQL: pgsql/src/backend/catalog/index.c,v 1.242 2004/12/01 19:00:39 tgl Exp $
*
*
* INTERFACE ROUTINES
......@@ -33,20 +33,15 @@
#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_index.h"
#include "catalog/pg_opclass.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
#include "executor/executor.h"
#include "miscadmin.h"
#include "optimizer/clauses.h"
#include "optimizer/prep.h"
#include "parser/parse_expr.h"
#include "parser/parse_func.h"
#include "storage/sinval.h"
#include "storage/smgr.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
......@@ -54,14 +49,6 @@
#include "utils/syscache.h"
/*
* macros used in guessing how many tuples are on a page.
*/
#define AVG_ATTR_SIZE 8
#define NTUPLES_PER_PAGE(natts) \
((BLCKSZ - MAXALIGN(sizeof(PageHeaderData))) / \
((natts) * AVG_ATTR_SIZE + MAXALIGN(sizeof(HeapTupleHeaderData))))
/* non-export function prototypes */
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
IndexInfo *indexInfo,
......@@ -1153,6 +1140,8 @@ setNewRelfilenode(Relation relation)
/* update the pg_class row */
rd_rel->relfilenode = newrelfilenode;
rd_rel->relpages = 0; /* it's empty until further notice */
rd_rel->reltuples = 0;
simple_heap_update(pg_class, &tuple->t_self, tuple);
CatalogUpdateIndexes(pg_class, tuple);
......@@ -1170,7 +1159,7 @@ setNewRelfilenode(Relation relation)
*
* Update pg_class' relpages and reltuples statistics for the given relation
* (which can be either a table or an index). Note that this is not used
* in the context of VACUUM.
* in the context of VACUUM, only CREATE INDEX.
* ----------------
*/
void
......@@ -1209,7 +1198,8 @@ UpdateStats(Oid relid, double reltuples)
* Find the tuple to update in pg_class. Normally we make a copy of
* the tuple using the syscache, modify it, and apply heap_update. But
* in bootstrap mode we can't use heap_update, so we cheat and
* overwrite the tuple in-place.
* overwrite the tuple in-place. (Note: as of PG 8.0 this isn't called
* during bootstrap, but leave the code here for possible future use.)
*
* We also must cheat if reindexing pg_class itself, because the target
* index may presently not be part of the set of indexes that
......@@ -1245,45 +1235,14 @@ UpdateStats(Oid relid, double reltuples)
elog(ERROR, "could not find tuple for relation %u", relid);
rd_rel = (Form_pg_class) GETSTRUCT(tuple);
/*
* Figure values to insert.
*
* If we found zero tuples in the scan, do NOT believe it; instead put a
* bogus estimate into the statistics fields. Otherwise, the common
* pattern "CREATE TABLE; CREATE INDEX; insert data" leaves the table
* with zero size statistics until a VACUUM is done. The optimizer
* will generate very bad plans if the stats claim the table is empty
* when it is actually sizable. See also CREATE TABLE in heap.c.
*
* Note: this path is also taken during bootstrap, because bootstrap.c
* passes reltuples = 0 after loading a table. We have to estimate
* some number for reltuples based on the actual number of pages.
*/
relpages = RelationGetNumberOfBlocks(whichRel);
if (reltuples == 0)
{
if (relpages == 0)
{
/* Bogus defaults for a virgin table, same as heap.c */
reltuples = 1000;
relpages = 10;
}
else if (whichRel->rd_rel->relkind == RELKIND_INDEX && relpages <= 2)
{
/* Empty index, leave bogus defaults in place */
reltuples = 1000;
}
else
reltuples = ((double) relpages) * NTUPLES_PER_PAGE(whichRel->rd_rel->relnatts);
}
/*
* Update statistics in pg_class, if they changed. (Avoiding an
* unnecessary update is not just a tiny performance improvement; it
* also reduces the window wherein concurrent CREATE INDEX commands
* may conflict.)
*/
relpages = RelationGetNumberOfBlocks(whichRel);
if (rd_rel->relpages != (int32) relpages ||
rd_rel->reltuples != (float4) reltuples)
{
......
......@@ -13,7 +13,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/commands/vacuum.c,v 1.295 2004/10/15 22:39:56 tgl Exp $
* $PostgreSQL: pgsql/src/backend/commands/vacuum.c,v 1.296 2004/12/01 19:00:41 tgl Exp $
*
*-------------------------------------------------------------------------
*/
......@@ -646,7 +646,7 @@ vac_update_relstats(Oid relid, BlockNumber num_pages, double num_tuples,
/* overwrite the existing statistics in the tuple */
pgcform = (Form_pg_class) GETSTRUCT(&rtup);
pgcform->relpages = (int32) num_pages;
pgcform->reltuples = num_tuples;
pgcform->reltuples = (float4) num_tuples;
pgcform->relhasindex = hasindex;
/*
......
......@@ -31,12 +31,14 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/commands/vacuumlazy.c,v 1.48 2004/10/25 15:42:02 tgl Exp $
* $PostgreSQL: pgsql/src/backend/commands/vacuumlazy.c,v 1.49 2004/12/01 19:00:42 tgl Exp $
*
*-------------------------------------------------------------------------
*/
#include "postgres.h"
#include <math.h>
#include "access/genam.h"
#include "access/heapam.h"
#include "access/xlog.h"
......@@ -67,6 +69,8 @@ typedef struct LVRelStats
/* Overall statistics about rel */
BlockNumber rel_pages;
double rel_tuples;
BlockNumber pages_removed;
double tuples_deleted;
BlockNumber nonempty_pages; /* actually, last nonempty page + 1 */
Size threshold; /* minimum interesting free space */
/* List of TIDs of tuples we intend to delete */
......@@ -94,12 +98,19 @@ static void lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
Relation *Irel, int nindexes);
static void lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats);
static void lazy_scan_index(Relation indrel, LVRelStats *vacrelstats);
static void lazy_vacuum_index(Relation indrel, LVRelStats *vacrelstats);
static void lazy_vacuum_index(Relation indrel,
double *index_tups_vacuumed,
BlockNumber *index_pages_removed,
LVRelStats *vacrelstats);
static int lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
int tupindex, LVRelStats *vacrelstats);
static void lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats);
static BlockNumber count_nondeletable_pages(Relation onerel,
LVRelStats *vacrelstats);
static void lazy_update_relstats(Relation rel, BlockNumber num_pages,
BlockNumber pages_removed,
double num_tuples, double tuples_removed,
bool hasindex);
static void lazy_space_alloc(LVRelStats *vacrelstats, BlockNumber relblocks);
static void lazy_record_dead_tuple(LVRelStats *vacrelstats,
ItemPointer itemptr);
......@@ -169,8 +180,10 @@ lazy_vacuum_rel(Relation onerel, VacuumStmt *vacstmt)
lazy_update_fsm(onerel, vacrelstats);
/* Update statistics in pg_class */
vac_update_relstats(RelationGetRelid(onerel), vacrelstats->rel_pages,
vacrelstats->rel_tuples, hasindex);
lazy_update_relstats(onerel, vacrelstats->rel_pages,
vacrelstats->pages_removed,
vacrelstats->rel_tuples, vacrelstats->tuples_deleted,
hasindex);
}
......@@ -195,6 +208,9 @@ lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
tups_vacuumed,
nkeep,
nunused;
double *index_tups_vacuumed;
BlockNumber *index_pages_removed;
bool did_vacuum_index = false;
int i;
VacRUsage ru0;
......@@ -209,6 +225,16 @@ lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
empty_pages = 0;
num_tuples = tups_vacuumed = nkeep = nunused = 0;
/*
* Because index vacuuming is done in multiple passes, we have to keep
* track of the total number of rows and pages removed from each index.
* index_tups_vacuumed[i] is the number removed so far from the i'th
* index. (For partial indexes this could well be different from
* tups_vacuumed.) Likewise for index_pages_removed[i].
*/
index_tups_vacuumed = (double *) palloc0(nindexes * sizeof(double));
index_pages_removed = (BlockNumber *) palloc0(nindexes * sizeof(BlockNumber));
nblocks = RelationGetNumberOfBlocks(onerel);
vacrelstats->rel_pages = nblocks;
vacrelstats->nonempty_pages = 0;
......@@ -238,7 +264,11 @@ lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
{
/* Remove index entries */
for (i = 0; i < nindexes; i++)
lazy_vacuum_index(Irel[i], vacrelstats);
lazy_vacuum_index(Irel[i],
&index_tups_vacuumed[i],
&index_pages_removed[i],
vacrelstats);
did_vacuum_index = true;
/* Remove tuples from heap */
lazy_vacuum_heap(onerel, vacrelstats);
/* Forget the now-vacuumed tuples, and press on */
......@@ -400,6 +430,7 @@ lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
/* save stats for use later */
vacrelstats->rel_tuples = num_tuples;
vacrelstats->tuples_deleted = tups_vacuumed;
/* If any tuples need to be deleted, perform final vacuum cycle */
/* XXX put a threshold on min number of tuples here? */
......@@ -407,11 +438,14 @@ lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
{
/* Remove index entries */
for (i = 0; i < nindexes; i++)
lazy_vacuum_index(Irel[i], vacrelstats);
lazy_vacuum_index(Irel[i],
&index_tups_vacuumed[i],
&index_pages_removed[i],
vacrelstats);
/* Remove tuples from heap */
lazy_vacuum_heap(onerel, vacrelstats);
}
else
else if (!did_vacuum_index)
{
/* Must do post-vacuum cleanup and statistics update anyway */
for (i = 0; i < nindexes; i++)
......@@ -588,9 +622,9 @@ lazy_scan_index(Relation indrel, LVRelStats *vacrelstats)
return;
/* now update statistics in pg_class */
vac_update_relstats(RelationGetRelid(indrel),
stats->num_pages, stats->num_index_tuples,
false);
lazy_update_relstats(indrel, stats->num_pages, stats->pages_removed,
stats->num_index_tuples, stats->tuples_removed,
false);
ereport(elevel,
(errmsg("index \"%s\" now contains %.0f row versions in %u pages",
......@@ -611,11 +645,17 @@ lazy_scan_index(Relation indrel, LVRelStats *vacrelstats)
* Delete all the index entries pointing to tuples listed in
* vacrelstats->dead_tuples.
*
* Increment *index_tups_vacuumed by the number of index entries
* removed, and *index_pages_removed by the number of pages removed.
*
* Finally, we arrange to update the index relation's statistics in
* pg_class.
*/
static void
lazy_vacuum_index(Relation indrel, LVRelStats *vacrelstats)
lazy_vacuum_index(Relation indrel,
double *index_tups_vacuumed,
BlockNumber *index_pages_removed,
LVRelStats *vacrelstats)
{
IndexBulkDeleteResult *stats;
IndexVacuumCleanupInfo vcinfo;
......@@ -652,10 +692,14 @@ lazy_vacuum_index(Relation indrel, LVRelStats *vacrelstats)
if (!stats)
return;
/* accumulate total removed over multiple index-cleaning cycles */
*index_tups_vacuumed += stats->tuples_removed;
*index_pages_removed += stats->pages_removed;
/* now update statistics in pg_class */
vac_update_relstats(RelationGetRelid(indrel),
stats->num_pages, stats->num_index_tuples,
false);
lazy_update_relstats(indrel, stats->num_pages, *index_pages_removed,
stats->num_index_tuples, *index_tups_vacuumed,
false);
ereport(elevel,
(errmsg("index \"%s\" now contains %.0f row versions in %u pages",
......@@ -741,8 +785,6 @@ lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats)
* Do the physical truncation.
*/
RelationTruncate(onerel, new_rel_pages);
vacrelstats->rel_pages = new_rel_pages; /* save new number of
* blocks */
/*
* Drop free-space info for removed blocks; these must not get entered
......@@ -763,6 +805,10 @@ lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats)
/* We destroyed the heap ordering, so mark array unordered */
vacrelstats->fs_is_heap = false;
/* update statistics */
vacrelstats->rel_pages = new_rel_pages;
vacrelstats->pages_removed = old_rel_pages - new_rel_pages;
/*
* We keep the exclusive lock until commit (perhaps not necessary)?
*/
......@@ -886,6 +932,51 @@ count_nondeletable_pages(Relation onerel, LVRelStats *vacrelstats)
return vacrelstats->nonempty_pages;
}
/*
* lazy_update_relstats - update pg_class statistics for a table or index
*
* We always want to set relpages to an accurate value. However, for lazy
* VACUUM it seems best to set reltuples to the average of the number of
* rows before vacuuming and the number after vacuuming, rather than just
* using the number after vacuuming. This will result in the best average
* performance in a steady-state situation where VACUUMs are performed
* regularly on a table of roughly constant size, assuming that the physical
* number of pages in the table stays about the same throughout. (Note that
* we do not apply the same logic to VACUUM FULL, because it repacks the table
* and thereby boosts the tuple density.)
*
* An important point is that when the table size has decreased a lot during
* vacuuming, the old reltuples count might give an overestimate of the tuple
* density. We handle this by scaling down the old reltuples count by the
* fraction by which the table has shortened before we merge it with the
* new reltuples count. In particular this means that when relpages goes to
* zero, reltuples will immediately go to zero as well, causing the planner
* to fall back on other estimation procedures as the table grows again.
*
* Because we do this math independently for the table and the indexes, it's
* quite possible to end up with an index's reltuples different from the
* table's, which is silly except in the case of partial indexes. We don't
* worry too much about that here; the planner contains filtering logic to
* ensure it only uses sane estimates.
*/
static void
lazy_update_relstats(Relation rel, BlockNumber num_pages,
BlockNumber pages_removed,
double num_tuples, double tuples_removed,
bool hasindex)
{
double old_num_tuples;
old_num_tuples = num_tuples + tuples_removed;
if (pages_removed > 0)
old_num_tuples *= (double) num_pages / (double) (num_pages + pages_removed);
if (old_num_tuples > num_tuples)
num_tuples = ceil((num_tuples + old_num_tuples) * 0.5);
vac_update_relstats(RelationGetRelid(rel), num_pages, num_tuples,
hasindex);
}
/*
* lazy_space_alloc - space allocation decisions for lazy vacuum
*
......
......@@ -9,7 +9,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/optimizer/util/plancat.c,v 1.97 2004/10/01 17:11:50 tgl Exp $
* $PostgreSQL: pgsql/src/backend/optimizer/util/plancat.c,v 1.98 2004/12/01 19:00:43 tgl Exp $
*
*-------------------------------------------------------------------------
*/
......@@ -38,6 +38,10 @@
#include "miscadmin.h"
static void estimate_rel_size(Relation rel, int32 *attr_widths,
BlockNumber *pages, double *tuples);
/*
* get_relation_info -
* Retrieves catalog information for a given relation.
......@@ -50,6 +54,10 @@
* indexlist list of IndexOptInfos for relation's indexes
* pages number of pages
* tuples number of tuples
*
* Also, initialize the attr_needed[] and attr_widths[] arrays. In most
* cases these are left as zeroes, but sometimes we need to compute attr
* widths here, and we may as well cache the results for costsize.c.
*/
void
get_relation_info(Oid relationObjectId, RelOptInfo *rel)
......@@ -64,6 +72,18 @@ get_relation_info(Oid relationObjectId, RelOptInfo *rel)
rel->min_attr = FirstLowInvalidHeapAttributeNumber + 1;
rel->max_attr = RelationGetNumberOfAttributes(relation);
Assert(rel->max_attr >= rel->min_attr);
rel->attr_needed = (Relids *)
palloc0((rel->max_attr - rel->min_attr + 1) * sizeof(Relids));
rel->attr_widths = (int32 *)
palloc0((rel->max_attr - rel->min_attr + 1) * sizeof(int32));
/*
* Estimate relation size.
*/
estimate_rel_size(relation, rel->attr_widths - rel->min_attr,
&rel->pages, &rel->tuples);
/*
* Make list of indexes. Ignore indexes on system catalogs if told
* to.
......@@ -121,8 +141,6 @@ get_relation_info(Oid relationObjectId, RelOptInfo *rel)
}
info->relam = indexRelation->rd_rel->relam;
info->pages = indexRelation->rd_rel->relpages;
info->tuples = indexRelation->rd_rel->reltuples;
info->amcostestimate = index_cost_estimator(indexRelation);
/*
......@@ -156,6 +174,26 @@ get_relation_info(Oid relationObjectId, RelOptInfo *rel)
info->predOK = false; /* set later in indxpath.c */
info->unique = index->indisunique;
/*
* Estimate the index size. If it's not a partial index, we
* lock the number-of-tuples estimate to equal the parent table;
* if it is partial then we have to use the same methods as we
* would for a table, except we can be sure that the index is
* not larger than the table.
*/
if (info->indpred == NIL)
{
info->pages = RelationGetNumberOfBlocks(indexRelation);
info->tuples = rel->tuples;
}
else
{
estimate_rel_size(indexRelation, NULL,
&info->pages, &info->tuples);
if (info->tuples > rel->tuples)
info->tuples = rel->tuples;
}
/* initialize cached join info to empty */
info->outer_relids = NULL;
info->inner_paths = NIL;
......@@ -170,13 +208,110 @@ get_relation_info(Oid relationObjectId, RelOptInfo *rel)
rel->indexlist = indexinfos;
rel->pages = relation->rd_rel->relpages;
rel->tuples = relation->rd_rel->reltuples;
/* XXX keep the lock here? */
heap_close(relation, AccessShareLock);
}
/*
* estimate_rel_size - estimate # pages and # tuples in a table or index
*
* If attr_widths isn't NULL, it points to the zero-index entry of the
* relation's attr_width[] cache; we fill this in if we have need to compute
* the attribute widths for estimation purposes.
*/
static void
estimate_rel_size(Relation rel, int32 *attr_widths,
BlockNumber *pages, double *tuples)
{
BlockNumber curpages;
BlockNumber relpages;
double reltuples;
double density;
switch (rel->rd_rel->relkind)
{
case RELKIND_RELATION:
case RELKIND_INDEX:
case RELKIND_TOASTVALUE:
/* it has storage, ok to call the smgr */
*pages = curpages = RelationGetNumberOfBlocks(rel);
/* quick exit if rel is clearly empty */
if (curpages == 0)
{
*tuples = 0;
break;
}
/* coerce values in pg_class to more desirable types */
relpages = (BlockNumber) rel->rd_rel->relpages;
reltuples = (double) rel->rd_rel->reltuples;
/*
* If it's an index, discount the metapage. This is a kluge
* because it assumes more than it ought to about index contents;
* it's reasonably OK for btrees but a bit suspect otherwise.
*/
if (rel->rd_rel->relkind == RELKIND_INDEX &&
relpages > 0)
{
curpages--;
relpages--;
}
/* estimate number of tuples from previous tuple density */
if (relpages > 0)
density = reltuples / (double) relpages;
else
{
/*
* When we have no data because the relation was truncated,
* estimate tuple width from attribute datatypes. We assume
* here that the pages are completely full, which is OK for
* tables (since they've presumably not been VACUUMed yet)
* but is probably an overestimate for indexes. Fortunately
* get_relation_info() can clamp the overestimate to the
* parent table's size.
*/
int32 tuple_width = 0;
int i;
for (i = 1; i <= RelationGetNumberOfAttributes(rel); i++)
{
Form_pg_attribute att = rel->rd_att->attrs[i - 1];
int32 item_width;
if (att->attisdropped)
continue;
/* This should match set_rel_width() in costsize.c */
item_width = get_attavgwidth(RelationGetRelid(rel), i);
if (item_width <= 0)
{
item_width = get_typavgwidth(att->atttypid,
att->atttypmod);
Assert(item_width > 0);
}
if (attr_widths != NULL)
attr_widths[i] = item_width;
tuple_width += item_width;
}
tuple_width = MAXALIGN(tuple_width);
tuple_width += MAXALIGN(sizeof(HeapTupleHeaderData));
tuple_width += sizeof(ItemPointerData);
/* note: integer division is intentional here */
density = (BLCKSZ - sizeof(PageHeaderData)) / tuple_width;
}
*tuples = rint(density * (double) curpages);
break;
case RELKIND_SEQUENCE:
/* Sequences always have a known size */
*pages = 1;
*tuples = 1;
break;
default:
/* else it has no disk storage; probably shouldn't get here? */
*pages = 0;
*tuples = 0;
break;
}
}
/*
* build_physical_tlist
*
......
......@@ -8,7 +8,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/optimizer/util/relnode.c,v 1.62 2004/08/29 05:06:44 momjian Exp $
* $PostgreSQL: pgsql/src/backend/optimizer/util/relnode.c,v 1.63 2004/12/01 19:00:43 tgl Exp $
*
*-------------------------------------------------------------------------
*/
......@@ -159,10 +159,14 @@ make_base_rel(Query *root, int relid)
break;
case RTE_SUBQUERY:
case RTE_FUNCTION:
/* Subquery or function --- need only set up attr range */
/* Subquery or function --- set up attr range and arrays */
/* Note: 0 is included in range to support whole-row Vars */
rel->min_attr = 0;
rel->max_attr = list_length(rte->eref->colnames);
rel->attr_needed = (Relids *)
palloc0((rel->max_attr - rel->min_attr + 1) * sizeof(Relids));
rel->attr_widths = (int32 *)
palloc0((rel->max_attr - rel->min_attr + 1) * sizeof(int32));
break;
default:
elog(ERROR, "unrecognized RTE kind: %d",
......@@ -170,12 +174,6 @@ make_base_rel(Query *root, int relid)
break;
}
Assert(rel->max_attr >= rel->min_attr);
rel->attr_needed = (Relids *)
palloc0((rel->max_attr - rel->min_attr + 1) * sizeof(Relids));
rel->attr_widths = (int32 *)
palloc0((rel->max_attr - rel->min_attr + 1) * sizeof(int32));
return rel;
}
......
......@@ -7,7 +7,7 @@
* Portions Copyright (c) 1996-2004, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* $PostgreSQL: pgsql/src/include/access/genam.h,v 1.45 2004/08/29 04:13:03 momjian Exp $
* $PostgreSQL: pgsql/src/include/access/genam.h,v 1.46 2004/12/01 19:00:45 tgl Exp $
*
*-------------------------------------------------------------------------
*/
......@@ -28,10 +28,15 @@
* an index AM could choose to have bulk-delete return a larger struct
* of which this is just the first field; this provides a way for bulk-delete
* to communicate additional private data to vacuum-cleanup.
*
* Note: pages_removed is the amount by which the index physically shrank,
* if any (ie the change in its total size on disk). pages_deleted and
* pages_free refer to free space within the index file.
*/
typedef struct IndexBulkDeleteResult
{
BlockNumber num_pages; /* pages remaining in index */
BlockNumber pages_removed; /* # removed by bulk-delete operation */
double num_index_tuples; /* tuples remaining */
double tuples_removed; /* # removed by bulk-delete operation */
BlockNumber pages_deleted; /* # unused pages in index */
......
......@@ -7,7 +7,7 @@
* Portions Copyright (c) 1996-2004, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* $PostgreSQL: pgsql/src/include/nodes/relation.h,v 1.100 2004/11/26 21:08:35 tgl Exp $
* $PostgreSQL: pgsql/src/include/nodes/relation.h,v 1.101 2004/12/01 19:00:53 tgl Exp $
*
*-------------------------------------------------------------------------
*/
......@@ -255,7 +255,7 @@ typedef struct IndexOptInfo
Oid indexoid; /* OID of the index relation */
/* statistics from pg_class */
long pages; /* number of disk pages in index */
BlockNumber pages; /* number of disk pages in index */
double tuples; /* number of index tuples in index */
/* index descriptor information */
......
......@@ -157,28 +157,28 @@ SELECT COALESCE(a.f, b.i, b.j)
coalesce
----------
10.1
10.1
10.1
10.1
10.1
10.1
20.2
20.2
20.2
20.2
20.2
20.2
-30.3
-30.3
-30.3
-30.3
-30.3
-30.3
1
10.1
20.2
-30.3
2
10.1
20.2
-30.3
3
10.1
20.2
-30.3
2
10.1
20.2
-30.3
1
10.1
20.2
-30.3
-6
(24 rows)
......@@ -197,28 +197,28 @@ SELECT '' AS Five, NULLIF(a.i,b.i) AS "NULLIF(a.i,b.i)",
five | NULLIF(a.i,b.i) | NULLIF(b.i,4)
------+-----------------+---------------
| | 1
| 1 | 2
| 1 | 3
| 1 | 2
| | 1
| 1 |
| 2 | 1
| | 2
| 2 | 3
| | 2
| 2 | 1
| 2 |
| 3 | 1
| 3 | 2
| | 3
| 3 | 2
| 3 | 1
| 3 |
| 4 | 1
| 1 | 2
| | 2
| 3 | 2
| 4 | 2
| 1 | 3
| 2 | 3
| | 3
| 4 | 3
| 1 | 2
| | 2
| 3 | 2
| 4 | 2
| | 1
| 2 | 1
| 3 | 1
| 4 | 1
| 1 |
| 2 |
| 3 |
| 4 |
(24 rows)
......
......@@ -561,13 +561,13 @@ SELECT relname, bar.* FROM bar, pg_class where bar.tableoid = pg_class.oid;
relname | f1 | f2
---------+----+-----
bar | 4 | 4
bar | 1 | 101
bar | 2 | 102
bar | 3 | 103
bar | 2 | 102
bar | 1 | 101
bar2 | 4 | 4
bar2 | 1 | 101
bar2 | 2 | 102
bar2 | 3 | 103
bar2 | 2 | 102
bar2 | 1 | 101
(8 rows)
/* Test inheritance of structure (LIKE) */
......
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment