Update examples in planstats.sgml for 8.3, and improve some aspects of

that discussion.  Add a link from perform.sgml.

doc/src/sgml/perform.sgml

-<!-- $PostgreSQL: pgsql/doc/src/sgml/perform.sgml,v 1.67 2007/11/28 15:42:31 petere Exp $ -->
+<!-- $PostgreSQL: pgsql/doc/src/sgml/perform.sgml,v 1.68 2007/12/28 21:03:31 tgl Exp $ -->
 
  <chapter id="performance-tips">
   <title>Performance Tips</title>
@@ -8,7 +8,7 @@
   </indexterm>
 
   <para>
-   Query performance can be affected by many things. Some of these can 
+   Query performance can be affected by many things. Some of these can
    be manipulated by the user, while others are fundamental to the underlying
    design of the system.  This chapter provides some hints about understanding
    and tuning <productname>PostgreSQL</productname> performance.
@@ -138,7 +138,7 @@ EXPLAIN SELECT * FROM tenk1;
     Rows output is a little tricky because it is <emphasis>not</emphasis> the
     number of rows processed or scanned by the plan node.  It is usually less,
     reflecting the estimated selectivity of any <literal>WHERE</>-clause
-    conditions that are being 
+    conditions that are being
     applied at the node.  Ideally the top-level rows estimate will
     approximate the number of rows actually returned, updated, or deleted
     by the query.
@@ -469,8 +469,8 @@ EXPLAIN ANALYZE SELECT * FROM tenk1 t1, tenk2 t2 WHERE t1.unique1 &lt; 100 AND t
    One component of the statistics is the total number of entries in
    each table and index, as well as the number of disk blocks occupied
    by each table and index.  This information is kept in the table
-   <link linkend="catalog-pg-class"><structname>pg_class</structname></link>, in
-   the columns <structfield>reltuples</structfield> and
+   <link linkend="catalog-pg-class"><structname>pg_class</structname></link>,
+   in the columns <structfield>reltuples</structfield> and
    <structfield>relpages</structfield>.  We can look at it with
    queries similar to this one:
 
@@ -493,7 +493,7 @@ SELECT relname, relkind, reltuples, relpages FROM pg_class WHERE relname LIKE 't
   </para>
 
   <para>
-   For efficiency reasons, <structfield>reltuples</structfield> 
+   For efficiency reasons, <structfield>reltuples</structfield>
    and <structfield>relpages</structfield> are not updated on-the-fly,
    and so they usually contain somewhat out-of-date values.
    They are updated by <command>VACUUM</>, <command>ANALYZE</>, and a
@@ -517,7 +517,8 @@ SELECT relname, relkind, reltuples, relpages FROM pg_class WHERE relname LIKE 't
    <firstterm>selectivity</> of <literal>WHERE</> clauses, that is,
    the fraction of rows that match each condition in the
    <literal>WHERE</> clause.  The information used for this task is
-   stored in the <link linkend="catalog-pg-statistic"><structname>pg_statistic</structname></link>
+   stored in the
+   <link linkend="catalog-pg-statistic"><structname>pg_statistic</structname></link>
    system catalog.  Entries in <structname>pg_statistic</structname>
    are updated by the <command>ANALYZE</> and <command>VACUUM
    ANALYZE</> commands, and are always approximate even when freshly
@@ -530,7 +531,8 @@ SELECT relname, relkind, reltuples, relpages FROM pg_class WHERE relname LIKE 't
 
   <para>
    Rather than look at <structname>pg_statistic</structname> directly,
-   it's better to look at its view <structname>pg_stats</structname>
+   it's better to look at its view
+   <link linkend="view-pg-stats"><structname>pg_stats</structname></link>
    when examining the statistics manually.  <structname>pg_stats</structname>
    is designed to be more easily readable.  Furthermore,
    <structname>pg_stats</structname> is readable by all, whereas
@@ -553,13 +555,8 @@ SELECT attname, n_distinct, most_common_vals FROM pg_stats WHERE tablename = 'ro
   </para>
 
   <para>
-   <structname>pg_stats</structname> is described in detail in
-   <xref linkend="view-pg-stats">.
-  </para>
-
-  <para>
-   The amount of information stored in <structname>pg_statistic</structname>,
-   in particular the maximum number of entries in the
+   The amount of information stored in <structname>pg_statistic</structname>
+   by <command>ANALYZE</>, in particular the maximum number of entries in the
    <structfield>most_common_vals</> and <structfield>histogram_bounds</>
    arrays for each column, can be set on a
    column-by-column basis using the <command>ALTER TABLE SET STATISTICS</>
@@ -570,7 +567,12 @@ SELECT attname, n_distinct, most_common_vals FROM pg_stats WHERE tablename = 'ro
    columns with irregular data distributions, at the price of consuming
    more space in <structname>pg_statistic</structname> and slightly more
    time to compute the estimates.  Conversely, a lower limit might be
-   appropriate for columns with simple data distributions.
+   sufficient for columns with simple data distributions.
+  </para>
+
+  <para>
+   Further details about the planner's use of statistics can be found in
+   <xref linkend="planner-stats-details">.
   </para>
 
  </sect1>
@@ -913,7 +915,7 @@ SELECT * FROM x, y, a, b, c WHERE something AND somethingelse;
     are designed not to write WAL at all if <varname>archive_mode</varname>
     is off.  (They can guarantee crash safety more cheaply by doing an
     <function>fsync</> at the end than by writing WAL.)
-    This applies to the following commands: 
+    This applies to the following commands:
     <itemizedlist>
      <listitem>
       <para>

doc/src/sgml/planstats.sgml

-<!-- $PostgreSQL: pgsql/doc/src/sgml/planstats.sgml,v 1.8 2007/01/31 20:56:18 momjian Exp $ -->
+<!-- $PostgreSQL: pgsql/doc/src/sgml/planstats.sgml,v 1.9 2007/12/28 21:03:31 tgl Exp $ -->
 
 <chapter id="planner-stats-details">
  <title>How the Planner Uses Statistics</title>
 
   <para>
-   This chapter builds on the material covered in <xref linkend="using-explain">
-   and <xref linkend="planner-stats">, and shows how the planner uses the 
-   system statistics to estimate the number of rows each stage in a query might
-   return. This is a significant part of the planning / optimizing process,
+   This chapter builds on the material covered in <xref
+   linkend="using-explain"> and <xref linkend="planner-stats"> to show some
+   additional details about how the planner uses the
+   system statistics to estimate the number of rows each part of a query might
+   return. This is a significant part of the planning process,
    providing much of the raw material for cost calculation.
   </para>
 
   <para>
-   The intent of this chapter is not to document the code &mdash; 
-   better done in the code itself, but to present an overview of how it works.
-   This will perhaps ease the learning curve for someone who subsequently 
-   wishes to read the code. As a consequence, the approach chosen is to analyze
-   a series of incrementally more complex examples. 
-  </para>
-
-  <para>
-   The outputs and algorithms shown below are taken from version 8.0. 
-   The behavior of earlier (or later) versions might vary.
+   The intent of this chapter is not to document the code in detail,
+   but to present an overview of how it works.
+   This will perhaps ease the learning curve for someone who subsequently
+   wishes to read the code.
   </para>
 
  <sect1 id="row-estimation-examples">
@@ -33,160 +28,163 @@
   </indexterm>
 
   <para>
-   Using examples drawn from the regression test database, let's start with a 
-   very simple query:
+   The examples shown below use tables in the <productname>PostgreSQL</>
+   regression test database.
+   The outputs shown are taken from version 8.3.
+   The behavior of earlier (or later) versions might vary.
+   Note also that since <command>ANALYZE</> uses random sampling
+   while producing statistics, the results will change slightly after
+   any new <command>ANALYZE</>.
+  </para>
+
+  <para>
+   Let's start with a very simple query:
+
 <programlisting>
 EXPLAIN SELECT * FROM tenk1;
 
                          QUERY PLAN
 -------------------------------------------------------------
- Seq Scan on tenk1  (cost=0.00..445.00 rows=10000 width=244)
+ Seq Scan on tenk1  (cost=0.00..458.00 rows=10000 width=244)
 </programlisting>
-   
-   How the planner determines the cardinality of <classname>tenk1</classname>
-   is covered in <xref linkend="using-explain">, but is repeated here for
-   completeness. The number of rows is looked up from 
-   <classname>pg_class</classname>:
+
+   How the planner determines the cardinality of <structname>tenk1</structname>
+   is covered in <xref linkend="planner-stats">, but is repeated here for
+   completeness. The number of pages and rows is looked up in
+   <structname>pg_class</structname>:
 
 <programlisting>
-SELECT reltuples, relpages FROM pg_class WHERE relname = 'tenk1';
+SELECT relpages, reltuples FROM pg_class WHERE relname = 'tenk1';
 
  relpages | reltuples
 ----------+-----------
-      345 |     10000
-</programlisting>
-    The planner will check the <structfield>relpages</structfield>
-    estimate (this is a cheap operation) and if incorrect might scale
-    <structfield>reltuples</structfield> to obtain a row estimate. In this
-    case it does not, thus:
-
-<programlisting>
-rows = 10000
+      358 |     10000
 </programlisting>
 
+    These numbers are current as of the last <command>VACUUM</> or
+    <command>ANALYZE</> on the table.  The planner then fetches the
+    actual current number of pages in the table (this is a cheap operation,
+    not requiring a table scan).  If that is different from
+    <structfield>relpages</structfield> then
+    <structfield>reltuples</structfield> is scaled accordingly to
+    arrive at a current number-of-rows estimate.  In this case the values
+    are correct so the rows estimate is the same as
+    <structfield>reltuples</structfield>.
   </para>
-   
+
   <para>
-   let's move on to an example with a range condition in its 
+   Let's move on to an example with a range condition in its
    <literal>WHERE</literal> clause:
 
 <programlisting>
 EXPLAIN SELECT * FROM tenk1 WHERE unique1 &lt; 1000;
 
-                         QUERY PLAN
-------------------------------------------------------------
- Seq Scan on tenk1  (cost=0.00..470.00 rows=1031 width=244)
-   Filter: (unique1 &lt; 1000)
-</programlisting>
-
-   The planner examines the <literal>WHERE</literal> clause condition: 
-
-<programlisting>
-unique1 &lt; 1000
+                                   QUERY PLAN
+--------------------------------------------------------------------------------
+ Bitmap Heap Scan on tenk1  (cost=24.06..394.64 rows=1007 width=244)
+   Recheck Cond: (unique1 &lt; 1000)
+   -&gt;  Bitmap Index Scan on tenk1_unique1  (cost=0.00..23.80 rows=1007 width=0)
+         Index Cond: (unique1 &lt; 1000)
 </programlisting>
 
-   and looks up the restriction function for the operator 
-   <literal>&lt;</literal> in <classname>pg_operator</classname>. 
-   This is held in the column <structfield>oprrest</structfield>, 
-   and the result in this case is <function>scalarltsel</function>.
-   The <function>scalarltsel</function> function retrieves the histogram for 
-   <structfield>unique1</structfield> from <classname>pg_statistics</classname>
-   - we can follow this by using the simpler <classname>pg_stats</classname> 
+   The planner examines the <literal>WHERE</literal> clause condition
+   and looks up the selectivity function for the operator
+   <literal>&lt;</literal> in <structname>pg_operator</structname>.
+   This is held in the column <structfield>oprrest</structfield>,
+   and the entry in this case is <function>scalarltsel</function>.
+   The <function>scalarltsel</function> function retrieves the histogram for
+   <structfield>unique1</structfield> from
+   <structname>pg_statistics</structname>.  For manual queries it is more
+   convenient to look in the simpler <structname>pg_stats</structname>
    view:
 
 <programlisting>
-SELECT histogram_bounds FROM pg_stats 
+SELECT histogram_bounds FROM pg_stats
 WHERE tablename='tenk1' AND attname='unique1';
 
                    histogram_bounds
 ------------------------------------------------------
- {1,970,1943,2958,3971,5069,6028,7007,7919,8982,9995}
+ {0,993,1997,3050,4040,5036,5957,7057,8029,9016,9995}
 </programlisting>
 
-   Next the fraction of the histogram occupied by <quote>&lt; 1000</quote> 
-   is worked out. This is the selectivity. The histogram divides the range 
-   into equal frequency buckets, so all we have to do is locate the bucket 
-   that our value is in and count <emphasis>part</emphasis> of it and 
-   <emphasis>all</emphasis> of the ones before. The value 1000 is clearly in 
-   the second (970 - 1943) bucket, so by assuming a linear distribution of 
-   values inside each bucket we can calculate the selectivity as:
+   Next the fraction of the histogram occupied by <quote>&lt; 1000</quote>
+   is worked out. This is the selectivity. The histogram divides the range
+   into equal frequency buckets, so all we have to do is locate the bucket
+   that our value is in and count <emphasis>part</emphasis> of it and
+   <emphasis>all</emphasis> of the ones before. The value 1000 is clearly in
+   the second bucket (993-1997).  Assuming a linear distribution of
+   values inside each bucket, we can calculate the selectivity as:
 
 <programlisting>
-selectivity = (1 + (1000 - bckt[2].min)/(bckt[2].max - bckt[2].min))/num_bckts
-            = (1 + (1000 - 970)/(1943 - 970))/10
-            = 0.1031
+selectivity = (1 + (1000 - bucket[2].min)/(bucket[2].max - bucket[2].min))/num_buckets
+            = (1 + (1000 - 993)/(1997 - 993))/10
+            = 0.100697
 </programlisting>
 
    that is, one whole bucket plus a linear fraction of the second, divided by
    the number of buckets. The estimated number of rows can now be calculated as
-   the product of the selectivity and the cardinality of 
-   <classname>tenk1</classname>:
+   the product of the selectivity and the cardinality of
+   <structname>tenk1</structname>:
 
 <programlisting>
 rows = rel_cardinality * selectivity
-     = 10000 * 0.1031
-     = 1031
+     = 10000 * 0.100697
+     = 1007  (rounding off)
 </programlisting>
-
   </para>
 
   <para>
-   Next let's consider an example with equality condition in its 
+   Next let's consider an example with an equality condition in its
    <literal>WHERE</literal> clause:
 
 <programlisting>
-EXPLAIN SELECT * FROM tenk1 WHERE stringu1 = 'ATAAAA';
+EXPLAIN SELECT * FROM tenk1 WHERE stringu1 = 'CRAAAA';
 
                         QUERY PLAN
 ----------------------------------------------------------
- Seq Scan on tenk1  (cost=0.00..470.00 rows=31 width=244)
-   Filter: (stringu1 = 'ATAAAA'::name)
+ Seq Scan on tenk1  (cost=0.00..483.00 rows=30 width=244)
+   Filter: (stringu1 = 'CRAAAA'::name)
 </programlisting>
 
-   Again the planner examines the <literal>WHERE</literal> clause condition: 
+   Again the planner examines the <literal>WHERE</literal> clause condition
+   and looks up the selectivity function for <literal>=</literal>, which is
+   <function>eqsel</function>.  For equality estimation the histogram is
+   not useful; instead the list of <firstterm>most
+   common values</> (<acronym>MCV</acronym>s) is used to determine the
+   selectivity. Let's have a look at the MCVs, with some additional columns
+   that will be useful later:
 
 <programlisting>
-stringu1 = 'ATAAAA'
-</programlisting>
-
-   and looks up the restriction function for <literal>=</literal>, which is 
-   <function>eqsel</function>. This case is a bit different, as the most
-   common values &mdash; <acronym>MCV</acronym>s, are used to determine the 
-   selectivity. Let's have a look at these, with some extra columns that will
-   be useful later:
-
-<programlisting>
-SELECT null_frac, n_distinct, most_common_vals, most_common_freqs FROM pg_stats 
+SELECT null_frac, n_distinct, most_common_vals, most_common_freqs FROM pg_stats
 WHERE tablename='tenk1' AND attname='stringu1';
 
 null_frac         | 0
-n_distinct        | 672
-most_common_vals  | {FDAAAA,NHAAAA,ATAAAA,BGAAAA,EBAAAA,MOAAAA,NDAAAA,OWAAAA,BHAAAA,BJAAAA}
-most_common_freqs | {0.00333333,0.00333333,0.003,0.003,0.003,0.003,0.003,0.003,0.00266667,0.00266667}
+n_distinct        | 676
+most_common_vals  | {EJAAAA,BBAAAA,CRAAAA,FCAAAA,FEAAAA,GSAAAA,JOAAAA,MCAAAA,NAAAAA,WGAAAA}
+most_common_freqs | {0.00333333,0.003,0.003,0.003,0.003,0.003,0.003,0.003,0.003,0.003}
+
 </programlisting>
 
-   The selectivity is merely the most common frequency (<acronym>MCF</acronym>)
-   corresponding to the third <acronym>MCV</acronym> &mdash; 'ATAAAA': 
+   Since <literal>CRAAAA</> appears in the list of MCVs, the selectivity is
+   merely the corresponding entry in the list of most common frequencies
+   (<acronym>MCF</acronym>s):
 
 <programlisting>
 selectivity = mcf[3]
             = 0.003
 </programlisting>
 
-   The estimated number of rows is just the product of this with the 
-   cardinality of <classname>tenk1</classname> as before:
+   As before, the estimated number of rows is just the product of this with the
+   cardinality of <structname>tenk1</structname>:
 
 <programlisting>
 rows = 10000 * 0.003
      = 30
 </programlisting>
-
-   The number displayed by <command>EXPLAIN</command> is one more than this,
-   due to some post estimation checks.
   </para>
 
   <para>
-   Now consider the same query, but with a constant that is not in the 
+   Now consider the same query, but with a constant that is not in the
    <acronym>MCV</acronym> list:
 
 <programlisting>
@@ -194,116 +192,197 @@ EXPLAIN SELECT * FROM tenk1 WHERE stringu1 = 'xxx';
 
                         QUERY PLAN
 ----------------------------------------------------------
- Seq Scan on tenk1  (cost=0.00..470.00 rows=15 width=244)
+ Seq Scan on tenk1  (cost=0.00..483.00 rows=15 width=244)
    Filter: (stringu1 = 'xxx'::name)
 </programlisting>
 
-   This is quite a different problem, how to estimate the selectivity when the
-   value is <emphasis>not</emphasis> in the <acronym>MCV</acronym> list. 
-   The approach is to use the fact that the value is not in the list, 
+   This is quite a different problem: how to estimate the selectivity when the
+   value is <emphasis>not</emphasis> in the <acronym>MCV</acronym> list.
+   The approach is to use the fact that the value is not in the list,
    combined with the knowledge of the frequencies for all of the
    <acronym>MCV</acronym>s:
 
 <programlisting>
 selectivity = (1 - sum(mvf))/(num_distinct - num_mcv)
-            = (1 - (0.00333333 + 0.00333333 + 0.003 + 0.003 + 0.003 
-            + 0.003 + 0.003 + 0.003 + 0.00266667 + 0.00266667))/(672 - 10)
-            = 0.001465
+            = (1 - (0.00333333 + 0.003 + 0.003 + 0.003 + 0.003 + 0.003 +
+                    0.003 + 0.003 + 0.003 + 0.003))/(676 - 10)
+            = 0.0014559
 </programlisting>
 
-   That is, add up all the frequencies for the <acronym>MCV</acronym>s and 
-   subtract them from one &mdash; because it is <emphasis>not</emphasis> one 
-   of these, and divide by the <emphasis>remaining</emphasis> distinct values. 
-   Notice that there are no null values so we don't have to worry about those. 
-   The estimated number of rows is calculated as usual:
+   That is, add up all the frequencies for the <acronym>MCV</acronym>s and
+   subtract them from one, then
+   divide by the number of <emphasis>other</emphasis> distinct values.
+   This amounts to assuming that the fraction of the column that is not any
+   of the MCVs is evenly distributed among all the other distinct values.
+   Notice that there are no null values so we don't have to worry about those
+   (otherwise we'd subtract the null fraction from the numerator as well).
+   The estimated number of rows is then calculated as usual:
 
 <programlisting>
-rows = 10000 * 0.001465
-     = 15
+rows = 10000 * 0.0014559
+     = 15  (rounding off)
 </programlisting>
-
   </para>
 
   <para>
-   Let's increase the complexity to consider a case with more than one 
-   condition in the <literal>WHERE</literal> clause:
+   The previous example with <literal>unique1 &lt; 1000</> was an
+   oversimplification of what <function>scalarltsel</function> really does;
+   now that we have seen an example of the use of MCVs, we can fill in some
+   more detail.  The example was correct as far as it went, because since
+   <structfield>unique1</> is a unique column it has no MCVs (obviously, no
+   value is any more common than any other value).  For a non-unique
+   column, there will normally be both a histogram and an MCV list, and
+   <emphasis>the histogram does not include the portion of the column
+   population represented by the MCVs</>.  We do things this way because
+   it allows more precise estimation.  In this situation
+   <function>scalarltsel</function> directly applies the condition (e.g.,
+   <quote>&lt; 1000</>) to each value of the MCV list, and adds up the
+   frequencies of the MCVs for which the condition is true.  This gives
+   an exact estimate of the selectivity within the portion of the table
+   that is MCVs.  The histogram is then used in the same way as above
+   to estimate the selectivity in the portion of the table that is not
+   MCVs, and then the two numbers are combined to estimate the overall
+   selectivity.  For example, consider
 
 <programlisting>
-EXPLAIN SELECT * FROM tenk1 WHERE unique1 &lt; 1000 AND stringu1 = 'xxx';
+EXPLAIN SELECT * FROM tenk1 WHERE stringu1 &lt; 'IAAAAA';
 
-                       QUERY PLAN
------------------------------------------------------------
- Seq Scan on tenk1  (cost=0.00..495.00 rows=2 width=244)
-   Filter: ((unique1 &lt; 1000) AND (stringu1 = 'xxx'::name))
+                         QUERY PLAN
+------------------------------------------------------------
+ Seq Scan on tenk1  (cost=0.00..483.00 rows=3077 width=244)
+   Filter: (stringu1 &lt; 'IAAAAA'::name)
 </programlisting>
 
-   An assumption of independence is made and the selectivities of the 
-   individual restrictions are multiplied together:
+   We already saw the MCV information for <structfield>stringu1</>,
+   and here is its histogram:
 
 <programlisting>
-selectivity = selectivity(unique1 &lt; 1000) * selectivity(stringu1 = 'xxx')
-            = 0.1031 * 0.001465
-            = 0.00015104
+SELECT histogram_bounds FROM pg_stats
+WHERE tablename='tenk1' AND attname='stringu1';
+
+                                histogram_bounds
+--------------------------------------------------------------------------------
+ {AAAAAA,CQAAAA,FRAAAA,IBAAAA,KRAAAA,NFAAAA,PSAAAA,SGAAAA,VAAAAA,XLAAAA,ZZAAAA}
 </programlisting>
 
-   The row estimates are calculated as before:
+   Checking the MCV list, we find that the condition <literal>stringu1 &lt;
+   'IAAAAA'</> is satisfied by the first six entries and not the last four,
+   so the selectivity within the MCV part of the population is
 
 <programlisting>
-rows = 10000 * 0.00015104
-     = 2
+selectivity = sum(relevant mvfs)
+            = 0.00333333 + 0.003 + 0.003 + 0.003 + 0.003 + 0.003
+            = 0.01833333
 </programlisting>
+
+   Summing all the MCFs also tells us that the total fraction of the
+   population represented by MCVs is 0.03033333, and therefore the
+   fraction represented by the histogram is 0.96966667 (again, there
+   are no nulls, else we'd have to exclude them here).  We can see
+   that the value <literal>IAAAAA</> falls nearly at the end of the
+   third histogram bucket.  Using some rather cheesy assumptions
+   about the frequency of different characters, the planner arrives
+   at the estimate 0.298387 for the portion of the histogram population
+   that is less than <literal>IAAAAA</>.  We then combine the estimates
+   for the MCV and non-MCV populations:
+
+<programlisting>
+selectivity = mcv_selectivity + histogram_selectivity * histogram_fraction
+            = 0.01833333 + 0.298387 * 0.96966667
+            = 0.307669
+
+rows        = 10000 * 0.307669
+            = 3077  (rounding off)
+</programlisting>
+
+   In this particular example, the correction from the MCV list is fairly
+   small, because the column distribution is actually quite flat (the
+   statistics showing these particular values as being more common than
+   others are mostly due to sampling error).  In a more typical case where
+   some values are significantly more common than others, this complicated
+   process gives a useful improvement in accuracy because the selectivity
+   for the most common values is found exactly.
   </para>
- 
+
   <para>
-   Finally we will examine a query that includes a <literal>JOIN</literal> 
-   together with a <literal>WHERE</literal> clause:
+   Now let's consider a case with more than one
+   condition in the <literal>WHERE</literal> clause:
 
 <programlisting>
-EXPLAIN SELECT *  FROM tenk1 t1, tenk2 t2 
-WHERE t1.unique1 &lt; 50 AND t1.unique2 = t2.unique2;
+EXPLAIN SELECT * FROM tenk1 WHERE unique1 &lt; 1000 AND stringu1 = 'xxx';
 
-                                      QUERY PLAN
------------------------------------------------------------------------------------------
- Nested Loop  (cost=0.00..346.90 rows=51 width=488)
-   ->  Index Scan using tenk1_unique1 on tenk1 t1  (cost=0.00..192.57 rows=51 width=244)
-         Index Cond: (unique1 &lt; 50)
-   ->  Index Scan using tenk2_unique2 on tenk2 t2  (cost=0.00..3.01 rows=1 width=244)
-         Index Cond: ("outer".unique2 = t2.unique2)
+                                   QUERY PLAN
+--------------------------------------------------------------------------------
+ Bitmap Heap Scan on tenk1  (cost=23.80..396.91 rows=1 width=244)
+   Recheck Cond: (unique1 &lt; 1000)
+   Filter: (stringu1 = 'xxx'::name)
+   -&gt;  Bitmap Index Scan on tenk1_unique1  (cost=0.00..23.80 rows=1007 width=0)
+         Index Cond: (unique1 &lt; 1000)
 </programlisting>
 
-   The restriction on <classname>tenk1</classname> 
-   <quote>unique1 &lt; 50</quote> is evaluated before the nested-loop join. 
-   This is handled analogously to the previous range example. The restriction 
-   operator for <literal>&lt;</literal> is <function>scalarlteqsel</function> 
-   as before, but this time the value 50 is in the first bucket of the 
-   <structfield>unique1</structfield> histogram:
+   The planner assumes that the two conditions are independent, so that
+   the individual selectivities of the clauses can be multiplied together:
+
+<programlisting>
+selectivity = selectivity(unique1 &lt; 1000) * selectivity(stringu1 = 'xxx')
+            = 0.100697 * 0.0014559
+            = 0.0001466
+
+rows        = 10000 * 0.0001466
+            = 1  (rounding off)
+</programlisting>
+
+   Notice that the number of rows estimated to be returned from the bitmap
+   index scan reflects only the condition used with the index; this is
+   important since it affects the cost estimate for the subsequent heap
+   fetches.
+  </para>
+
+  <para>
+   Finally we will examine a query that involves a join:
 
 <programlisting>
-selectivity = (0 + (50 - bckt[1].min)/(bckt[1].max - bckt[1].min))/num_bckts
-            = (0 + (50 - 1)/(970 - 1))/10
-            = 0.005057
+EXPLAIN SELECT * FROM tenk1 t1, tenk2 t2
+WHERE t1.unique1 &lt; 50 AND t1.unique2 = t2.unique2;
 
-rows        = 10000 * 0.005057
-            = 51
+                                      QUERY PLAN
+--------------------------------------------------------------------------------------
+ Nested Loop  (cost=4.64..456.23 rows=50 width=488)
+   -&gt;  Bitmap Heap Scan on tenk1 t1  (cost=4.64..142.17 rows=50 width=244)
+         Recheck Cond: (unique1 &lt; 50)
+         -&gt;  Bitmap Index Scan on tenk1_unique1  (cost=0.00..4.63 rows=50 width=0)
+               Index Cond: (unique1 &lt; 50)
+   -&gt;  Index Scan using tenk2_unique2 on tenk2 t2  (cost=0.00..6.27 rows=1 width=244)
+         Index Cond: (t2.unique2 = t1.unique2)
 </programlisting>
 
-   The restriction for the join is:
+   The restriction on <structname>tenk1</structname>,
+   <literal>unique1 &lt; 50</literal>,
+   is evaluated before the nested-loop join.
+   This is handled analogously to the previous range example.  This time the
+   value 50 falls into the first bucket of the
+   <structfield>unique1</structfield> histogram:
 
 <programlisting>
-t2.unique2 = t1.unique2
+selectivity = (0 + (50 - bucket[1].min)/(bucket[1].max - bucket[1].min))/num_buckets
+            = (0 + (50 - 0)/(993 - 0))/10
+            = 0.005035
+
+rows        = 10000 * 0.005035
+            = 50  (rounding off)
 </programlisting>
 
-   This is due to the join method being nested-loop, with 
-   <classname>tenk1</classname> being in the outer loop. The operator is just 
-   our familiar <literal>=</literal>, however the restriction function is 
-   obtained from the <structfield>oprjoin</structfield> column of 
-   <classname>pg_operator</classname> - and is <function>eqjoinsel</function>.
-   Additionally we use the statistical information for both 
-   <classname>tenk2</classname> and <classname>tenk1</classname>:
+   The restriction for the join is <literal>t2.unique2 = t1.unique2</>.
+   The operator is just
+   our familiar <literal>=</literal>, however the selectivity function is
+   obtained from the <structfield>oprjoin</structfield> column of
+   <structname>pg_operator</structname>, and is <function>eqjoinsel</function>.
+   <function>eqjoinsel</function> looks up the statistical information for both
+   <structname>tenk2</structname> and <structname>tenk1</structname>:
 
 <programlisting>
-SELECT tablename, null_frac,n_distinct, most_common_vals FROM pg_stats 
-WHERE tablename IN ('tenk1', 'tenk2') AND attname='unique2';  
+SELECT tablename, null_frac,n_distinct, most_common_vals FROM pg_stats
+WHERE tablename IN ('tenk1', 'tenk2') AND attname='unique2';
 
 tablename  | null_frac | n_distinct | most_common_vals
 -----------+-----------+------------+------------------
@@ -311,41 +390,62 @@ tablename  | null_frac | n_distinct | most_common_vals
  tenk2     |         0 |         -1 |
 </programlisting>
 
-   In this case there is no <acronym>MCV</acronym> information for 
-   <structfield>unique2</structfield> because all the values appear to be 
-   unique, so we can use an algorithm that relies only on the number of 
+   In this case there is no <acronym>MCV</acronym> information for
+   <structfield>unique2</structfield> because all the values appear to be
+   unique, so we use an algorithm that relies only on the number of
    distinct values for both relations together with their null fractions:
 
 <programlisting>
 selectivity = (1 - null_frac1) * (1 - null_frac2) * min(1/num_distinct1, 1/num_distinct2)
-            = (1 - 0) * (1 - 0) * min(1/10000, 1/1000)
+            = (1 - 0) * (1 - 0) / max(10000, 10000)
             = 0.0001
 </programlisting>
 
-   This is, subtract the null fraction from one for each of the relations, 
-   and divide by the maximum  of the two distinct values. The number of rows 
-   that the join is likely to emit is calculated as the cardinality of 
-   Cartesian product of the two nodes in the nested-loop, multiplied by the 
+   This is, subtract the null fraction from one for each of the relations,
+   and divide by the maximum of the numbers of distinct values.
+   The number of rows
+   that the join is likely to emit is calculated as the cardinality of the
+   Cartesian product of the two inputs, multiplied by the
    selectivity:
 
 <programlisting>
 rows = (outer_cardinality * inner_cardinality) * selectivity
-     = (51 * 10000) * 0.0001
-     = 51
+     = (50 * 10000) * 0.0001
+     = 50
 </programlisting>
   </para>
 
   <para>
-   For those interested in further details, estimation of the number of rows in
-   a relation is covered in 
-   <filename>src/backend/optimizer/util/plancat.c</filename>. The calculation
-   logic for clause selectivities is in 
-   <filename>src/backend/optimizer/path/clausesel.c</filename>. The actual
-   implementations of the operator and join restriction functions can be found 
+   Had there been MCV lists for the two columns,
+   <function>eqjoinsel</function> would have used direct comparison of the MCV
+   lists to determine the join selectivity within the part of the column
+   populations represented by the MCVs.  The estimate for the remainder of the
+   populations follows the same approach shown here.
+  </para>
+
+  <para>
+   Notice that we showed <literal>inner_cardinality</> as 10000, that is,
+   the unmodified size of <structname>tenk2</>.  It might appear from
+   inspection of the <command>EXPLAIN</> output that the estimate of
+   join rows comes from 50 * 1, that is, the number of outer rows times
+   the estimated number of rows obtained by each inner indexscan on
+   <structname>tenk2</>.  But this is not the case: the join relation size
+   is estimated before any particular join plan has been considered.  If
+   everything is working well then the two ways of estimating the join
+   size will produce about the same answer, but due to roundoff error and
+   other factors they sometimes diverge significantly.
+  </para>
+
+  <para>
+   For those interested in further details, estimation of the size of
+   a table (before any <literal>WHERE</> clauses) is done in
+   <filename>src/backend/optimizer/util/plancat.c</filename>. The generic
+   logic for clause selectivities is in
+   <filename>src/backend/optimizer/path/clausesel.c</filename>.  The
+   operator-specific selectivity functions are mostly found
    in <filename>src/backend/utils/adt/selfuncs.c</filename>.
   </para>
 
  </sect1>
 
-
 </chapter>