Skip to content

P
Postgres FD Implementation
Commit b13c9686 authored by Tom Lane's avatar Tom Lane
Browse files
Options

Take the statistics collector out of the loop for monitoring backends' 

current commands; instead, store current-status information in shared
memory.  This substantially reduces the overhead of stats_command_string
and also ensures that pg_stat_activity is fully up to date at all times.
Per my recent proposal.
parent 6075feed
Expand all Show whitespace changes
Inline Side-by-side
Showing with 838 additions and 1156 deletions
doc/src/sgml/config.sgml
View file @ b13c9686
<!-- $PostgreSQL: pgsql/doc/src/sgml/config.sgml,v 1.65 2006/06/18 15:38:35 petere Exp $ -->
<!-- $PostgreSQL: pgsql/doc/src/sgml/config.sgml,v 1.66 2006/06/19 01:51:21 tgl Exp $ -->
<chapter Id="runtime-config">
<title>Server Configuration</title>
......@@ -2845,43 +2845,6 @@ SELECT * FROM parent WHERE key = 2400;
<sect1 id="runtime-config-statistics">
<title>Run-Time Statistics</title>
<sect2 id="runtime-config-statistics-monitor">
<title>Statistics Monitoring</title>
<variablelist>
<varlistentry>
<term><varname>log_statement_stats</varname> (<type>boolean</type>)</term>
<term><varname>log_parser_stats</varname> (<type>boolean</type>)</term>
<term><varname>log_planner_stats</varname> (<type>boolean</type>)</term>
<term><varname>log_executor_stats</varname> (<type>boolean</type>)</term>
<indexterm>
<primary><varname>log_statement_stats</> configuration parameter</primary>
</indexterm>
<indexterm>
<primary><varname>log_parser_stats</> configuration parameter</primary>
</indexterm>
<indexterm>
<primary><varname>log_planner_stats</> configuration parameter</primary>
</indexterm>
<indexterm>
<primary><varname>log_executor_stats</> configuration parameter</primary>
</indexterm>
<listitem>
<para>
For each query, write performance statistics of the respective
module to the server log. This is a crude profiling
instrument. <varname>log_statement_stats</varname> reports total
statement statistics, while the others report per-module statistics.
<varname>log_statement_stats</varname> cannot be enabled together with
any of the per-module options. All of these options are disabled by
default. Only superusers can change these settings.
</para>
</listitem>
</varlistentry>
</variablelist>
</sect2>
<sect2 id="runtime-config-statistics-collector">
<title>Query and Index Statistics Collector</title>
......@@ -2893,26 +2856,18 @@ SELECT * FROM parent WHERE key = 2400;
Refer to <xref linkend="monitoring"> for more information.
</para>
<variablelist>
<varlistentry id="guc-stats-start-collector" xreflabel="stats_start_collector">
<term><varname>stats_start_collector</varname> (<type>boolean</type>)</term>
<indexterm>
<primary><varname>stats_start_collector</> configuration parameter</primary>
</indexterm>
<listitem>
<note>
<para>
Controls whether the server should start the
statistics-collection subprocess. This is on by default, but
may be turned off if you know you have no interest in
collecting statistics or running autovacuum.
This parameter can only be set at server start, because the collection
subprocess cannot be started or stopped on-the-fly. (However, the
extent to which statistics are actually gathered can be changed while
the server is running, so long as the subprocess exists.)
As of <productname>PostgreSQL</productname> 8.2,
<varname>stats_command_string</varname> controls a separate data
collection mechanism that can be turned on or off independently
of whether the statistics-collection subprocess is running.
The subprocess is only needed to support collection of
block-level or row-level statistics.
</para>
</listitem>
</varlistentry>
</note>
<variablelist>
<varlistentry id="guc-stats-command-string" xreflabel="stats_command_string">
<term><varname>stats_command_string</varname> (<type>boolean</type>)</term>
......@@ -2921,7 +2876,7 @@ SELECT * FROM parent WHERE key = 2400;
</indexterm>
<listitem>
<para>
Enables the collection of statistics on the currently
Enables the collection of information on the currently
executing command of each session, along with the time at
which that command began execution. This parameter is off by
default. Note that even when enabled, this information is not
......@@ -2933,6 +2888,25 @@ SELECT * FROM parent WHERE key = 2400;
</listitem>
</varlistentry>
<varlistentry id="guc-stats-start-collector" xreflabel="stats_start_collector">
<term><varname>stats_start_collector</varname> (<type>boolean</type>)</term>
<indexterm>
<primary><varname>stats_start_collector</> configuration parameter</primary>
</indexterm>
<listitem>
<para>
Controls whether the server should start the
statistics-collection subprocess. This is on by default, but
may be turned off if you know you have no interest in
collecting statistics or running autovacuum.
This parameter can only be set at server start, because the collection
subprocess cannot be started or stopped on-the-fly. (However, the
extent to which statistics are actually gathered can be changed while
the server is running, so long as the subprocess exists.)
</para>
</listitem>
</varlistentry>
<varlistentry id="guc-stats-block-level" xreflabel="stats_block_level">
<term><varname>stats_block_level</varname> (<type>boolean</type>)</term>
<indexterm>
......@@ -2968,15 +2942,53 @@ SELECT * FROM parent WHERE key = 2400;
</indexterm>
<listitem>
<para>
If on, collected statistics are zeroed out whenever the server
is restarted. If off, statistics are accumulated across server
restarts. The default is <literal>off</>. This parameter can only
be set at server start.
If on, collected block-level and row-level statistics are zeroed out
whenever the server is restarted. If off, statistics are accumulated
across server restarts. This parameter is off by default.
This parameter can only be set at server start.
</para>
</listitem>
</varlistentry>
</variablelist>
</sect2>
<sect2 id="runtime-config-statistics-monitor">
<title>Statistics Monitoring</title>
<variablelist>
<varlistentry>
<term><varname>log_statement_stats</varname> (<type>boolean</type>)</term>
<term><varname>log_parser_stats</varname> (<type>boolean</type>)</term>
<term><varname>log_planner_stats</varname> (<type>boolean</type>)</term>
<term><varname>log_executor_stats</varname> (<type>boolean</type>)</term>
<indexterm>
<primary><varname>log_statement_stats</> configuration parameter</primary>
</indexterm>
<indexterm>
<primary><varname>log_parser_stats</> configuration parameter</primary>
</indexterm>
<indexterm>
<primary><varname>log_planner_stats</> configuration parameter</primary>
</indexterm>
<indexterm>
<primary><varname>log_executor_stats</> configuration parameter</primary>
</indexterm>
<listitem>
<para>
For each query, write performance statistics of the respective
module to the server log. This is a crude profiling
instrument. <varname>log_statement_stats</varname> reports total
statement statistics, while the others report per-module statistics.
<varname>log_statement_stats</varname> cannot be enabled together with
any of the per-module options. All of these options are disabled by
default. Only superusers can change these settings.
</para>
</listitem>
</varlistentry>
</variablelist>
</sect2>
</sect1>
......
doc/src/sgml/monitoring.sgml
View file @ b13c9686
<!-- $PostgreSQL: pgsql/doc/src/sgml/monitoring.sgml,v 1.33 2006/06/18 15:38:35 petere Exp $ -->
<!-- $PostgreSQL: pgsql/doc/src/sgml/monitoring.sgml,v 1.34 2006/06/19 01:51:21 tgl Exp $ -->
<chapter id="monitoring">
<title>Monitoring Database Activity</title>
......@@ -109,9 +109,14 @@ postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
<productname>PostgreSQL</productname>'s <firstterm>statistics collector</>
is a subsystem that supports collection and reporting of information about
server activity. Presently, the collector can count accesses to tables
and indexes in both disk-block and individual-row terms. It also supports
determining the exact command currently being executed by other server
processes.
and indexes in both disk-block and individual-row terms.
</para>
<para>
<productname>PostgreSQL</productname> also supports determining the exact
command currently being executed by other server processes. This is an
independent facility that can be enabled or disabled whether or not
block-level and row-level statistics are being collected.
</para>
<sect2 id="monitoring-stats-setup">
......@@ -136,15 +141,21 @@ postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
</para>
<para>
The parameters <xref linkend="guc-stats-command-string">,
<xref linkend="guc-stats-block-level">, and <xref
The parameters <xref linkend="guc-stats-block-level"> and <xref
linkend="guc-stats-row-level"> control how much information is
actually sent to the collector and thus determine how much run-time
overhead occurs. These respectively determine whether a server
process sends its current command string, disk-block-level access
statistics, and row-level access statistics to the collector.
process tracks disk-block-level access
statistics and row-level access statistics and sends these to the collector.
Additionally, per-database transaction commit and abort statistics
are collected if any of these parameters are set.
are collected if either of these parameters are set.
</para>
<para>
The parameter <xref linkend="guc-stats-command-string"> enables monitoring
of the current command being executed by any server process.
The statistics collector subprocess need not be running to enable this
feature.
</para>
<para>
......@@ -165,7 +176,7 @@ postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
very few statistics are collected in the default
configuration. Enabling one or more of these configuration
variables will significantly enhance the amount of useful data
produced by the statistics collector, at the expense of
produced by the statistics facilities, at the expense of
additional run-time overhead.
</para>
</note>
......@@ -190,10 +201,9 @@ postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
progress does not affect the displayed totals. Also, the collector itself
emits a new report at most once per <varname>PGSTAT_STAT_INTERVAL</varname>
milliseconds (500 unless altered while building the server). So the
displayed information lags behind actual activity. Current-query
information is reported to the collector immediately, but is still subject
to the <varname>PGSTAT_STAT_INTERVAL</varname> delay before it becomes
visible.
displayed information lags behind actual activity. However, current-query
information collected by <varname>stats_command_string</varname> is
always up-to-date.
</para>
<para>
......@@ -202,7 +212,10 @@ postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
the collector process and then continues to use this snapshot for all
statistical views and functions until the end of its current transaction.
So the statistics will appear not to change as long as you continue the
current transaction.
current transaction. Similarly, information about the current queries of
all processes is collected when any such information is first requested
within a transaction, and the same information will be displayed throughout
the transaction.
This is a feature, not a bug, because it allows you to perform several
queries on the statistics and correlate the results without worrying that
the numbers are changing underneath you. But if you want to see new
......@@ -232,9 +245,8 @@ postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
parameter <varname>stats_command_string</varname> has been
turned on. Furthermore, these columns read as null unless the
user examining the view is a superuser or the same as the user
owning the process being reported on. (Note that because of the
collector's reporting delay, the current query will only be
up-to-date for long-running queries.)</entry>
owning the process being reported on.
</entry>
</row>
<row>
......@@ -549,7 +561,7 @@ postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
<entry><literal><function>pg_stat_get_last_vacuum_time</function>(<type>oid</type>)</literal></entry>
<entry><type>timestamptz</type></entry>
<entry>
Time of the last vacuum initiated by the user
Time of the last vacuum initiated by the user on this table
</entry>
</row>
......@@ -557,7 +569,7 @@ postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
<entry><literal><function>pg_stat_get_last_autovacuum_time</function>(<type>oid</type>)</literal></entry>
<entry><type>timestamptz</type></entry>
<entry>
Time of the last vacuum initiated by the autovacuum daemon
Time of the last vacuum initiated by the autovacuum daemon on this table
</entry>
</row>
......@@ -565,7 +577,7 @@ postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
<entry><literal><function>pg_stat_get_last_analyze_time</function>(<type>oid</type>)</literal></entry>
<entry><type>timestamptz</type></entry>
<entry>
Time of the last analyze initiated by the user
Time of the last analyze initiated by the user on this table
</entry>
</row>
......@@ -573,7 +585,8 @@ postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
<entry><literal><function>pg_stat_get_last_autoanalyze_time</function>(<type>oid</type>)</literal></entry>
<entry><type>timestamptz</type></entry>
<entry>
Time of the last analyze initiated by the autovacuum daemon
Time of the last analyze initiated by the autovacuum daemon on this
table
</entry>
</row>
......@@ -677,7 +690,7 @@ postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
<entry><literal><function>pg_stat_reset</function>()</literal></entry>
<entry><type>boolean</type></entry>
<entry>
Reset all currently collected statistics
Reset all block-level and row-level statistics to zero
</entry>
</row>
</tbody>
......
src/backend/postmaster/pgstat.c
View file @ b13c9686
This diff is collapsed.
src/backend/postmaster/postmaster.c
View file @ b13c9686
......@@ -37,7 +37,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/postmaster/postmaster.c,v 1.486 2006/06/18 15:38:37 petere Exp $
* $PostgreSQL: pgsql/src/backend/postmaster/postmaster.c,v 1.487 2006/06/19 01:51:21 tgl Exp $
*
* NOTES
*
......@@ -2109,9 +2109,6 @@ reaper(SIGNAL_ARGS)
{
AutoVacPID = 0;
autovac_stopped();
/* Tell the collector about process termination */
pgstat_beterm(pid);
if (exitstatus != 0)
HandleChildCrash(pid, exitstatus,
_("autovacuum process"));
......@@ -2252,8 +2249,6 @@ CleanupBackend(int pid,
#ifdef EXEC_BACKEND
ShmemBackendArrayRemove(pid);
#endif
/* Tell the collector about backend termination */
pgstat_beterm(pid);
break;
}
}
......@@ -2299,8 +2294,6 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
#ifdef EXEC_BACKEND
ShmemBackendArrayRemove(pid);
#endif
/* Tell the collector about backend termination */
pgstat_beterm(pid);
/* Keep looping so we can signal remaining backends */
}
else
......
src/backend/storage/ipc/ipci.c
View file @ b13c9686
......@@ -8,7 +8,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/storage/ipc/ipci.c,v 1.83 2006/05/08 00:00:10 tgl Exp $
* $PostgreSQL: pgsql/src/backend/storage/ipc/ipci.c,v 1.84 2006/06/19 01:51:21 tgl Exp $
*
*-------------------------------------------------------------------------
*/
......@@ -21,6 +21,7 @@
#include "access/twophase.h"
#include "access/xlog.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "postmaster/bgwriter.h"
#include "postmaster/postmaster.h"
#include "storage/bufmgr.h"
......@@ -86,6 +87,7 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
size = add_size(size, MultiXactShmemSize());
size = add_size(size, LWLockShmemSize());
size = add_size(size, ProcArrayShmemSize());
size = add_size(size, BackendStatusShmemSize());
size = add_size(size, SInvalShmemSize());
size = add_size(size, FreeSpaceShmemSize());
size = add_size(size, BgWriterShmemSize());
......@@ -167,6 +169,7 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
if (!IsUnderPostmaster)
InitProcGlobal();
CreateSharedProcArray();
CreateSharedBackendStatus();
/*
* Set up shared-inval messaging
......
src/backend/storage/ipc/procarray.c
View file @ b13c9686
......@@ -23,7 +23,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/storage/ipc/procarray.c,v 1.11 2006/03/05 15:58:37 momjian Exp $
* $PostgreSQL: pgsql/src/backend/storage/ipc/procarray.c,v 1.12 2006/06/19 01:51:21 tgl Exp $
*
*-------------------------------------------------------------------------
*/
......@@ -732,42 +732,6 @@ IsBackendPid(int pid)
return (BackendPidGetProc(pid) != NULL);
}
/*
* GetAllBackendPids -- get an array of all current backends' PIDs
*
* The result is a palloc'd array with the number of active backends in
* entry [0], their PIDs in entries [1] .. [n]. The caller must bear in
* mind that the result may already be obsolete when returned.
*/
int *
GetAllBackendPids(void)
{
int *result;
int npids;
ProcArrayStruct *arrayP = procArray;
int index;
result = (int *) palloc((MaxBackends + 1) * sizeof(int));
npids = 0;
LWLockAcquire(ProcArrayLock, LW_SHARED);
for (index = 0; index < arrayP->numProcs; index++)
{
PGPROC *proc = arrayP->procs[index];
if (proc->pid != 0) /* ignore dummy procs */
result[++npids] = proc->pid;
}
LWLockRelease(ProcArrayLock);
Assert(npids <= MaxBackends);
result[0] = npids;
return result;
}
/*
* CountActiveBackends --- count backends (other than myself) that are in
* active transactions. This is used as a heuristic to decide if
......
src/backend/utils/adt/pgstatfuncs.c
View file @ b13c9686
This diff is collapsed.
src/backend/utils/misc/guc.c
View file @ b13c9686
......@@ -10,7 +10,7 @@
* Written by Peter Eisentraut <peter_e@gmx.net>.
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/utils/misc/guc.c,v 1.321 2006/06/05 02:49:58 tgl Exp $
* $PostgreSQL: pgsql/src/backend/utils/misc/guc.c,v 1.322 2006/06/19 01:51:21 tgl Exp $
*
*--------------------------------------------------------------------
*/
......@@ -700,16 +700,6 @@ static struct config_bool ConfigureNamesBool[] =
&pgstat_collect_resetonpmstart,
false, NULL, NULL
},
{
{"stats_command_string", PGC_SUSET, STATS_COLLECTOR,
gettext_noop("Collects statistics about executing commands."),
gettext_noop("Enables the collection of statistics on the currently "
"executing command of each session, along with the time "
"at which that command began execution.")
},
&pgstat_collect_querystring,
false, NULL, NULL
},
{
{"stats_row_level", PGC_SUSET, STATS_COLLECTOR,
gettext_noop("Collects row-level statistics on database activity."),
......@@ -727,6 +717,17 @@ static struct config_bool ConfigureNamesBool[] =
false, NULL, NULL
},
{
{"stats_command_string", PGC_SUSET, STATS_COLLECTOR,
gettext_noop("Collects information about executing commands."),
gettext_noop("Enables the collection of information on the currently "
"executing command of each session, along with the time "
"at which that command began execution.")
},
&pgstat_collect_querystring,
false, NULL, NULL
},
{
{"autovacuum", PGC_SIGHUP, AUTOVACUUM,
gettext_noop("Starts the autovacuum subprocess."),
......
src/backend/utils/misc/postgresql.conf.sample
View file @ b13c9686
......@@ -320,21 +320,21 @@
# RUNTIME STATISTICS
#---------------------------------------------------------------------------
# - Statistics Monitoring -
#log_parser_stats = off
#log_planner_stats = off
#log_executor_stats = off
#log_statement_stats = off
# - Query/Index Statistics Collector -
#stats_start_collector = on
#stats_command_string = off
#stats_start_collector = on # needed for block or row stats
#stats_block_level = off
#stats_row_level = off
#stats_reset_on_server_start = off
# - Statistics Monitoring -
#log_parser_stats = off
#log_planner_stats = off
#log_executor_stats = off
#log_statement_stats = off
#---------------------------------------------------------------------------
# AUTOVACUUM PARAMETERS
......
src/include/pgstat.h
View file @ b13c9686
This diff is collapsed.
src/include/storage/procarray.h
View file @ b13c9686
......@@ -7,7 +7,7 @@
* Portions Copyright (c) 1996-2006, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* $PostgreSQL: pgsql/src/include/storage/procarray.h,v 1.8 2006/03/05 15:59:00 momjian Exp $
* $PostgreSQL: pgsql/src/include/storage/procarray.h,v 1.9 2006/06/19 01:51:22 tgl Exp $
*
*-------------------------------------------------------------------------
*/
......@@ -29,7 +29,6 @@ extern TransactionId GetOldestXmin(bool allDbs);
extern PGPROC *BackendPidGetProc(int pid);
extern int BackendXidGetPid(TransactionId xid);
extern bool IsBackendPid(int pid);
extern int *GetAllBackendPids(void);
extern bool DatabaseHasActiveBackends(Oid databaseId, bool ignoreMyself);
extern int CountActiveBackends(void);
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment