Skip to content

P
Postgres FD Implementation
Commit 7e96269a authored by Tatsuo Ishii's avatar Tatsuo Ishii
Browse files
Options

Various pgbench enhancements. Patch contributed by ITAGAKI Takahiro. 

Also tweak README.pgbench/README.pgbench_jis:
  Remove history after pgbench was added to PostgreSQL contrib module.
  Those info was not only redundant since it has already been in CVS
  log, but also incomplete.
--------------------------------------------------------------------------
The attached is a patch to optimize contrib/pgbench using new 8.3 features.

- Use DROP IF EXISTS to suppress errors for initial loadings.
- Use a combination of TRUNCATE and COPY to reduce WAL on creating
  the accounts table.

Also, there are some cosmetic changes.

- Change the output of -v option from "starting full vacuum..."
  to "starting vacuum accounts..." in reflection of the fact.
- Shape duplicated error checks into executeStatement().


There is a big performance win in "COPY with no WAL" feature.
Thanks for the efforts!
--------------------------------------------------------------------------
parent 37a609b2
Show whitespace changes
Inline Side-by-side
Showing with 75 additions and 254 deletions
contrib/pgbench/README.pgbench
View file @ 7e96269a
pgbench README 2006/10/21 Tatsuo Ishii $PostgreSQL: pgsql/contrib/pgbench/README.pgbench,v 1.16 2007/04/06 08:49:44 ishii Exp $
pgbench README
o What is pgbench? o What is pgbench?
...@@ -233,54 +235,7 @@ o License? ...@@ -233,54 +235,7 @@ o License?
Basically it is same as BSD license. See pgbench.c for more details. Basically it is same as BSD license. See pgbench.c for more details.
o History o History before contributed to PostgreSQL
2006/10/21
* more fix with handling default scaling factor in the default
scenarios
2006/09/14
* change "tps" to "scale" to avoid confusion
* fix bug with handling default scaling factor in the default
scenarios
2006/07/26
* New features contributed by Tomoaki Sato.
* predefined variable "tps"
The value of variable tps is taken from the scaling factor
specified by -s option.
* -D option
Variable values can be defined by -D option.
* \set command now allows arithmetic calculations.
2005/09/29
* add -f option. contributed by Tomoaki Sato.
[updation records were missing]
2003/11/26
* create indexes after data insertion to reduce time.
patch from Yutaka Tanida.
2003/06/10
* fix uninitialized memory bug
* add support for PGHOST, PGPORT, PGUSER environment variables
2002/07/20
* patch contributed by Neil Conway.
* code/document clean up and add -l option.
2002/02/24
* do not CHECKPOINT anymore while initializing benchmark
* database. Add -N option.
2001/10/24
* "time"->"mtime"
2001/09/09
* Add -U, -P, -C options
2000/1/15 pgbench-1.2 contributed to PostgreSQL 2000/1/15 pgbench-1.2 contributed to PostgreSQL
* Add -v option * Add -v option
......
contrib/pgbench/README.pgbench_jis
View file @ 7e96269a
pgbench README 2006/10/21 Tatsuo Ishii $PostgreSQL: pgsql/contrib/pgbench/README.pgbench_jis,v 1.17 2007/04/06 08:49:44 ishii Exp $
pgbench README
$B"#(Bpgbench $B$H$O!)(B $B"#(Bpgbench $B$H$O!)(B
...@@ -283,57 +285,7 @@ pgbench $B$O@P0f(B $BC#IW$K$h$C$F=q$+$l$^$7$?!%%i%$%;%s%9>r7o$O(B pgbench.c ...@@ -283,57 +285,7 @@ pgbench $B$O@P0f(B $BC#IW$K$h$C$F=q$+$l$^$7$?!%%i%$%;%s%9>r7o$O(B pgbench.c
$BKAF,$K=q$$$F$"$j$^$9!%$3$N>r7o$r<i$k8B$jL5=~$GMxMQ$7!$$^$?<+M3$K:FG[IU(B $BKAF,$K=q$$$F$"$j$^$9!%$3$N>r7o$r<i$k8B$jL5=~$GMxMQ$7!$$^$?<+M3$K:FG[IU(B
$B$G$-$^$9!%(B $B$G$-$^$9!%(B
$B"#2~DjMzNr(B $B"#(BPostgreSQL$B$N(Bcontrib$B%b%8%e!<%k$H$7$F<h$j9~$^$l$k$^$G$N2~DjMzNr(B
2006/10/21
* $B99$K%G%U%)%k%H$N%9%1!<%j%s%0%U%!%/%?$r(Bbranches$B$+$i<h$C$F$3$J(B
$B$$%P%0$r=$@5!%(B
2006/09/13
* $BJQ?t(Btps$B$OJ6$i$o$7$$$N$G(Bscale$B$KJQ99!%%G%U%)%k%H%7%J%j%*$N;~$K!$(B
$B%G%U%)%k%H$N%9%1!<%j%s%0%U%!%/%?$r(Bbranches$B$+$i<h$C$F$3$J$$%P%0$r=$@5!%(B
2006/07/26
* $B:4F#$5$s$N%Q%C%A$rE,MQ!%0J2<$N5!G=DI2C!%(BPostgreSQL 8.2$B$K<h$j(B
$B9~$^$l$^$9!%(B
$BJQ?t(B tps
-s $B%*%W%7%g%s$G;XDj$7$?%9%1!<%j%s%0%U%!%/%?!<$r%U%!%$%kFb$GJQ?t$H$7(B
$B$F;2>H$9$k5!G=(B
-D $B%*%W%7%g%s(B
$B%3%^%s%I$N%*%W%7%g%s$H$7$FDj5A$7$?JQ?t$r%U%!%$%kFb$+$i;2>H$9$k5!G=(B
\set $B%3%^%s%I(B
$B%U%!%$%kFb$G;MB'1i;;$r9T$$!"$=$N7k2L$rJQ?t$KBeF~$9$k5!G=(B
2005/09/29
* $B:4F#$5$s$N%Q%C%A$rE,MQ!%(B-f $B%*%W%7%g%s$NDI2C!%(B
[$B$3$N4V$$$m$$$mJQ99$,$"$C$?$h$&$@$,(BREADME$B$O%a%$%s%F%J%s%9$5$l$F$$$J$$(B]
2003/11/26
* $BC+ED$5$s$N%Q%C%A$rE,MQ!%(Bpgbench -i$B$N:]$K!$8e$+$i<g%-!<$r:n@.(B
$B$9$k$h$&$K$7$?!%$3$l$K$h$C$F=i4|2=$N<B9T;~4V$,BgI}$KC;=L$G$-(B
$B$k(B($B$O$:(B)$B!%(B
2003/06/10
* $B%a%b%j$,=i4|2=$5$l$F$$$J$$%P%0$r=$@5(B
* $B4D6-JQ?t(BPGHOST, PGPORT, PGUSER$B$rG'<1$9$k$h$&$K$7$?!%(B
2002/07/20
* Nei Conway$B$5$s$N%Q%C%A$rE,MQ!%(B
* -l $B%*%W%7%g%s$NDI2C!%(B
2002/02/24
* $B$3$3$+$i$O(B7.3$BMQ$NJQ99$G$9!%(B
* CHECKPOINT$B$NH/9T$r$d$a$^$7$?!%(B
* -N $B%*%W%7%g%s$rDI2C$7$^$7$?!%(B
2001/10/24
* PostgreSQL 7.2$B$G!$(B"time"$B$,M=Ls8l$K$J$C$?$N$G!$(B"mtime"$B$KJQ99(B
$B$7$?!%(B
2001/09/09
* PostgreSQL 7.2$BMQ$K!$(B-U, -P, -C $B%*%W%7%g%s$rDI2C$7$^$7$?!%(B
2000/1/15 pgbench-1.2 $B$O(B PostgreSQL $B$K(B contribute $B$5$l$^$7$?!%(B 2000/1/15 pgbench-1.2 $B$O(B PostgreSQL $B$K(B contribute $B$5$l$^$7$?!%(B
* -v $B%*%W%7%g%sDI2C(B * -v $B%*%W%7%g%sDI2C(B
......
contrib/pgbench/pgbench.c
View file @ 7e96269a
/* /*
* $PostgreSQL: pgsql/contrib/pgbench/pgbench.c,v 1.62 2007/03/13 09:06:35 mha Exp $ * $PostgreSQL: pgsql/contrib/pgbench/pgbench.c,v 1.63 2007/04/06 08:49:44 ishii Exp $
* *
* pgbench: a simple benchmark program for PostgreSQL * pgbench: a simple benchmark program for PostgreSQL
* written by Tatsuo Ishii * written by Tatsuo Ishii
...@@ -188,12 +188,26 @@ getrand(int min, int max) ...@@ -188,12 +188,26 @@ getrand(int min, int max)
return min + (int) (((max - min) * (double) random()) / MAX_RANDOM_VALUE + 0.5); return min + (int) (((max - min) * (double) random()) / MAX_RANDOM_VALUE + 0.5);
} }
/* call PQexec() and exit() on failure */
static void
executeStatement(PGconn *con, const char* sql)
{
PGresult *res;
res = PQexec(con, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
fprintf(stderr, "%s", PQerrorMessage(con));
exit(1);
}
PQclear(res);
}
/* set up a connection to the backend */ /* set up a connection to the backend */
static PGconn * static PGconn *
doConnect(void) doConnect(void)
{ {
PGconn *con; PGconn *con;
PGresult *res;
con = PQsetdbLogin(pghost, pgport, pgoptions, pgtty, dbName, con = PQsetdbLogin(pghost, pgport, pgoptions, pgtty, dbName,
login, pwd); login, pwd);
...@@ -216,13 +230,7 @@ doConnect(void) ...@@ -216,13 +230,7 @@ doConnect(void)
return (NULL); return (NULL);
} }
res = PQexec(con, "SET search_path = public"); executeStatement(con, "SET search_path = public");
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
fprintf(stderr, "%s", PQerrorMessage(con));
exit(1);
}
PQclear(res);
return (con); return (con);
} }
...@@ -720,13 +728,13 @@ init(void) ...@@ -720,13 +728,13 @@ init(void)
PGconn *con; PGconn *con;
PGresult *res; PGresult *res;
static char *DDLs[] = { static char *DDLs[] = {
"drop table branches", "drop table if exists branches",
"create table branches(bid int not null,bbalance int,filler char(88))", "create table branches(bid int not null,bbalance int,filler char(88))",
"drop table tellers", "drop table if exists tellers",
"create table tellers(tid int not null,bid int,tbalance int,filler char(84))", "create table tellers(tid int not null,bid int,tbalance int,filler char(84))",
"drop table accounts", "drop table if exists accounts",
"create table accounts(aid int not null,bid int,abalance int,filler char(84))", "create table accounts(aid int not null,bid int,abalance int,filler char(84))",
"drop table history", "drop table if exists history",
"create table history(tid int,bid int,aid int,delta int,mtime timestamp,filler char(22))"}; "create table history(tid int,bid int,aid int,delta int,mtime timestamp,filler char(22))"};
static char *DDLAFTERs[] = { static char *DDLAFTERs[] = {
"alter table branches add primary key (bid)", "alter table branches add primary key (bid)",
...@@ -741,68 +749,34 @@ init(void) ...@@ -741,68 +749,34 @@ init(void)
if ((con = doConnect()) == NULL) if ((con = doConnect()) == NULL)
exit(1); exit(1);
for (i = 0; i < (sizeof(DDLs) / sizeof(char *)); i++) for (i = 0; i < lengthof(DDLs); i++)
{ executeStatement(con, DDLs[i]);
res = PQexec(con, DDLs[i]);
if (strncmp(DDLs[i], "drop", 4) && PQresultStatus(res) != PGRES_COMMAND_OK)
{
fprintf(stderr, "%s", PQerrorMessage(con));
exit(1);
}
PQclear(res);
}
res = PQexec(con, "begin"); executeStatement(con, "begin");
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
fprintf(stderr, "%s", PQerrorMessage(con));
exit(1);
}
PQclear(res);
for (i = 0; i < nbranches * scale; i++) for (i = 0; i < nbranches * scale; i++)
{ {
snprintf(sql, 256, "insert into branches(bid,bbalance) values(%d,0)", i + 1); snprintf(sql, 256, "insert into branches(bid,bbalance) values(%d,0)", i + 1);
res = PQexec(con, sql); executeStatement(con, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
fprintf(stderr, "%s", PQerrorMessage(con));
exit(1);
}
PQclear(res);
} }
for (i = 0; i < ntellers * scale; i++) for (i = 0; i < ntellers * scale; i++)
{ {
snprintf(sql, 256, "insert into tellers(tid,bid,tbalance) values (%d,%d,0)" snprintf(sql, 256, "insert into tellers(tid,bid,tbalance) values (%d,%d,0)"
,i + 1, i / ntellers + 1); ,i + 1, i / ntellers + 1);
res = PQexec(con, sql); executeStatement(con, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
fprintf(stderr, "%s", PQerrorMessage(con));
exit(1);
}
PQclear(res);
} }
res = PQexec(con, "end"); executeStatement(con, "commit");
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
fprintf(stderr, "%s", PQerrorMessage(con));
exit(1);
}
PQclear(res);
/* /*
* occupy accounts table with some data * fill the accounts table with some data
*/ */
fprintf(stderr, "creating tables...\n"); fprintf(stderr, "creating tables...\n");
for (i = 0; i < naccounts * scale; i++)
{
int j = i + 1;
if (j % 10000 == 1) executeStatement(con, "begin");
{ executeStatement(con, "truncate accounts");
res = PQexec(con, "copy accounts from stdin"); res = PQexec(con, "copy accounts from stdin");
if (PQresultStatus(res) != PGRES_COPY_IN) if (PQresultStatus(res) != PGRES_COPY_IN)
{ {
...@@ -810,7 +784,10 @@ init(void) ...@@ -810,7 +784,10 @@ init(void)
exit(1); exit(1);
} }
PQclear(res); PQclear(res);
}
for (i = 0; i < naccounts * scale; i++)
{
int j = i + 1;
snprintf(sql, 256, "%d\t%d\t%d\t\n", j, i / naccounts + 1, 0); snprintf(sql, 256, "%d\t%d\t%d\t\n", j, i / naccounts + 1, 0);
if (PQputline(con, sql)) if (PQputline(con, sql))
...@@ -820,62 +797,32 @@ init(void) ...@@ -820,62 +797,32 @@ init(void)
} }
if (j % 10000 == 0) if (j % 10000 == 0)
{
/*
* every 10000 tuples, we commit the copy command. this should
* avoid generating too much WAL logs
*/
fprintf(stderr, "%d tuples done.\n", j); fprintf(stderr, "%d tuples done.\n", j);
}
if (PQputline(con, "\\.\n")) if (PQputline(con, "\\.\n"))
{ {
fprintf(stderr, "very last PQputline failed\n"); fprintf(stderr, "very last PQputline failed\n");
exit(1); exit(1);
} }
if (PQendcopy(con)) if (PQendcopy(con))
{ {
fprintf(stderr, "PQendcopy failed\n"); fprintf(stderr, "PQendcopy failed\n");
exit(1); exit(1);
} }
executeStatement(con, "commit");
#ifdef NOT_USED
/* /*
* do a checkpoint to purge the old WAL logs * create indexes
*/ */
res = PQexec(con, "checkpoint");
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
fprintf(stderr, "%s", PQerrorMessage(con));
exit(1);
}
PQclear(res);
#endif /* NOT_USED */
}
}
fprintf(stderr, "set primary key...\n"); fprintf(stderr, "set primary key...\n");
for (i = 0; i < (sizeof(DDLAFTERs) / sizeof(char *)); i++) for (i = 0; i < lengthof(DDLAFTERs); i++)
{ executeStatement(con, DDLAFTERs[i]);
res = PQexec(con, DDLAFTERs[i]);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
fprintf(stderr, "%s", PQerrorMessage(con));
exit(1);
}
PQclear(res);
}
/* vacuum */ /* vacuum */
fprintf(stderr, "vacuum..."); fprintf(stderr, "vacuum...");
res = PQexec(con, "vacuum analyze"); executeStatement(con, "vacuum analyze");
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
fprintf(stderr, "%s", PQerrorMessage(con));
exit(1);
}
PQclear(res);
fprintf(stderr, "done.\n");
fprintf(stderr, "done.\n");
PQfinish(con); PQfinish(con);
} }
...@@ -1155,7 +1102,7 @@ main(int argc, char **argv) ...@@ -1155,7 +1102,7 @@ main(int argc, char **argv)
int c; int c;
int is_init_mode = 0; /* initialize mode? */ int is_init_mode = 0; /* initialize mode? */
int is_no_vacuum = 0; /* no vacuum at all before testing? */ int is_no_vacuum = 0; /* no vacuum at all before testing? */
int is_full_vacuum = 0; /* do full vacuum before testing? */ int do_vacuum_accounts = 0; /* do vacuum accounts before testing? */
int debug = 0; /* debug flag */ int debug = 0; /* debug flag */
int ttype = 0; /* transaction type. 0: TPC-B, 1: SELECT only, int ttype = 0; /* transaction type. 0: TPC-B, 1: SELECT only,
* 2: skip update of branches and tellers */ * 2: skip update of branches and tellers */
...@@ -1219,7 +1166,7 @@ main(int argc, char **argv) ...@@ -1219,7 +1166,7 @@ main(int argc, char **argv)
is_no_vacuum++; is_no_vacuum++;
break; break;
case 'v': case 'v':
is_full_vacuum++; do_vacuum_accounts++;
break; break;
case 'p': case 'p':
pgport = optarg; pgport = optarg;
...@@ -1456,49 +1403,16 @@ main(int argc, char **argv) ...@@ -1456,49 +1403,16 @@ main(int argc, char **argv)
if (!is_no_vacuum) if (!is_no_vacuum)
{ {
fprintf(stderr, "starting vacuum..."); fprintf(stderr, "starting vacuum...");
res = PQexec(con, "vacuum branches"); executeStatement(con, "vacuum branches");
if (PQresultStatus(res) != PGRES_COMMAND_OK) executeStatement(con, "vacuum tellers");
{ executeStatement(con, "delete from history");
fprintf(stderr, "%s", PQerrorMessage(con)); executeStatement(con, "vacuum history");
exit(1);
}
PQclear(res);
res = PQexec(con, "vacuum tellers");
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
fprintf(stderr, "%s", PQerrorMessage(con));
exit(1);
}
PQclear(res);
res = PQexec(con, "delete from history");
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
fprintf(stderr, "%s", PQerrorMessage(con));
exit(1);
}
PQclear(res);
res = PQexec(con, "vacuum history");
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
fprintf(stderr, "%s", PQerrorMessage(con));
exit(1);
}
PQclear(res);
fprintf(stderr, "end.\n"); fprintf(stderr, "end.\n");
if (is_full_vacuum) if (do_vacuum_accounts)
{
fprintf(stderr, "starting full vacuum...");
res = PQexec(con, "vacuum analyze accounts");
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{ {
fprintf(stderr, "%s", PQerrorMessage(con)); fprintf(stderr, "starting vacuum accounts...");
exit(1); executeStatement(con, "vacuum analyze accounts");
}
PQclear(res);
fprintf(stderr, "end.\n"); fprintf(stderr, "end.\n");
} }
} }
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment