Skip to content

P
Postgres FD Implementation
Commit c2d1b391 authored by Tom Lane's avatar Tom Lane
Browse files
Options

Code review: minor cleanups, make the world safe for unsigned OIDs. 

Improve documentation, too.
parent 90cfa9ac
Hide whitespace changes
Inline Side-by-side
Showing with 198 additions and 136 deletions
contrib/pg_dumplo/README.pg_dumplo
View file @ c2d1b391
$Header: /cvsroot/pgsql/contrib/pg_dumplo/Attic/README.pg_dumplo,v 1.2 2000/11/22 00:00:55 tgl Exp $
pg_dumplo - PostgreSQL large object dumper
==========================================
By Karel Zak <zakkr@zf.jcu.cz>
Compilation:
===========
* run master ./configure in the PG source top directory
* gmake all
* gmake install
THANKS:
======
<??? I lost his e-mail ???>
* option '--all' and pg_class usage
Pavel Janík ml. <Pavel.Janik@linux.cz>
* HOWTO (the rest of this file)
How to use pg_dumplo?
=====================
......@@ -19,7 +43,7 @@ CREATE DATABASE
Ok, our database with the name 'test' is created. Now we should create demo
table which will contain only one column with the name 'id' which will hold
the oid number of Large Object:
the OID number of a Large Object:
SnowWhite:$ psql test
Welcome to psql, the PostgreSQL interactive terminal.
......@@ -49,12 +73,12 @@ Object" - the file /etc/aliases. It has an oid of 19338 so we have inserted
this oid number to the database table lo to the column id. The final SELECT
shows that we have one record in the table.
Now we can demonstrate the work of pg_dumplo. We will create dump directory
Now we can demonstrate the work of pg_dumplo. We will create a dump directory
which will contain the whole dump of large objects (/tmp/dump):
mkdir -p /tmp/dump
Now we can dump all large objects from the database `test' which has an oid
Now we can dump all large objects from the database `test' which have OIDs
stored in the column `id' in the table `lo':
SnowWhite:$ pg_dumplo -s /tmp/dump -d test -l lo.id
......@@ -73,45 +97,52 @@ SnowWhite:$ tree /tmp/dump/
3 directories, 2 files
SnowWhite:$
Isn't it nice :-) Yes, it is, but we are on the half of our way. We should
also be able to recreate the contents of the table lo and the Large Object
database when something went wrong. It is very easy, we will demonstrate
this via dropping the database and recreating it from scratch with
pg_dumplo:
In practice, we'd probably use
SnowwWite:$ dropdb test
DROP DATABASE
SnowWhite:$ pg_dumplo -s /tmp/dump -d test -e
SnowWhite:$ createdb test
CREATE DATABASE
to export all large objects that are referenced by any OID-type column
in the database. Calling out specific column(s) with -l is only needed
for a selective dump.
Ok, our database with the name `test' is created again. We should also
create the table `lo' again:
For routine backup purposes, the dump directory could now be converted into
an archive file with tar and stored on tape. Notice that a single dump
directory can hold the dump of multiple databases.
SnowWhite:$ psql test
Welcome to psql, the PostgreSQL interactive terminal.
Now, how can we recreate the contents of the table lo and the Large Object
database when something went wrong? To do this, we expect that pg_dump is
also used to store the definition and contents of the regular tables in
the database.
Type: \copyright for distribution terms
\h for help with SQL commands
\? for help on internal slash commands
\g or terminate with semicolon to execute query
\q to quit
SnowWhite:$ pg_dump test >test.backup
test=# CREATE TABLE lo (id oid);
CREATE
test=# \q
SnowWhite:$
Now, if we lose the database:
Now the database with the table `lo' is created again, but we do not have
any information stored in it. But have the dump of complete Large Object
database, so we can recreate the contents of the whole database from the
directory /tmp/dump:
SnowWhite:$ dropdb test
DROP DATABASE
we can recreate it and reload the regular tables from the dump file:
SnowWhite:$ createdb test
CREATE DATABASE
SnowWhite:$ psql test <test.backup
But at this point our database has no large objects in it. What's more,
the large-object-referencing columns contain the OIDs of the old large
objects, which will not be the OIDs they'll have when reloaded. Never
fear: pg_dumplo will fix the large object references at the same time
it reloads the large objects. We reload the LO data from the dump
directory like this:
SnowWhite:$ pg_dumplo -s /tmp/dump -d test -i
19338 lo id test/lo/id/19338
SnowWhite:$
And this is everything.
And this is everything. The contents of table lo will be automatically
updated to refer to the new large object OIDs.
Summary: In this small example we have shown that pg_dumplo can be used to
completely dump the database's Large Objects very easily.
For more information see the help ( pg_dumplo -h ).
contrib/pg_dumplo/lo_export.c
View file @ c2d1b391
/* -------------------------------------------------------------------------
* pg_dumplo
*
* Portions Copyright (c) 1999-2000, PostgreSQL, Inc
*
* $Header: /cvsroot/pgsql/contrib/pg_dumplo/Attic/lo_export.c,v 1.4 2000/11/22 00:00:55 tgl Exp $
*
* Karel Zak 1999-2000
* -------------------------------------------------------------------------
*/
#include <stdio.h>
#include <unistd.h>
......@@ -16,15 +26,6 @@
extern int errno;
#define LOAD_LOLIST_QUERY "\
SELECT c.relname, a.attname \
FROM pg_class c, pg_attribute a, pg_type t \
WHERE a.attnum > 0 \
AND a.attrelid = c.oid \
AND a.atttypid = t.oid \
AND t.typname = 'oid' \
AND c.relname NOT LIKE 'pg_%'"
void
load_lolist( LODumpMaster *pgLO )
......@@ -34,19 +35,33 @@ load_lolist( LODumpMaster *pgLO )
int n;
/* ----------
* Now find any candidate tables who have columns of type oid (the
* column oid is ignored, as it has attnum < 1)
* Now find any candidate tables who have columns of type oid.
*
* NOTE: System tables including pg_largeobject will be ignored.
* Otherwise we'd end up dumping all LOs, referenced or not.
*
* NOTE: the system oid column is ignored, as it has attnum < 1.
* This shouldn't matter for correctness, but it saves time.
* ----------
*/
if (!(pgLO->res = PQexec(pgLO->conn, LOAD_LOLIST_QUERY))) {
fprintf(stderr, "%s: Select from pg_class failed.\n", progname);
exit(RE_ERROR);
}
pgLO->res = PQexec(pgLO->conn,
"SELECT c.relname, a.attname "
"FROM pg_class c, pg_attribute a, pg_type t "
"WHERE a.attnum > 0 "
" AND a.attrelid = c.oid "
" AND a.atttypid = t.oid "
" AND t.typname = 'oid' "
" AND c.relkind = 'r' "
" AND c.relname NOT LIKE 'pg_%'");
if (PQresultStatus(pgLO->res) != PGRES_TUPLES_OK) {
fprintf(stderr, "%s: Failed to get LO OIDs:\n%s", progname,
PQerrorMessage(pgLO->conn));
exit(RE_ERROR);
}
if ((n = PQntuples(pgLO->res)) == 0) {
fprintf(stderr, "%s: No large objects in the database.\n", progname);
fprintf(stderr, "%s: No OID columns in the database.\n", progname);
exit(RE_ERROR);
}
......@@ -61,10 +76,9 @@ load_lolist( LODumpMaster *pgLO )
ll->lo_table = strdup(PQgetvalue(pgLO->res, i, 0));
ll->lo_attr = strdup(PQgetvalue(pgLO->res, i, 1));
}
ll->lo_table = ll->lo_attr = (char *) NULL;
PQclear(pgLO->res);
ll++;
ll->lo_table = ll->lo_attr = (char *) NULL;
}
void
......@@ -91,24 +105,25 @@ pglo_export(LODumpMaster *pgLO)
for(ll=pgLO->lolist; ll->lo_table != NULL; ll++) {
/* ----------
* Query
* Query: find the LOs referenced by this column
* ----------
*/
sprintf(Qbuff, "SELECT DISTINCT x.\"%s\" FROM \"%s\" x, pg_largeobject l WHERE x.\"%s\" = l.loid",
ll->lo_attr, ll->lo_table, ll->lo_attr);
sprintf(Qbuff, "SELECT DISTINCT l.loid FROM \"%s\" x, pg_largeobject l WHERE x.\"%s\" = l.loid",
ll->lo_table, ll->lo_attr);
/* puts(Qbuff); */
pgLO->res = PQexec(pgLO->conn, Qbuff);
if ((tuples = PQntuples(pgLO->res)) == 0) {
if (PQresultStatus(pgLO->res) != PGRES_TUPLES_OK) {
fprintf(stderr, "%s: Failed to get LO OIDs:\n%s", progname,
PQerrorMessage(pgLO->conn));
}
else if ((tuples = PQntuples(pgLO->res)) == 0) {
if (!pgLO->quiet && pgLO->action == ACTION_EXPORT_ATTR)
printf("%s: no large objects in '%s'\n",
progname, ll->lo_table);
continue;
} else if (check_res(pgLO)) {
printf("%s: no large objects in \"%s\".\"%s\"\n",
progname, ll->lo_table, ll->lo_attr);
} else {
int t;
char *val;
......@@ -117,9 +132,10 @@ pglo_export(LODumpMaster *pgLO)
* Create DIR/FILE
* ----------
*/
if (tuples && pgLO->action != ACTION_SHOW) {
if (pgLO->action != ACTION_SHOW) {
sprintf(path, "%s/%s/%s", pgLO->space, pgLO->db, ll->lo_table);
sprintf(path, "%s/%s/%s", pgLO->space, pgLO->db,
ll->lo_table);
if (mkdir(path, DIR_UMASK) == -1) {
if (errno != EEXIST) {
......@@ -127,8 +143,9 @@ pglo_export(LODumpMaster *pgLO)
exit(RE_ERROR);
}
}
sprintf(path, "%s/%s", path, ll->lo_attr);
sprintf(path, "%s/%s/%s/%s", pgLO->space, pgLO->db,
ll->lo_table, ll->lo_attr);
if (mkdir(path, DIR_UMASK) == -1) {
if (errno != EEXIST) {
......@@ -145,19 +162,14 @@ pglo_export(LODumpMaster *pgLO)
pgLO->counter += tuples;
for(t=0; t<tuples; t++) {
Oid lo = (Oid) 0;
Oid lo;
val = PQgetvalue(pgLO->res, t, 0);
if (!val)
continue;
else
lo = (Oid) atol(val);
lo = atooid(val);
if (pgLO->action == ACTION_SHOW) {
printf("%s.%s: %ld\n", ll->lo_table,
ll->lo_attr, (long) lo);
printf("%s.%s: %u\n", ll->lo_table, ll->lo_attr, lo);
continue;
}
......@@ -165,13 +177,15 @@ pglo_export(LODumpMaster *pgLO)
pgLO->db, ll->lo_table, ll->lo_attr, val);
if (lo_export(pgLO->conn, lo, path) < 0)
fprintf(stderr, "%s: %s\n", PQerrorMessage(pgLO->conn), progname);
fprintf(stderr, "%s: lo_export failed:\n%s", progname,
PQerrorMessage(pgLO->conn));
else
fprintf(pgLO->index, "%s\t%s\t%s\t%s/%s/%s/%s\n", val,
ll->lo_table, ll->lo_attr, pgLO->db, ll->lo_table, ll->lo_attr, val);
}
}
}
}
PQclear(pgLO->res);
}
}
contrib/pg_dumplo/lo_import.c
View file @ c2d1b391
/* -------------------------------------------------------------------------
* pg_dumplo
*
* Portions Copyright (c) 1999-2000, PostgreSQL, Inc
*
* $Header: /cvsroot/pgsql/contrib/pg_dumplo/Attic/lo_import.c,v 1.2 2000/11/22 00:00:55 tgl Exp $
*
* Karel Zak 1999-2000
* -------------------------------------------------------------------------
*/
#include <stdio.h>
#include <unistd.h>
......@@ -20,7 +30,7 @@ void
pglo_import(LODumpMaster *pgLO)
{
LOlist loa;
long new_oid;
Oid new_oid;
char tab[MAX_TABLE_NAME], attr[MAX_ATTR_NAME],
path[BUFSIZ], lo_path[BUFSIZ],
Qbuff[QUERY_BUFSIZ];
......@@ -33,7 +43,7 @@ pglo_import(LODumpMaster *pgLO)
if (! pgLO->remove && ! pgLO->quiet)
printf(Qbuff);
sscanf(Qbuff, "%ld\t%s\t%s\t%s\n", &loa.lo_oid, tab, attr, path);
sscanf(Qbuff, "%u\t%s\t%s\t%s\n", &loa.lo_oid, tab, attr, path);
loa.lo_table = tab;
loa.lo_attr = attr;
......@@ -43,7 +53,7 @@ pglo_import(LODumpMaster *pgLO)
* Import LO
* ----------
*/
if ((new_oid = lo_import(pgLO->conn, lo_path)) <= 0) {
if ((new_oid = lo_import(pgLO->conn, lo_path)) == 0) {
fprintf(stderr, "%s: %s\n", progname, PQerrorMessage(pgLO->conn));
......@@ -54,12 +64,12 @@ pglo_import(LODumpMaster *pgLO)
if (pgLO->remove) {
notice(pgLO, FALSE);
if (lo_unlink(pgLO->conn, (Oid) loa.lo_oid) < 0)
fprintf(stderr, "%s: can't remove LO: %ld (%s)\n",
if (lo_unlink(pgLO->conn, loa.lo_oid) < 0)
fprintf(stderr, "%s: can't remove LO %u:\n%s",
progname, loa.lo_oid, PQerrorMessage(pgLO->conn));
else if (!pgLO->quiet)
printf("remove old %ld and create new %ld\n",
printf("remove old %u and create new %u\n",
loa.lo_oid, new_oid);
notice(pgLO, TRUE);
}
......@@ -70,20 +80,20 @@ pglo_import(LODumpMaster *pgLO)
* UPDATE oid in tab
* ----------
*/
sprintf(Qbuff, "UPDATE %s SET %s=%ld WHERE %s=%ld",
sprintf(Qbuff, "UPDATE \"%s\" SET \"%s\"=%u WHERE \"%s\"=%u",
loa.lo_table, loa.lo_attr, new_oid, loa.lo_attr, loa.lo_oid);
/*fprintf(stderr, Qbuff);*/
pgLO->res = PQexec(pgLO->conn, Qbuff);
if (!pgLO->res && PQresultStatus(pgLO->res) != PGRES_COMMAND_OK) {
fprintf(stderr, "%s: %s\n",progname, PQerrorMessage(pgLO->conn));
PQclear(pgLO->res);
PQexec(pgLO->conn, "ROLLBACK");
if (PQresultStatus(pgLO->res) != PGRES_COMMAND_OK) {
fprintf(stderr, "%s: %s\n",progname, PQerrorMessage(pgLO->conn));
PQclear(pgLO->res);
PQexec(pgLO->conn, "ROLLBACK");
fprintf(stderr, "\n%s: ROLLBACK\n", progname);
exit(RE_ERROR);
}
}
PQclear(pgLO->res);
}
}
contrib/pg_dumplo/main.c
View file @ c2d1b391
/* -------------------------------------------------------------------------
* pg_dumplo
*
* Portions Copyright (c) 1999-2000, PostgreSQL, Inc
*
* $Header: /cvsroot/pgsql/contrib/pg_dumplo/Attic/main.c,v 1.3 2000/07/03 16:03:22 momjian Exp $
* $Header: /cvsroot/pgsql/contrib/pg_dumplo/Attic/main.c,v 1.4 2000/11/22 00:00:55 tgl Exp $
*
* Karel Zak 1999-2000
* -------------------------------------------------------------------------
*/
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
/* We import postgres.h mostly to get the HAVE_GETOPT_LONG configure result. */
#ifndef OUT_OF_PG
#include "postgres.h"
#include "postgres.h"
#endif
#include <libpq-fe.h>
#include <libpq/libpq-fs.h>
#include "pg_dumplo.h"
#ifdef HAVE_GETOPT_LONG
#include <getopt.h>
#define no_argument 0
......@@ -34,8 +35,6 @@ extern int errno;
char *progname = NULL;
#include "pg_dumplo.h"
int main(int argc, char **argv);
static void usage(void);
static void parse_lolist (LODumpMaster *pgLO);
......@@ -97,11 +96,11 @@ main(int argc, char **argv)
#else
while((arg = getopt(argc, argv, "?aehu:p:qd:l:t:irs:w")) != -1) {
#endif
switch(arg) {
switch(arg) {
case '?':
case 'h':
usage();
exit(RE_OK);
exit(RE_OK);
case 'u':
pgLO->user = strdup(optarg);
break;
......@@ -127,11 +126,11 @@ main(int argc, char **argv)
break;
case 'e':
case 'a':
pgLO->action = ACTION_EXPORT_ALL;
break;
case 'w':
pgLO->action = ACTION_SHOW;
break;
pgLO->action = ACTION_EXPORT_ALL;
break;
case 'w':
pgLO->action = ACTION_SHOW;
break;
case 'r':
pgLO->remove = TRUE;
break;
......@@ -139,10 +138,10 @@ main(int argc, char **argv)
pgLO->quiet = TRUE;
break;
default:
fprintf(stderr, "%s: bad arg!\n", progname);
fprintf(stderr, "%s: bad arg -%c\n", progname, arg);
usage();
exit(RE_ERROR);
}
}
}
} else {
usage();
......@@ -172,12 +171,12 @@ main(int argc, char **argv)
pgLO->conn = PQsetdbLogin(pgLO->host, NULL, NULL, NULL, pgLO->db,
pgLO->user, pwd);
if (PQstatus(pgLO->conn) == CONNECTION_BAD) {
fprintf(stderr, "%s (connection): %s\n", progname, PQerrorMessage(pgLO->conn));
exit(RE_ERROR);
}
pgLO->host = PQhost(pgLO->conn) ? PQhost(pgLO->conn) : "localhost";
pgLO->db = PQdb(pgLO->conn);
if (PQstatus(pgLO->conn) == CONNECTION_BAD) {
fprintf(stderr, "%s (connection): %s\n", progname, PQerrorMessage(pgLO->conn));
exit(RE_ERROR);
}
pgLO->host = PQhost(pgLO->conn) ? PQhost(pgLO->conn) : "localhost";
pgLO->db = PQdb(pgLO->conn);
pgLO->user = PQuser(pgLO->conn);
......@@ -195,6 +194,7 @@ main(int argc, char **argv)
case ACTION_SHOW:
case ACTION_EXPORT_ALL:
load_lolist(pgLO);
/* FALL THROUGH */
case ACTION_EXPORT_ATTR:
pglo_export(pgLO);
......
contrib/pg_dumplo/pg_dumplo.h
View file @ c2d1b391
/* -------------------------------------------------------------------------
* pg_dumplo
*
* Portions Copyright (c) 1999-2000, PostgreSQL, Inc
*
* $Header: /cvsroot/pgsql/contrib/pg_dumplo/Attic/pg_dumplo.h,v 1.2 2000/11/22 00:00:55 tgl Exp $
*
* Karel Zak 1999-2000
* -------------------------------------------------------------------------
*/
#ifndef PG_DUMPLO_H
#define PG_DUMPLO_H
#ifndef _PG_LODUMP_H_
#define _PG_LODUMP_H_
#include "postgres_ext.h"
#define VERSION "0.0.5"
#define VERSION "7.1.0"
/* ----------
* Define
......@@ -10,7 +22,7 @@
*/
#define QUERY_BUFSIZ (8*1024)
#define DIR_UMASK 0755
#define FILE_UMASK 0666
#define FILE_UMASK 0644
#define TRUE 1
#define FALSE 0
......@@ -20,16 +32,16 @@
#define MAX_TABLE_NAME 128
#define MAX_ATTR_NAME 128
extern char *progname;
#define atooid(x) ((Oid) strtoul((x), NULL, 10))
/* ----------
* LO struct
* ----------
*/
typedef struct {
char *lo_table,
char *lo_table,
*lo_attr;
long lo_oid;
Oid lo_oid;
} LOlist;
typedef struct {
......@@ -58,11 +70,12 @@ typedef enum {
ACTION_IMPORT
} PGLODUMP_ACTIONS;
extern char *progname;
extern void notice (LODumpMaster *pgLO, int set);
extern int check_res (LODumpMaster *pgLO);
extern void index_file (LODumpMaster *pgLO);
extern void load_lolist (LODumpMaster *pgLO);
extern void pglo_export (LODumpMaster *pgLO);
extern void pglo_import (LODumpMaster *pgLO);
#endif /* _PG_LODUMP_H */
#endif /* PG_DUMPLO_H */
contrib/pg_dumplo/utils.c
View file @ c2d1b391
/* -------------------------------------------------------------------------
* pg_dumplo
*
* Portions Copyright (c) 1999-2000, PostgreSQL, Inc
*
* $Header: /cvsroot/pgsql/contrib/pg_dumplo/Attic/utils.c,v 1.2 2000/11/22 00:00:55 tgl Exp $
*
* Karel Zak 1999-2000
* -------------------------------------------------------------------------
*/
#include <stdio.h>
#include <unistd.h>
......@@ -58,22 +68,6 @@ index_file(LODumpMaster *pgLO)
}
}
int
check_res(LODumpMaster *pgLO)
{
if (!pgLO->res && PQresultStatus(pgLO->res) != PGRES_COMMAND_OK) {
fprintf(stderr, "%s: %s\n", progname, PQerrorMessage(pgLO->conn));
PQclear(pgLO->res);
return FALSE;
}
if (PQresultStatus(pgLO->res) != PGRES_TUPLES_OK) {
fprintf(stderr, "%s: Tuples is not OK.\n", progname);
PQclear(pgLO->res);
return FALSE;
}
return TRUE;
}
static
void Dummy_NoticeProcessor(void * arg, const char * message)
{
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment