Skip to content

P
Postgres FD Implementation
Commit 874148fe authored by Bruce Momjian's avatar Bruce Momjian
Browse files
Options

IMPROVED VERSION APPLIED: 

The attached patch completes the following TODO item:

    * Generate failure on short COPY lines rather than pad NULLs

I also restructed a lot of the existing COPY code, did some code
review on the column list patch sent in by Brent Verner a little
while ago, and added some regression tests. I also added an
explicit check (and resultant error) for extra data before
the end-of-line.

Neil Conway
parent 1dedbf2d
Show whitespace changes
Inline Side-by-side
Showing with 319 additions and 210 deletions
doc/src/sgml/ref/copy.sgml
View file @ 874148fe
<!-- <!--
$Header: /cvsroot/pgsql/doc/src/sgml/ref/copy.sgml,v 1.33 2002/07/18 04:43:50 momjian Exp $ $Header: /cvsroot/pgsql/doc/src/sgml/ref/copy.sgml,v 1.34 2002/07/30 16:55:05 momjian Exp $
PostgreSQL documentation PostgreSQL documentation
--> -->
...@@ -200,11 +200,11 @@ ERROR: <replaceable>reason</replaceable> ...@@ -200,11 +200,11 @@ ERROR: <replaceable>reason</replaceable>
</para> </para>
<para> <para>
When using the optional column list syntax, <command>COPY TO</command> If a list of columns is specified, <command>COPY</command> will
and <command>COPY FROM</command> will only copy the specified only copy the data in the specified columns to or from the table.
columns' values to/from the table. If a column in the table If there are any columns in the table that are not in the table,
is not in the column list, <command>COPY FROM</command> will insert <command>COPY FROM</command> will insert the default value for
default values for that column if a default value is defined. that column.
</para> </para>
<para> <para>
...@@ -266,8 +266,8 @@ ERROR: <replaceable>reason</replaceable> ...@@ -266,8 +266,8 @@ ERROR: <replaceable>reason</replaceable>
</para> </para>
<para> <para>
<command>COPY FROM</command> neither invokes rules nor acts on column <command>COPY FROM</command> will invoke any triggers or check
defaults. It does invoke triggers and check constraints. constraints. However, it will not invoke rules.
</para> </para>
<para> <para>
...@@ -330,7 +330,12 @@ ERROR: <replaceable>reason</replaceable> ...@@ -330,7 +330,12 @@ ERROR: <replaceable>reason</replaceable>
The attribute values themselves are strings generated by the The attribute values themselves are strings generated by the
output function, or acceptable to the input function, of each output function, or acceptable to the input function, of each
attribute's data type. The specified null-value string is used in attribute's data type. The specified null-value string is used in
place of attributes that are NULL. place of attributes that are NULL. When using <command>COPY
FROM</command> without a column list, each row of the input file
must contain data for each attribute in the table: no missing data
is allowed. Similarly, <command>COPY FROM</command> will raise
an error if it encounters any data in the input file that would
not be inserted into the table: extra data is not allowed.
</para> </para>
<para> <para>
If OIDS is specified, the OID is read or written as the first column, If OIDS is specified, the OID is read or written as the first column,
......
src/backend/commands/copy.c
View file @ 874148fe
...@@ -7,7 +7,7 @@ ...@@ -7,7 +7,7 @@
* *
* *
* IDENTIFICATION * IDENTIFICATION
* $Header: /cvsroot/pgsql/src/backend/commands/copy.c,v 1.160 2002/07/20 05:16:57 momjian Exp $ * $Header: /cvsroot/pgsql/src/backend/commands/copy.c,v 1.161 2002/07/30 16:55:06 momjian Exp $
* *
*------------------------------------------------------------------------- *-------------------------------------------------------------------------
*/ */
...@@ -45,16 +45,21 @@ ...@@ -45,16 +45,21 @@
#define ISOCTAL(c) (((c) >= '0') && ((c) <= '7')) #define ISOCTAL(c) (((c) >= '0') && ((c) <= '7'))
#define OCTVALUE(c) ((c) - '0') #define OCTVALUE(c) ((c) - '0')
typedef enum CopyReadResult
{
NORMAL_ATTR,
END_OF_LINE,
END_OF_FILE
} CopyReadResult;
/* non-export function prototypes */ /* non-export function prototypes */
static void CopyTo(Relation rel, List *attlist, bool binary, bool oids, FILE *fp, char *delim, char *null_print); static void CopyTo(Relation rel, List *attlist, bool binary, bool oids, FILE *fp, char *delim, char *null_print);
static void CopyFrom(Relation rel, List *attlist, bool binary, bool oids, FILE *fp, char *delim, char *null_print); static void CopyFrom(Relation rel, List *attlist, bool binary, bool oids, FILE *fp, char *delim, char *null_print);
static Oid GetInputFunction(Oid type); static Oid GetInputFunction(Oid type);
static Oid GetTypeElement(Oid type); static Oid GetTypeElement(Oid type);
static void CopyReadNewline(FILE *fp, int *newline); static char *CopyReadAttribute(FILE *fp, const char *delim, CopyReadResult *result);
static char *CopyReadAttribute(FILE *fp, bool *isnull, char *delim, int *newline, char *null_print);
static void CopyAttributeOut(FILE *fp, char *string, char *delim); static void CopyAttributeOut(FILE *fp, char *string, char *delim);
static void CopyAssertAttlist(Relation rel, List* attlist, bool from); static void CopyCheckAttlist(Relation rel, List *attlist);
static const char BinarySignature[12] = "PGBCOPY\n\377\r\n\0"; static const char BinarySignature[12] = "PGBCOPY\n\377\r\n\0";
...@@ -271,14 +276,10 @@ DoCopy(const CopyStmt *stmt) ...@@ -271,14 +276,10 @@ DoCopy(const CopyStmt *stmt)
bool pipe = (stmt->filename == NULL); bool pipe = (stmt->filename == NULL);
List *option; List *option;
List *attlist = stmt->attlist; List *attlist = stmt->attlist;
DefElem *dbinary = NULL;
DefElem *doids = NULL;
DefElem *ddelim = NULL;
DefElem *dnull = NULL;
bool binary = false; bool binary = false;
bool oids = false; bool oids = false;
char *delim = "\t"; char *delim = NULL;
char *null_print = "\\N"; char *null_print = NULL;
FILE *fp; FILE *fp;
Relation rel; Relation rel;
AclMode required_access = (is_from ? ACL_INSERT : ACL_SELECT); AclMode required_access = (is_from ? ACL_INSERT : ACL_SELECT);
...@@ -289,52 +290,54 @@ DoCopy(const CopyStmt *stmt) ...@@ -289,52 +290,54 @@ DoCopy(const CopyStmt *stmt)
{ {
DefElem *defel = (DefElem *) lfirst(option); DefElem *defel = (DefElem *) lfirst(option);
/* XXX: Should we bother checking for doubled options? */
if (strcmp(defel->defname, "binary") == 0) if (strcmp(defel->defname, "binary") == 0)
{ {
if (dbinary) if (binary)
/* should this really be an error? */
elog(ERROR, "COPY: BINARY option appears more than once"); elog(ERROR, "COPY: BINARY option appears more than once");
dbinary = defel;
binary = intVal(defel->arg);
} }
else if (strcmp(defel->defname, "oids") == 0) else if (strcmp(defel->defname, "oids") == 0)
{ {
if (doids) if (oids)
/* should this really be an error? */
elog(ERROR, "COPY: OIDS option appears more than once"); elog(ERROR, "COPY: OIDS option appears more than once");
doids = defel;
oids = intVal(defel->arg);
} }
else if (strcmp(defel->defname, "delimiter") == 0) else if (strcmp(defel->defname, "delimiter") == 0)
{ {
if (ddelim) if (delim)
elog(ERROR, "COPY: DELIMITER string may only be defined once in query"); elog(ERROR, "COPY: DELIMITER string may only be defined once in query");
ddelim = defel;
delim = strVal(defel->arg);
} }
else if (strcmp(defel->defname, "null") == 0) else if (strcmp(defel->defname, "null") == 0)
{ {
if (dnull) if (null_print)
elog(ERROR, "COPY: NULL representation may only be defined once in query"); elog(ERROR, "COPY: NULL representation may only be defined once in query");
dnull = defel;
null_print = strVal(defel->arg);
} }
else else
elog(ERROR, "COPY: option \"%s\" not recognized", elog(ERROR, "COPY: option \"%s\" not recognized",
defel->defname); defel->defname);
} }
if (dbinary) if (binary && delim)
binary = intVal(dbinary->arg);
if (doids)
oids = intVal(doids->arg);
if (ddelim)
delim = strVal(ddelim->arg);
if (dnull)
null_print = strVal(dnull->arg);
if (binary && ddelim)
elog(ERROR, "You can not specify the DELIMITER in BINARY mode."); elog(ERROR, "You can not specify the DELIMITER in BINARY mode.");
if (binary && dnull) if (binary && null_print)
elog(ERROR, "You can not specify NULL in BINARY mode."); elog(ERROR, "You can not specify NULL in BINARY mode.");
/* Set defaults */
if (!delim)
delim = "\t";
if (!null_print)
null_print = "\\N";
/* /*
* Open and lock the relation, using the appropriate lock type. * Open and lock the relation, using the appropriate lock type.
*/ */
...@@ -363,6 +366,13 @@ DoCopy(const CopyStmt *stmt) ...@@ -363,6 +366,13 @@ DoCopy(const CopyStmt *stmt)
if (strlen(delim) != 1) if (strlen(delim) != 1)
elog(ERROR, "COPY delimiter must be a single character"); elog(ERROR, "COPY delimiter must be a single character");
/*
* Don't allow COPY w/ OIDs to or from a table without them
*/
if (oids && !rel->rd_rel->relhasoids)
elog(ERROR, "COPY: table \"%s\" does not have OIDs",
RelationGetRelationName(rel));
/* /*
* Set up variables to avoid per-attribute overhead. * Set up variables to avoid per-attribute overhead.
*/ */
...@@ -372,22 +382,24 @@ DoCopy(const CopyStmt *stmt) ...@@ -372,22 +382,24 @@ DoCopy(const CopyStmt *stmt)
server_encoding = GetDatabaseEncoding(); server_encoding = GetDatabaseEncoding();
#endif #endif
if( attlist == NIL ){ if (attlist == NIL)
{
/* get list of attributes in the relation */ /* get list of attributes in the relation */
TupleDesc desc = RelationGetDescr(rel); TupleDesc desc = RelationGetDescr(rel);
int i; int i;
for(i = 0; i < desc->natts; ++i){ for (i = 0; i < desc->natts; ++i)
Ident* id = makeNode(Ident); {
Ident *id = makeNode(Ident);
id->name = NameStr(desc->attrs[i]->attname); id->name = NameStr(desc->attrs[i]->attname);
attlist = lappend(attlist,id); attlist = lappend(attlist,id);
} }
} }
else{ else
if( binary ){ {
if (binary)
elog(ERROR,"COPY: BINARY format cannot be used with specific column list"); elog(ERROR,"COPY: BINARY format cannot be used with specific column list");
}
/* verify that any user-specified attributes exist in the relation */ CopyCheckAttlist(rel, attlist);
CopyAssertAttlist(rel,attlist,is_from);
} }
if (is_from) if (is_from)
...@@ -533,29 +545,30 @@ CopyTo(Relation rel, List *attlist, bool binary, bool oids, ...@@ -533,29 +545,30 @@ CopyTo(Relation rel, List *attlist, bool binary, bool oids,
char *string; char *string;
Snapshot mySnapshot; Snapshot mySnapshot;
int copy_attr_count; int copy_attr_count;
int* attmap; int *attmap;
int p = 0; int p = 0;
List* cur; List *cur;
if (oids && !rel->rd_rel->relhasoids)
elog(ERROR, "COPY: table %s does not have OIDs",
RelationGetRelationName(rel));
tupDesc = rel->rd_att; tupDesc = rel->rd_att;
attr_count = rel->rd_att->natts; attr_count = rel->rd_att->natts;
attr = rel->rd_att->attrs; attr = rel->rd_att->attrs;
copy_attr_count = length(attlist); copy_attr_count = length(attlist);
attmap = (int *) palloc(copy_attr_count * sizeof(int));
foreach(cur, attlist)
{
const char *currAtt = strVal(lfirst(cur));
for (i = 0; i < attr_count; i++)
{
if (namestrcmp(&attr[i]->attname, currAtt) == 0)
{ {
attmap = (int*)palloc(copy_attr_count * sizeof(int));
foreach(cur,attlist){
for (i = 0; i < attr_count; i++){
if( strcmp(strVal(lfirst(cur)),NameStr(attr[i]->attname)) == 0){
attmap[p++] = i; attmap[p++] = i;
continue; continue;
} }
} }
} }
}
/* /*
* For binary copy we really only need isvarlena, but compute it * For binary copy we really only need isvarlena, but compute it
...@@ -767,19 +780,15 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids, ...@@ -767,19 +780,15 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids,
Oid in_func_oid; Oid in_func_oid;
Datum *values; Datum *values;
char *nulls; char *nulls;
bool isnull;
int done = 0; int done = 0;
char *string;
ResultRelInfo *resultRelInfo; ResultRelInfo *resultRelInfo;
EState *estate = CreateExecutorState(); /* for ExecConstraints() */ EState *estate = CreateExecutorState(); /* for ExecConstraints() */
TupleTable tupleTable; TupleTable tupleTable;
TupleTableSlot *slot; TupleTableSlot *slot;
Oid loaded_oid = InvalidOid;
bool skip_tuple = false;
bool file_has_oids; bool file_has_oids;
int* attmap = NULL; int *attmap = NULL;
int* defmap = NULL; int *defmap = NULL;
Node** defexprs = NULL; /* array of default att expressions */ Node **defexprs = NULL; /* array of default att expressions */
ExprContext *econtext; /* used for ExecEvalExpr for default atts */ ExprContext *econtext; /* used for ExecEvalExpr for default atts */
ExprDoneCond isdone; ExprDoneCond isdone;
...@@ -810,40 +819,47 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids, ...@@ -810,40 +819,47 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids,
slot = ExecAllocTableSlot(tupleTable); slot = ExecAllocTableSlot(tupleTable);
ExecSetSlotDescriptor(slot, tupDesc, false); ExecSetSlotDescriptor(slot, tupDesc, false);
if (!binary) if (!binary)
{ {
/* /*
* pick up the input function and default expression (if any) for * pick up the input function and default expression (if any) for
* each attribute in the relation. * each attribute in the relation.
*/ */
List* cur; attmap = (int *) palloc(sizeof(int) * attr_count);
attmap = (int*)palloc(sizeof(int) * attr_count); defmap = (int *) palloc(sizeof(int) * attr_count);
defmap = (int*)palloc(sizeof(int) * attr_count); defexprs = (Node **) palloc(sizeof(Node *) * attr_count);
defexprs = (Node**)palloc(sizeof(Node*) * attr_count);
in_functions = (FmgrInfo *) palloc(attr_count * sizeof(FmgrInfo)); in_functions = (FmgrInfo *) palloc(attr_count * sizeof(FmgrInfo));
elements = (Oid *) palloc(attr_count * sizeof(Oid)); elements = (Oid *) palloc(attr_count * sizeof(Oid));
for (i = 0; i < attr_count; i++) for (i = 0; i < attr_count; i++)
{ {
List *l;
int p = 0; int p = 0;
bool specified = false; bool specified = false;
in_func_oid = (Oid) GetInputFunction(attr[i]->atttypid); in_func_oid = (Oid) GetInputFunction(attr[i]->atttypid);
fmgr_info(in_func_oid, &in_functions[i]); fmgr_info(in_func_oid, &in_functions[i]);
elements[i] = GetTypeElement(attr[i]->atttypid); elements[i] = GetTypeElement(attr[i]->atttypid);
foreach(cur,attlist){
if( strcmp(strVal(lfirst(cur)),NameStr(attr[i]->attname)) == 0){ foreach(l, attlist)
{
if (namestrcmp(&attr[i]->attname, strVal(lfirst(l))) == 0)
{
attmap[p] = i; attmap[p] = i;
specified = true; specified = true;
continue; continue;
} }
++p; p++;
} }
if( ! specified ){
/* column not specified, try to get a default */ /* if column not specified, use default value if one exists */
defexprs[def_attr_count] = build_column_default(rel,i+1); if (! specified)
if( defexprs[def_attr_count] != NULL ){ {
defexprs[def_attr_count] = build_column_default(rel, i + 1);
if (defexprs[def_attr_count] != NULL)
{
defmap[def_attr_count] = i; defmap[def_attr_count] = i;
++def_attr_count; def_attr_count++;
} }
} }
} }
...@@ -857,13 +873,11 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids, ...@@ -857,13 +873,11 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids,
/* Signature */ /* Signature */
CopyGetData(readSig, 12, fp); CopyGetData(readSig, 12, fp);
if (CopyGetEof(fp) || if (CopyGetEof(fp) || memcmp(readSig, BinarySignature, 12) != 0)
memcmp(readSig, BinarySignature, 12) != 0)
elog(ERROR, "COPY BINARY: file signature not recognized"); elog(ERROR, "COPY BINARY: file signature not recognized");
/* Integer layout field */ /* Integer layout field */
CopyGetData(&tmp, sizeof(int32), fp); CopyGetData(&tmp, sizeof(int32), fp);
if (CopyGetEof(fp) || if (CopyGetEof(fp) || tmp != 0x01020304)
tmp != 0x01020304)
elog(ERROR, "COPY BINARY: incompatible integer layout"); elog(ERROR, "COPY BINARY: incompatible integer layout");
/* Flags field */ /* Flags field */
CopyGetData(&tmp, sizeof(int32), fp); CopyGetData(&tmp, sizeof(int32), fp);
...@@ -875,8 +889,7 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids, ...@@ -875,8 +889,7 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids,
elog(ERROR, "COPY BINARY: unrecognized critical flags in header"); elog(ERROR, "COPY BINARY: unrecognized critical flags in header");
/* Header extension length */ /* Header extension length */
CopyGetData(&tmp, sizeof(int32), fp); CopyGetData(&tmp, sizeof(int32), fp);
if (CopyGetEof(fp) || if (CopyGetEof(fp) || tmp < 0)
tmp < 0)
elog(ERROR, "COPY BINARY: bogus file header (missing length)"); elog(ERROR, "COPY BINARY: bogus file header (missing length)");
/* Skip extension header, if present */ /* Skip extension header, if present */
while (tmp-- > 0) while (tmp-- > 0)
...@@ -890,10 +903,6 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids, ...@@ -890,10 +903,6 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids,
elements = NULL; elements = NULL;
} }
/* Silently drop incoming OIDs if table does not have OIDs */
if (!rel->rd_rel->relhasoids)
oids = false;
values = (Datum *) palloc(attr_count * sizeof(Datum)); values = (Datum *) palloc(attr_count * sizeof(Datum));
nulls = (char *) palloc(attr_count * sizeof(char)); nulls = (char *) palloc(attr_count * sizeof(char));
...@@ -904,6 +913,9 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids, ...@@ -904,6 +913,9 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids,
while (!done) while (!done)
{ {
bool skip_tuple;
Oid loaded_oid;
CHECK_FOR_INTERRUPTS(); CHECK_FOR_INTERRUPTS();
copy_lineno++; copy_lineno++;
...@@ -917,16 +929,17 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids, ...@@ -917,16 +929,17 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids,
if (!binary) if (!binary)
{ {
int newline = 0; CopyReadResult result;
char *string;
if (file_has_oids) if (file_has_oids)
{ {
string = CopyReadAttribute(fp, &isnull, delim, string = CopyReadAttribute(fp, delim, &result);
&newline, null_print);
if (isnull) if (result == END_OF_FILE)
done = 1;
else if (string == NULL || strcmp(string, null_print) == 0)
elog(ERROR, "COPY TEXT: NULL Oid"); elog(ERROR, "COPY TEXT: NULL Oid");
else if (string == NULL)
done = 1; /* end of file */
else else
{ {
loaded_oid = DatumGetObjectId(DirectFunctionCall1(oidin, loaded_oid = DatumGetObjectId(DirectFunctionCall1(oidin,
...@@ -943,14 +956,19 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids, ...@@ -943,14 +956,19 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids,
for (i = 0; i < copy_attr_count && !done; i++) for (i = 0; i < copy_attr_count && !done; i++)
{ {
int m = attmap[i]; int m = attmap[i];
string = CopyReadAttribute(fp, &isnull, delim,
&newline, null_print);
if( isnull ){ string = CopyReadAttribute(fp, delim, &result);
/* nothing */
/* If we got an end-of-line before we expected, bail out */
if (result == END_OF_LINE && i < (copy_attr_count - 1))
elog(ERROR, "COPY TEXT: Missing data for attribute %d", i + 1);
if (result == END_OF_FILE)
done = 1;
else if (strcmp(string, null_print) == 0)
{
/* we read an SQL NULL, no need to do anything */
} }
else if (string == NULL)
done = 1; /* end of file */
else else
{ {
values[m] = FunctionCall3(&in_functions[m], values[m] = FunctionCall3(&in_functions[m],
...@@ -961,18 +979,22 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids, ...@@ -961,18 +979,22 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids,
} }
} }
if (result == NORMAL_ATTR && !done)
elog(ERROR, "COPY TEXT: Extra data encountered");
/* /*
* as above, we only try a default lookup if one is * as above, we only try a default lookup if one is
* known to be available * known to be available
*/ */
for (i = 0; i < def_attr_count && !done; i++){ for (i = 0; i < def_attr_count && !done; i++)
{
bool isnull; bool isnull;
values[defmap[i]] = ExecEvalExpr(defexprs[i],econtext,&isnull,&isdone); values[defmap[i]] = ExecEvalExpr(defexprs[i], econtext,
if( ! isnull ) &isnull, &isdone);
if (! isnull)
nulls[defmap[i]] = ' '; nulls[defmap[i]] = ' ';
} }
if (!done)
CopyReadNewline(fp, &newline);
} }
else else
{ /* binary */ { /* binary */
...@@ -980,8 +1002,7 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids, ...@@ -980,8 +1002,7 @@ CopyFrom(Relation rel, List *attlist, bool binary, bool oids,
fld_size; fld_size;
CopyGetData(&fld_count, sizeof(int16), fp); CopyGetData(&fld_count, sizeof(int16), fp);
if (CopyGetEof(fp) || if (CopyGetEof(fp) || fld_count == -1)
fld_count == -1)
done = 1; done = 1;
else else
{ {
...@@ -1173,36 +1194,25 @@ GetTypeElement(Oid type) ...@@ -1173,36 +1194,25 @@ GetTypeElement(Oid type)
return result; return result;
} }
/*
* Reads input from fp until an end of line is seen.
*/
static void
CopyReadNewline(FILE *fp, int *newline)
{
if (!*newline)
{
elog(WARNING, "CopyReadNewline: extra fields ignored");
while (!CopyGetEof(fp) && (CopyGetChar(fp) != '\n'));
}
*newline = 0;
}
/* /*
* Read the value of a single attribute. * Read the value of a single attribute.
* *
* Result is either a string, or NULL (if EOF or a null attribute). * Results are returned in the status indicator, as well as the
* Note that the caller should not pfree the string! * return value. If a value was successfully read but there is
* more to read before EOL, NORMAL_ATTR is set and the value read
* is returned. If a value was read and we hit EOL, END_OF_LINE
* is set and the value read is returned. If we hit the EOF,
* END_OF_FILE is set and NULL is returned.
*
* Note: This function does not care about SQL NULL values -- it
* is the caller's responsibility to check if the returned string
* matches what the user specified for the SQL NULL value.
* *
* *isnull is set true if a null attribute, else false. * delim is the column delimiter string.
* delim is the column delimiter string (currently always 1 character).
* *newline remembers whether we've seen a newline ending this tuple.
* null_print says how NULL values are represented
*/ */
static char * static char *
CopyReadAttribute(FILE *fp, bool *isnull, char *delim, int *newline, char *null_print) CopyReadAttribute(FILE *fp, const char *delim, CopyReadResult *result)
{ {
int c; int c;
int delimc = (unsigned char)delim[0]; int delimc = (unsigned char)delim[0];
...@@ -1220,23 +1230,20 @@ CopyReadAttribute(FILE *fp, bool *isnull, char *delim, int *newline, char *null_ ...@@ -1220,23 +1230,20 @@ CopyReadAttribute(FILE *fp, bool *isnull, char *delim, int *newline, char *null_
attribute_buf.len = 0; attribute_buf.len = 0;
attribute_buf.data[0] = '\0'; attribute_buf.data[0] = '\0';
/* if last delimiter was a newline return a NULL attribute */ /* set default */
if (*newline) *result = NORMAL_ATTR;
{
*isnull = (bool) true;
return NULL;
}
*isnull = (bool) false; /* set default */
for (;;) for (;;)
{ {
c = CopyGetChar(fp); c = CopyGetChar(fp);
if (c == EOF) if (c == EOF)
goto endOfFile; {
*result = END_OF_FILE;
return NULL;
}
if (c == '\n') if (c == '\n')
{ {
*newline = 1; *result = END_OF_LINE;
break; break;
} }
if (c == delimc) if (c == delimc)
...@@ -1245,7 +1252,10 @@ CopyReadAttribute(FILE *fp, bool *isnull, char *delim, int *newline, char *null_ ...@@ -1245,7 +1252,10 @@ CopyReadAttribute(FILE *fp, bool *isnull, char *delim, int *newline, char *null_
{ {
c = CopyGetChar(fp); c = CopyGetChar(fp);
if (c == EOF) if (c == EOF)
goto endOfFile; {
*result = END_OF_FILE;
return NULL;
}
switch (c) switch (c)
{ {
case '0': case '0':
...@@ -1274,14 +1284,20 @@ CopyReadAttribute(FILE *fp, bool *isnull, char *delim, int *newline, char *null_ ...@@ -1274,14 +1284,20 @@ CopyReadAttribute(FILE *fp, bool *isnull, char *delim, int *newline, char *null_
else else
{ {
if (c == EOF) if (c == EOF)
goto endOfFile; {
*result = END_OF_FILE;
return NULL;
}
CopyDonePeek(fp, c, false /* put back */ ); CopyDonePeek(fp, c, false /* put back */ );
} }
} }
else else
{ {
if (c == EOF) if (c == EOF)
goto endOfFile; {
*result = END_OF_FILE;
return NULL;
}
CopyDonePeek(fp, c, false /* put back */ ); CopyDonePeek(fp, c, false /* put back */ );
} }
c = val & 0377; c = val & 0377;
...@@ -1319,7 +1335,8 @@ CopyReadAttribute(FILE *fp, bool *isnull, char *delim, int *newline, char *null_ ...@@ -1319,7 +1335,8 @@ CopyReadAttribute(FILE *fp, bool *isnull, char *delim, int *newline, char *null_
c = CopyGetChar(fp); c = CopyGetChar(fp);
if (c != '\n') if (c != '\n')
elog(ERROR, "CopyReadAttribute: end of record marker corrupted"); elog(ERROR, "CopyReadAttribute: end of record marker corrupted");
goto endOfFile; *result = END_OF_FILE;
return NULL;
} }
} }
appendStringInfoCharMacro(&attribute_buf, c); appendStringInfoCharMacro(&attribute_buf, c);
...@@ -1334,7 +1351,10 @@ CopyReadAttribute(FILE *fp, bool *isnull, char *delim, int *newline, char *null_ ...@@ -1334,7 +1351,10 @@ CopyReadAttribute(FILE *fp, bool *isnull, char *delim, int *newline, char *null_
{ {
c = CopyGetChar(fp); c = CopyGetChar(fp);
if (c == EOF) if (c == EOF)
goto endOfFile; {
*result = END_OF_FILE;
return NULL;
}
appendStringInfoCharMacro(&attribute_buf, c); appendStringInfoCharMacro(&attribute_buf, c);
} }
} }
...@@ -1357,13 +1377,7 @@ CopyReadAttribute(FILE *fp, bool *isnull, char *delim, int *newline, char *null_ ...@@ -1357,13 +1377,7 @@ CopyReadAttribute(FILE *fp, bool *isnull, char *delim, int *newline, char *null_
} }
#endif #endif
if (strcmp(attribute_buf.data, null_print) == 0)
*isnull = true;
return attribute_buf.data; return attribute_buf.data;
endOfFile:
return NULL;
} }
static void static void
...@@ -1454,23 +1468,19 @@ CopyAttributeOut(FILE *fp, char *server_string, char *delim) ...@@ -1454,23 +1468,19 @@ CopyAttributeOut(FILE *fp, char *server_string, char *delim)
} }
/* /*
* CopyAssertAttlist: elog(ERROR,...) if the specified attlist * CopyCheckAttlist: elog(ERROR,...) if the specified attlist
* is not valid for the Relation * is not valid for the Relation
*/ */
static void static void
CopyAssertAttlist(Relation rel, List* attlist, bool from) CopyCheckAttlist(Relation rel, List *attlist)
{ {
TupleDesc tupDesc; TupleDesc tupDesc;
List* cur;
char* illegalattname = NULL;
int attr_count; int attr_count;
const char* to_or_from; List *l;
if( attlist == NIL ) if (attlist == NIL)
return; return;
to_or_from = (from == true ? "FROM" : "TO");
tupDesc = RelationGetDescr(rel); tupDesc = RelationGetDescr(rel);
Assert(tupDesc != NULL); Assert(tupDesc != NULL);
...@@ -1478,25 +1488,30 @@ CopyAssertAttlist(Relation rel, List* attlist, bool from) ...@@ -1478,25 +1488,30 @@ CopyAssertAttlist(Relation rel, List* attlist, bool from)
* make sure there aren't more columns specified than are in the table * make sure there aren't more columns specified than are in the table
*/ */
attr_count = tupDesc->natts; attr_count = tupDesc->natts;
if( attr_count < length(attlist) ) if (attr_count < length(attlist))
elog(ERROR,"More columns specified in COPY %s command than in target relation",to_or_from); elog(ERROR, "COPY: Too many columns specified");
/* /*
* make sure no columns are specified that don't exist in the table * make sure no columns are specified that don't exist in the table
*/ */
foreach(cur,attlist) foreach(l, attlist)
{
char *colName = strVal(lfirst(l));
bool found = false;
int i;
for (i = 0; i < attr_count; i++)
{ {
int found = 0; if (namestrcmp(&tupDesc->attrs[i]->attname, colName) == 0)
int i = 0;
for(;i<attr_count;++i)
{ {
if( strcmp(strVal(lfirst(cur)),NameStr(tupDesc->attrs[i]->attname)) == 0) found = true;
++found; break;
}
} }
if( ! found )
illegalattname = strVal(lfirst(cur)); if (!found)
elog(ERROR, "COPY: Specified column \"%s\" does not exist",
colName);
} }
if( illegalattname )
elog(ERROR,"Attribute referenced in COPY %s command does not exist: \"%s.%s\"",to_or_from,RelationGetRelationName(rel),illegalattname);
} }
src/test/regress/expected/copy2.out
View file @ 874148fe
...@@ -23,10 +23,44 @@ CREATE TRIGGER trg_x_after AFTER INSERT ON x ...@@ -23,10 +23,44 @@ CREATE TRIGGER trg_x_after AFTER INSERT ON x
FOR EACH ROW EXECUTE PROCEDURE fn_x_after(); FOR EACH ROW EXECUTE PROCEDURE fn_x_after();
CREATE TRIGGER trg_x_before BEFORE INSERT ON x CREATE TRIGGER trg_x_before BEFORE INSERT ON x
FOR EACH ROW EXECUTE PROCEDURE fn_x_before(); FOR EACH ROW EXECUTE PROCEDURE fn_x_before();
COPY x (a,b,c,d,e) from stdin; COPY x (a, b, c, d, e) from stdin;
COPY x (b,d) from stdin; COPY x (b, d) from stdin;
COPY x (b,d) from stdin; COPY x (b, d) from stdin;
COPY x (a,b,c,d,e) from stdin; COPY x (a, b, c, d, e) from stdin;
-- non-existent column in column list: should fail
COPY x (xyz) from stdin;
ERROR: COPY: Specified column "xyz" does not exist
-- too many columns in column list: should fail
COPY x (a, b, c, d, e, d, c) from stdin;
ERROR: COPY: Too many columns specified
-- missing data: should fail
COPY x from stdin;
ERROR: copy: line 1, COPY TEXT: Missing data for attribute 1
lost synchronization with server, resetting connection
COPY x from stdin;
ERROR: copy: line 1, COPY TEXT: Missing data for attribute 4
lost synchronization with server, resetting connection
COPY x from stdin;
ERROR: copy: line 1, COPY TEXT: Missing data for attribute 4
lost synchronization with server, resetting connection
-- extra data: should fail
COPY x from stdin;
ERROR: copy: line 1, COPY TEXT: Extra data encountered
lost synchronization with server, resetting connection
-- various COPY options: delimiters, oids, NULL string
COPY x (b, c, d, e) from stdin with oids delimiter ',' null 'x';
-- COPY w/ oids on a table w/o oids should fail
CREATE TABLE no_oids (
a int,
b int
) WITHOUT OIDS;
INSERT INTO no_oids (a, b) VALUES (5, 10);
INSERT INTO no_oids (a, b) VALUES (20, 30);
-- should fail
COPY no_oids FROM stdin WITH OIDS;
ERROR: COPY: table "no_oids" does not have OIDs
COPY no_oids TO stdout WITH OIDS;
ERROR: COPY: table "no_oids" does not have OIDs
COPY x TO stdout; COPY x TO stdout;
10000 21 31 41 before trigger fired 10000 21 31 41 before trigger fired
10001 22 32 42 before trigger fired 10001 22 32 42 before trigger fired
...@@ -34,11 +68,25 @@ COPY x TO stdout; ...@@ -34,11 +68,25 @@ COPY x TO stdout;
10003 24 34 44 before trigger fired 10003 24 34 44 before trigger fired
10004 25 35 45 before trigger fired 10004 25 35 45 before trigger fired
10005 26 36 46 before trigger fired 10005 26 36 46 before trigger fired
6 \N 45 80 before trigger fired
1 1 stuff test_1 after trigger fired 1 1 stuff test_1 after trigger fired
2 2 stuff test_2 after trigger fired 2 2 stuff test_2 after trigger fired
3 3 stuff test_3 after trigger fired 3 3 stuff test_3 after trigger fired
4 4 stuff test_4 after trigger fired 4 4 stuff test_4 after trigger fired
5 5 stuff test_5 after trigger fired 5 5 stuff test_5 after trigger fired
COPY x (c, e) TO stdout;
31 before trigger fired
32 before trigger fired
33 before trigger fired
34 before trigger fired
35 before trigger fired
36 before trigger fired
45 before trigger fired
stuff after trigger fired
stuff after trigger fired
stuff after trigger fired
stuff after trigger fired
stuff after trigger fired
DROP TABLE x; DROP TABLE x;
DROP FUNCTION fn_x_before(); DROP FUNCTION fn_x_before();
DROP FUNCTION fn_x_after(); DROP FUNCTION fn_x_after();
src/test/regress/sql/copy2.sql
View file @ 874148fe
...@@ -26,22 +26,22 @@ FOR EACH ROW EXECUTE PROCEDURE fn_x_after(); ...@@ -26,22 +26,22 @@ FOR EACH ROW EXECUTE PROCEDURE fn_x_after();
CREATE TRIGGER trg_x_before BEFORE INSERT ON x CREATE TRIGGER trg_x_before BEFORE INSERT ON x
FOR EACH ROW EXECUTE PROCEDURE fn_x_before(); FOR EACH ROW EXECUTE PROCEDURE fn_x_before();
COPY x (a,b,c,d,e) from stdin; COPY x (a, b, c, d, e) from stdin;
10000 21 31 41 51 10000 21 31 41 51
\. \.
COPY x (b,d) from stdin; COPY x (b, d) from stdin;
1 test_1 1 test_1
\. \.
COPY x (b,d) from stdin; COPY x (b, d) from stdin;
2 test_2 2 test_2
3 test_3 3 test_3
4 test_4 4 test_4
5 test_5 5 test_5
\. \.
COPY x (a,b,c,d,e) from stdin; COPY x (a, b, c, d, e) from stdin;
10001 22 32 42 52 10001 22 32 42 52
10002 23 33 43 53 10002 23 33 43 53
10003 24 34 44 54 10003 24 34 44 54
...@@ -49,7 +49,48 @@ COPY x (a,b,c,d,e) from stdin; ...@@ -49,7 +49,48 @@ COPY x (a,b,c,d,e) from stdin;
10005 26 36 46 56 10005 26 36 46 56
\. \.
-- non-existent column in column list: should fail
COPY x (xyz) from stdin;
-- too many columns in column list: should fail
COPY x (a, b, c, d, e, d, c) from stdin;
-- missing data: should fail
COPY x from stdin;
\.
COPY x from stdin;
2000 230 23 23
\.
COPY x from stdin;
2001 231 \N \N
\.
-- extra data: should fail
COPY x from stdin;
2002 232 40 50 60 70 80
\.
-- various COPY options: delimiters, oids, NULL string
COPY x (b, c, d, e) from stdin with oids delimiter ',' null 'x';
500000,x,45,80,90
\.
-- COPY w/ oids on a table w/o oids should fail
CREATE TABLE no_oids (
a int,
b int
) WITHOUT OIDS;
INSERT INTO no_oids (a, b) VALUES (5, 10);
INSERT INTO no_oids (a, b) VALUES (20, 30);
-- should fail
COPY no_oids FROM stdin WITH OIDS;
COPY no_oids TO stdout WITH OIDS;
COPY x TO stdout; COPY x TO stdout;
COPY x (c, e) TO stdout;
DROP TABLE x; DROP TABLE x;
DROP FUNCTION fn_x_before(); DROP FUNCTION fn_x_before();
DROP FUNCTION fn_x_after(); DROP FUNCTION fn_x_after();
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment