Allow contrib/file_fdw to read from a program, like COPY FROM PROGRAM.

This patch just exposes COPY's FROM PROGRAM option in contrib/file_fdw.
There don't seem to be any security issues with that that are any worse
than what already exist with file_fdw and COPY; as in the existing cases,
only superusers are allowed to control what gets executed.

A regression test case might be nice here, but choosing a 100% portable
command to run is hard.  (We haven't got a test for COPY FROM PROGRAM
itself, either.)

Corey Huinker and Adam Gomaa, reviewed by Amit Langote

Discussion: <CADkLM=dGDGmaEiZ=UDepzumWg-CVn7r8MHPjr2NArj8S3TsROQ@mail.gmail.com>

contrib/file_fdw/output/file_fdw.source

@@ -76,7 +76,7 @@ CREATE FOREIGN TABLE tbl () SERVER file_server OPTIONS (format 'csv', null '
 ');       -- ERROR
 ERROR:  COPY null representation cannot use newline or carriage return
 CREATE FOREIGN TABLE tbl () SERVER file_server;  -- ERROR
-ERROR:  filename is required for file_fdw foreign tables
+ERROR:  either filename or program is required for file_fdw foreign tables
 CREATE FOREIGN TABLE agg_text (
 	a	int2 CHECK (a >= 0),
 	b	float4
@@ -132,7 +132,7 @@ ERROR:  invalid option "force_not_null"
 HINT:  There are no valid options in this context.
 CREATE FOREIGN TABLE tbl () SERVER file_server OPTIONS (force_not_null '*'); -- ERROR
 ERROR:  invalid option "force_not_null"
-HINT:  Valid options in this context are: filename, format, header, delimiter, quote, escape, null, encoding
+HINT:  Valid options in this context are: filename, program, format, header, delimiter, quote, escape, null, encoding
 -- force_null is not allowed to be specified at any foreign object level:
 ALTER FOREIGN DATA WRAPPER file_fdw OPTIONS (ADD force_null '*'); -- ERROR
 ERROR:  invalid option "force_null"
@@ -145,7 +145,7 @@ ERROR:  invalid option "force_null"
 HINT:  There are no valid options in this context.
 CREATE FOREIGN TABLE tbl () SERVER file_server OPTIONS (force_null '*'); -- ERROR
 ERROR:  invalid option "force_null"
-HINT:  Valid options in this context are: filename, format, header, delimiter, quote, escape, null, encoding
+HINT:  Valid options in this context are: filename, program, format, header, delimiter, quote, escape, null, encoding
 -- basic query tests
 SELECT * FROM agg_text WHERE b > 10.0 ORDER BY a;
   a  |   b    

doc/src/sgml/file-fdw.sgml

@@ -10,10 +10,11 @@
  <para>
   The <filename>file_fdw</> module provides the foreign-data wrapper
   <function>file_fdw</function>, which can be used to access data
-  files in the server's file system.  Data files must be in a format
+  files in the server's file system, or to execute programs on the server
+  and read their output.  The data file or program output must be in a format
   that can be read by <command>COPY FROM</command>;
   see <xref linkend="sql-copy"> for details.
-  Access to such data files is currently read-only.
+  Access to data files is currently read-only.
  </para>
 
  <para>
@@ -27,7 +28,22 @@
 
    <listitem>
     <para>
-     Specifies the file to be read.  Required.  Must be an absolute path name.
+     Specifies the file to be read.  Must be an absolute path name.
+     Either <literal>filename</literal> or <literal>program</literal> must be
+     specified, but not both.
+    </para>
+   </listitem>
+  </varlistentry>
+
+  <varlistentry>
+   <term><literal>program</literal></term>
+
+   <listitem>
+    <para>
+     Specifies the command to be executed.  The standard output of this
+     command will be read as though <command>COPY FROM PROGRAM</> were used.
+     Either <literal>program</literal> or <literal>filename</literal> must be
+     specified, but not both.
     </para>
    </listitem>
   </varlistentry>
@@ -37,7 +53,7 @@
 
    <listitem>
     <para>
-     Specifies the file's format,
+     Specifies the data format,
      the same as <command>COPY</>'s <literal>FORMAT</literal> option.
     </para>
    </listitem>
@@ -48,7 +64,7 @@
 
    <listitem>
     <para>
-     Specifies whether the file has a header line,
+     Specifies whether the data has a header line,
      the same as <command>COPY</>'s <literal>HEADER</literal> option.
     </para>
    </listitem>
@@ -59,7 +75,7 @@
 
    <listitem>
     <para>
-     Specifies the file's delimiter character,
+     Specifies the data delimiter character,
      the same as <command>COPY</>'s <literal>DELIMITER</literal> option.
     </para>
    </listitem>
@@ -70,7 +86,7 @@
 
    <listitem>
     <para>
-     Specifies the file's quote character,
+     Specifies the data quote character,
      the same as <command>COPY</>'s <literal>QUOTE</literal> option.
     </para>
    </listitem>
@@ -81,7 +97,7 @@
 
    <listitem>
     <para>
-     Specifies the file's escape character,
+     Specifies the data escape character,
      the same as <command>COPY</>'s <literal>ESCAPE</literal> option.
     </para>
    </listitem>
@@ -92,7 +108,7 @@
 
    <listitem>
     <para>
-     Specifies the file's null string,
+     Specifies the data null string,
      the same as <command>COPY</>'s <literal>NULL</literal> option.
     </para>
    </listitem>
@@ -103,7 +119,7 @@
 
    <listitem>
     <para>
-     Specifies the file's encoding,
+     Specifies the data encoding,
      the same as <command>COPY</>'s <literal>ENCODING</literal> option.
     </para>
    </listitem>
@@ -112,11 +128,11 @@
  </variablelist>
 
  <para>
-  Note that while <command>COPY</> allows options such as OIDS and HEADER
-  to be specified without a corresponding value, the foreign data wrapper
+  Note that while <command>COPY</> allows options such as <literal>HEADER</>
+  to be specified without a corresponding value, the foreign table option
   syntax requires a value to be present in all cases.  To activate
-  <command>COPY</> options normally supplied without a value, you can
-  instead pass the value TRUE.
+  <command>COPY</> options typically written without a value, you can pass
+  the value TRUE, since all such options are Booleans.
  </para>
 
  <para>
@@ -133,7 +149,7 @@
     <para>
      This is a Boolean option.  If true, it specifies that values of the
      column should not be matched against the null string (that is, the
-     file-level <literal>null</literal> option).  This has the same effect
+     table-level <literal>null</literal> option).  This has the same effect
      as listing the column in <command>COPY</>'s
      <literal>FORCE_NOT_NULL</literal> option.
     </para>
@@ -171,14 +187,24 @@
 
  <para>
   Changing table-level options requires superuser privileges, for security
-  reasons: only a superuser should be able to determine which file is read.
-  In principle non-superusers could be allowed to change the other options,
-  but that's not supported at present.
+  reasons: only a superuser should be able to control which file is read
+  or which program is run.  In principle non-superusers could be allowed to
+  change the other options, but that's not supported at present.
+ </para>
+
+ <para>
+  When specifying the <literal>program</> option, keep in mind that the option
+  string is executed by the shell.  If you need to pass any arguments to the
+  command that come from an untrusted source, you must be careful to strip or
+  escape any characters that might have special meaning to the shell.
+  For security reasons, it is best to use a fixed command string, or at least
+  avoid passing any user input in it.
  </para>
 
  <para>
   For a foreign table using <literal>file_fdw</>, <command>EXPLAIN</> shows
-  the name of the file to be read.  Unless <literal>COSTS OFF</> is
+  the name of the file to be read or program to be run.
+  For a file, unless <literal>COSTS OFF</> is
   specified, the file size (in bytes) is shown as well.
  </para>
 
@@ -186,7 +212,7 @@
  <title id="csvlog-fdw">Create a Foreign Table for PostgreSQL CSV Logs</title>
 
   <para>
-   One of the obvious uses for the <literal>file_fdw</> is to make
+   One of the obvious uses for <literal>file_fdw</> is to make
    the PostgreSQL activity log available as a table for querying.  To
    do this, first you must be logging to a CSV file, which here we
    will call <literal>pglog.csv</>.  First, install <literal>file_fdw</>