Commit 69f4b9c8 authored by Andres Freund's avatar Andres Freund

Move targetlist SRF handling from expression evaluation to new executor node.

Evaluation of set returning functions (SRFs_ in the targetlist (like SELECT
generate_series(1,5)) so far was done in the expression evaluation (i.e.
ExecEvalExpr()) and projection (i.e. ExecProject/ExecTargetList) code.

This meant that most executor nodes performing projection, and most
expression evaluation functions, had to deal with the possibility that an
evaluated expression could return a set of return values.

That's bad because it leads to repeated code in a lot of places. It also,
and that's my (Andres's) motivation, made it a lot harder to implement a
more efficient way of doing expression evaluation.

To fix this, introduce a new executor node (ProjectSet) that can evaluate
targetlists containing one or more SRFs. To avoid the complexity of the old
way of handling nested expressions returning sets (e.g. having to pass up
ExprDoneCond, and dealing with arguments to functions returning sets etc.),
those SRFs can only be at the top level of the node's targetlist.  The
planner makes sure (via split_pathtarget_at_srfs()) that SRF evaluation is
only necessary in ProjectSet nodes and that SRFs are only present at the
top level of the node's targetlist. If there are nested SRFs the planner
creates multiple stacked ProjectSet nodes.  The ProjectSet nodes always get
input from an underlying node.

We also discussed and prototyped evaluating targetlist SRFs using ROWS
FROM(), but that turned out to be more complicated than we'd hoped.

While moving SRF evaluation to ProjectSet would allow to retain the old
"least common multiple" behavior when multiple SRFs are present in one
targetlist (i.e.  continue returning rows until all SRFs are at the end of
their input at the same time), we decided to instead only return rows till
all SRFs are exhausted, returning NULL for already exhausted ones.  We
deemed the previous behavior to be too confusing, unexpected and actually
not particularly useful.

As a side effect, the previously prohibited case of multiple set returning
arguments to a function, is now allowed. Not because it's particularly
desirable, but because it ends up working and there seems to be no argument
for adding code to prohibit it.

Currently the behavior for COALESCE and CASE containing SRFs has changed,
returning multiple rows from the expression, even when the SRF containing
"arm" of the expression is not evaluated. That's because the SRFs are
evaluated in a separate ProjectSet node.  As that's quite confusing, we're
likely to instead prohibit SRFs in those places.  But that's still being
discussed, and the code would reside in places not touched here, so that's
a task for later.

There's a lot of, now superfluous, code dealing with set return expressions
around. But as the changes to get rid of those are verbose largely boring,
it seems better for readability to keep the cleanup as a separate commit.

Author: Tom Lane and Andres Freund
Discussion: https://postgr.es/m/20160822214023.aaxz5l4igypowyri@alap3.anarazel.de
parent e37360d5
...@@ -962,12 +962,11 @@ SELECT name, child FROM nodes, LATERAL listchildren(name) AS child; ...@@ -962,12 +962,11 @@ SELECT name, child FROM nodes, LATERAL listchildren(name) AS child;
</para> </para>
<para> <para>
Currently, functions returning sets can also be called in the select list Functions returning sets can also be called in the select list
of a query. For each row that the query of a query. For each row that the query
generates by itself, the function returning set is invoked, and an output generates by itself, the set-returning function is invoked, and an output
row is generated for each element of the function's result set. Note, row is generated for each element of the function's result set.
however, that this capability is deprecated and might be removed in future The previous example could also be done with queries like
releases. The previous example could also be done with queries like
these: these:
<screen> <screen>
...@@ -998,6 +997,33 @@ SELECT name, listchildren(name) FROM nodes; ...@@ -998,6 +997,33 @@ SELECT name, listchildren(name) FROM nodes;
the <literal>LATERAL</> syntax. the <literal>LATERAL</> syntax.
</para> </para>
<para>
If there is more than one set-returning function in the same select
list, the behavior is similar to what you get from putting the functions
into a single <literal>LATERAL ROWS FROM( ... )</> <literal>FROM</>-clause
item. For each row from the underlying query, there is an output row
using the first result from each function, then an output row using the
second result, and so on. If some of the set-returning functions
produce fewer outputs than others, null values are substituted for the
missing data, so that the total number of rows emitted for one
underlying row is the same as for the set-returning function that
produced the most outputs.
</para>
<para>
Set-returning functions can be nested in a select list, although that is
not allowed in <literal>FROM</>-clause items. In such cases, each level
of nesting is treated separately, as though it were
another <literal>LATERAL ROWS FROM( ... )</> item. For example, in
<programlisting>
SELECT srf1(srf2(x), srf3(y)), srf4(srf5(z)) FROM ...
</programlisting>
the set-returning functions <function>srf2</>, <function>srf3</>,
and <function>srf5</> would be run in lockstep for each row of the
underlying query, and then <function>srf1</> and <function>srf4</> would
be applied in lockstep to each row produced by the lower functions.
</para>
<note> <note>
<para> <para>
If a function's last command is <command>INSERT</>, <command>UPDATE</>, If a function's last command is <command>INSERT</>, <command>UPDATE</>,
...@@ -1012,14 +1038,14 @@ SELECT name, listchildren(name) FROM nodes; ...@@ -1012,14 +1038,14 @@ SELECT name, listchildren(name) FROM nodes;
<note> <note>
<para> <para>
The key problem with using set-returning functions in the select list, Before <productname>PostgreSQL</> 10, putting more than one
rather than the <literal>FROM</> clause, is that putting more than one set-returning function in the same select list did not behave very
set-returning function in the same select list does not behave very sensibly unless they always produced equal numbers of rows. Otherwise,
sensibly. (What you actually get if you do so is a number of output what you got was a number of output rows equal to the least common
rows equal to the least common multiple of the numbers of rows produced multiple of the numbers of rows produced by the set-returning
by each set-returning function.) The <literal>LATERAL</> syntax functions. Furthermore, nested set-returning functions did not work at
produces less surprising results when calling multiple set-returning all. Use of the <literal>LATERAL</> syntax is recommended when writing
functions, and should usually be used instead. queries that need to work in older <productname>PostgreSQL</> versions.
</para> </para>
</note> </note>
</sect2> </sect2>
......
...@@ -852,6 +852,9 @@ ExplainNode(PlanState *planstate, List *ancestors, ...@@ -852,6 +852,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
case T_Result: case T_Result:
pname = sname = "Result"; pname = sname = "Result";
break; break;
case T_ProjectSet:
pname = sname = "ProjectSet";
break;
case T_ModifyTable: case T_ModifyTable:
sname = "ModifyTable"; sname = "ModifyTable";
switch (((ModifyTable *) plan)->operation) switch (((ModifyTable *) plan)->operation)
......
...@@ -17,11 +17,12 @@ OBJS = execAmi.o execCurrent.o execGrouping.o execIndexing.o execJunk.o \ ...@@ -17,11 +17,12 @@ OBJS = execAmi.o execCurrent.o execGrouping.o execIndexing.o execJunk.o \
execScan.o execTuples.o \ execScan.o execTuples.o \
execUtils.o functions.o instrument.o nodeAppend.o nodeAgg.o \ execUtils.o functions.o instrument.o nodeAppend.o nodeAgg.o \
nodeBitmapAnd.o nodeBitmapOr.o \ nodeBitmapAnd.o nodeBitmapOr.o \
nodeBitmapHeapscan.o nodeBitmapIndexscan.o nodeCustom.o nodeGather.o \ nodeBitmapHeapscan.o nodeBitmapIndexscan.o \
nodeCustom.o nodeFunctionscan.o nodeGather.o \
nodeHash.o nodeHashjoin.o nodeIndexscan.o nodeIndexonlyscan.o \ nodeHash.o nodeHashjoin.o nodeIndexscan.o nodeIndexonlyscan.o \
nodeLimit.o nodeLockRows.o \ nodeLimit.o nodeLockRows.o \
nodeMaterial.o nodeMergeAppend.o nodeMergejoin.o nodeModifyTable.o \ nodeMaterial.o nodeMergeAppend.o nodeMergejoin.o nodeModifyTable.o \
nodeNestloop.o nodeFunctionscan.o nodeRecursiveunion.o nodeResult.o \ nodeNestloop.o nodeProjectSet.o nodeRecursiveunion.o nodeResult.o \
nodeSamplescan.o nodeSeqscan.o nodeSetOp.o nodeSort.o nodeUnique.o \ nodeSamplescan.o nodeSeqscan.o nodeSetOp.o nodeSort.o nodeUnique.o \
nodeValuesscan.o nodeCtescan.o nodeWorktablescan.o \ nodeValuesscan.o nodeCtescan.o nodeWorktablescan.o \
nodeGroup.o nodeSubplan.o nodeSubqueryscan.o nodeTidscan.o \ nodeGroup.o nodeSubplan.o nodeSubqueryscan.o nodeTidscan.o \
......
...@@ -39,6 +39,7 @@ ...@@ -39,6 +39,7 @@
#include "executor/nodeMergejoin.h" #include "executor/nodeMergejoin.h"
#include "executor/nodeModifyTable.h" #include "executor/nodeModifyTable.h"
#include "executor/nodeNestloop.h" #include "executor/nodeNestloop.h"
#include "executor/nodeProjectSet.h"
#include "executor/nodeRecursiveunion.h" #include "executor/nodeRecursiveunion.h"
#include "executor/nodeResult.h" #include "executor/nodeResult.h"
#include "executor/nodeSamplescan.h" #include "executor/nodeSamplescan.h"
...@@ -130,6 +131,10 @@ ExecReScan(PlanState *node) ...@@ -130,6 +131,10 @@ ExecReScan(PlanState *node)
ExecReScanResult((ResultState *) node); ExecReScanResult((ResultState *) node);
break; break;
case T_ProjectSetState:
ExecReScanProjectSet((ProjectSetState *) node);
break;
case T_ModifyTableState: case T_ModifyTableState:
ExecReScanModifyTable((ModifyTableState *) node); ExecReScanModifyTable((ModifyTableState *) node);
break; break;
......
...@@ -88,6 +88,7 @@ ...@@ -88,6 +88,7 @@
#include "executor/nodeCustom.h" #include "executor/nodeCustom.h"
#include "executor/nodeForeignscan.h" #include "executor/nodeForeignscan.h"
#include "executor/nodeFunctionscan.h" #include "executor/nodeFunctionscan.h"
#include "executor/nodeGather.h"
#include "executor/nodeGroup.h" #include "executor/nodeGroup.h"
#include "executor/nodeHash.h" #include "executor/nodeHash.h"
#include "executor/nodeHashjoin.h" #include "executor/nodeHashjoin.h"
...@@ -100,7 +101,7 @@ ...@@ -100,7 +101,7 @@
#include "executor/nodeMergejoin.h" #include "executor/nodeMergejoin.h"
#include "executor/nodeModifyTable.h" #include "executor/nodeModifyTable.h"
#include "executor/nodeNestloop.h" #include "executor/nodeNestloop.h"
#include "executor/nodeGather.h" #include "executor/nodeProjectSet.h"
#include "executor/nodeRecursiveunion.h" #include "executor/nodeRecursiveunion.h"
#include "executor/nodeResult.h" #include "executor/nodeResult.h"
#include "executor/nodeSamplescan.h" #include "executor/nodeSamplescan.h"
...@@ -155,6 +156,11 @@ ExecInitNode(Plan *node, EState *estate, int eflags) ...@@ -155,6 +156,11 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
estate, eflags); estate, eflags);
break; break;
case T_ProjectSet:
result = (PlanState *) ExecInitProjectSet((ProjectSet *) node,
estate, eflags);
break;
case T_ModifyTable: case T_ModifyTable:
result = (PlanState *) ExecInitModifyTable((ModifyTable *) node, result = (PlanState *) ExecInitModifyTable((ModifyTable *) node,
estate, eflags); estate, eflags);
...@@ -392,6 +398,10 @@ ExecProcNode(PlanState *node) ...@@ -392,6 +398,10 @@ ExecProcNode(PlanState *node)
result = ExecResult((ResultState *) node); result = ExecResult((ResultState *) node);
break; break;
case T_ProjectSetState:
result = ExecProjectSet((ProjectSetState *) node);
break;
case T_ModifyTableState: case T_ModifyTableState:
result = ExecModifyTable((ModifyTableState *) node); result = ExecModifyTable((ModifyTableState *) node);
break; break;
...@@ -634,6 +644,10 @@ ExecEndNode(PlanState *node) ...@@ -634,6 +644,10 @@ ExecEndNode(PlanState *node)
ExecEndResult((ResultState *) node); ExecEndResult((ResultState *) node);
break; break;
case T_ProjectSetState:
ExecEndProjectSet((ProjectSetState *) node);
break;
case T_ModifyTableState: case T_ModifyTableState:
ExecEndModifyTable((ModifyTableState *) node); ExecEndModifyTable((ModifyTableState *) node);
break; break;
......
This diff is collapsed.
/*-------------------------------------------------------------------------
*
* nodeProjectSet.c
* support for evaluating targetlists containing set-returning functions
*
* DESCRIPTION
*
* ProjectSet nodes are inserted by the planner to evaluate set-returning
* functions in the targetlist. It's guaranteed that all set-returning
* functions are directly at the top level of the targetlist, i.e. they
* can't be inside more-complex expressions. If that'd otherwise be
* the case, the planner adds additional ProjectSet nodes.
*
* Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* IDENTIFICATION
* src/backend/executor/nodeProjectSet.c
*
*-------------------------------------------------------------------------
*/
#include "postgres.h"
#include "executor/executor.h"
#include "executor/nodeProjectSet.h"
#include "utils/memutils.h"
static TupleTableSlot *ExecProjectSRF(ProjectSetState *node, bool continuing);
/* ----------------------------------------------------------------
* ExecProjectSet(node)
*
* Return tuples after evaluating the targetlist (which contains set
* returning functions).
* ----------------------------------------------------------------
*/
TupleTableSlot *
ExecProjectSet(ProjectSetState *node)
{
TupleTableSlot *outerTupleSlot;
TupleTableSlot *resultSlot;
PlanState *outerPlan;
ExprContext *econtext;
econtext = node->ps.ps_ExprContext;
/*
* Check to see if we're still projecting out tuples from a previous scan
* tuple (because there is a function-returning-set in the projection
* expressions). If so, try to project another one.
*/
if (node->pending_srf_tuples)
{
resultSlot = ExecProjectSRF(node, true);
if (resultSlot != NULL)
return resultSlot;
}
/*
* Reset per-tuple memory context to free any expression evaluation
* storage allocated in the previous tuple cycle. Note this can't happen
* until we're done projecting out tuples from a scan tuple.
*/
ResetExprContext(econtext);
/*
* Get another input tuple and project SRFs from it.
*/
for (;;)
{
/*
* Retrieve tuples from the outer plan until there are no more.
*/
outerPlan = outerPlanState(node);
outerTupleSlot = ExecProcNode(outerPlan);
if (TupIsNull(outerTupleSlot))
return NULL;
/*
* Prepare to compute projection expressions, which will expect to
* access the input tuples as varno OUTER.
*/
econtext->ecxt_outertuple = outerTupleSlot;
/* Evaluate the expressions */
resultSlot = ExecProjectSRF(node, false);
/*
* Return the tuple unless the projection produced no rows (due to an
* empty set), in which case we must loop back to see if there are
* more outerPlan tuples.
*/
if (resultSlot)
return resultSlot;
}
return NULL;
}
/* ----------------------------------------------------------------
* ExecProjectSRF
*
* Project a targetlist containing one or more set-returning functions.
*
* 'continuing' indicates whether to continue projecting rows for the
* same input tuple; or whether a new input tuple is being projected.
*
* Returns NULL if no output tuple has been produced.
*
* ----------------------------------------------------------------
*/
static TupleTableSlot *
ExecProjectSRF(ProjectSetState *node, bool continuing)
{
TupleTableSlot *resultSlot = node->ps.ps_ResultTupleSlot;
ExprContext *econtext = node->ps.ps_ExprContext;
bool hassrf PG_USED_FOR_ASSERTS_ONLY = false;
bool hasresult;
int argno;
ListCell *lc;
ExecClearTuple(resultSlot);
/*
* Assume no further tuples are produced unless an ExprMultipleResult is
* encountered from a set returning function.
*/
node->pending_srf_tuples = false;
hasresult = false;
argno = 0;
foreach(lc, node->ps.targetlist)
{
GenericExprState *gstate = (GenericExprState *) lfirst(lc);
ExprDoneCond *isdone = &node->elemdone[argno];
Datum *result = &resultSlot->tts_values[argno];
bool *isnull = &resultSlot->tts_isnull[argno];
if (continuing && *isdone == ExprEndResult)
{
/*
* If we're continuing to project output rows from a source tuple,
* return NULLs once the SRF has been exhausted.
*/
*result = (Datum) 0;
*isnull = true;
hassrf = true;
}
else if (IsA(gstate->arg, FuncExprState) &&
((FuncExprState *) gstate->arg)->funcReturnsSet)
{
/*
* Evaluate SRF - possibly continuing previously started output.
*/
*result = ExecMakeFunctionResultSet((FuncExprState *) gstate->arg,
econtext, isnull, isdone);
if (*isdone != ExprEndResult)
hasresult = true;
if (*isdone == ExprMultipleResult)
node->pending_srf_tuples = true;
hassrf = true;
}
else
{
/* Non-SRF tlist expression, just evaluate normally. */
*result = ExecEvalExpr(gstate->arg, econtext, isnull, NULL);
*isdone = ExprSingleResult;
}
argno++;
}
/* ProjectSet should not be used if there's no SRFs */
Assert(hassrf);
/*
* If all the SRFs returned EndResult, we consider that as no row being
* produced.
*/
if (hasresult)
{
ExecStoreVirtualTuple(resultSlot);
return resultSlot;
}
return NULL;
}
/* ----------------------------------------------------------------
* ExecInitProjectSet
*
* Creates the run-time state information for the ProjectSet node
* produced by the planner and initializes outer relations
* (child nodes).
* ----------------------------------------------------------------
*/
ProjectSetState *
ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
{
ProjectSetState *state;
/* check for unsupported flags */
Assert(!(eflags & (EXEC_FLAG_MARK | EXEC_FLAG_BACKWARD)));
/*
* create state structure
*/
state = makeNode(ProjectSetState);
state->ps.plan = (Plan *) node;
state->ps.state = estate;
state->pending_srf_tuples = false;
/*
* Miscellaneous initialization
*
* create expression context for node
*/
ExecAssignExprContext(estate, &state->ps);
/*
* tuple table initialization
*/
ExecInitResultTupleSlot(estate, &state->ps);
/*
* initialize child expressions
*/
state->ps.targetlist = (List *)
ExecInitExpr((Expr *) node->plan.targetlist,
(PlanState *) state);
Assert(node->plan.qual == NIL);
/*
* initialize child nodes
*/
outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags);
/*
* we don't use inner plan
*/
Assert(innerPlan(node) == NULL);
/*
* initialize tuple type and projection info
*/
ExecAssignResultTypeFromTL(&state->ps);
/* Create workspace for per-SRF is-done state */
state->nelems = list_length(node->plan.targetlist);
state->elemdone = (ExprDoneCond *)
palloc(sizeof(ExprDoneCond) * state->nelems);
return state;
}
/* ----------------------------------------------------------------
* ExecEndProjectSet
*
* frees up storage allocated through C routines
* ----------------------------------------------------------------
*/
void
ExecEndProjectSet(ProjectSetState *node)
{
/*
* Free the exprcontext
*/
ExecFreeExprContext(&node->ps);
/*
* clean out the tuple table
*/
ExecClearTuple(node->ps.ps_ResultTupleSlot);
/*
* shut down subplans
*/
ExecEndNode(outerPlanState(node));
}
void
ExecReScanProjectSet(ProjectSetState *node)
{
/* Forget any incompletely-evaluated SRFs */
node->pending_srf_tuples = false;
/*
* If chgParam of subnode is not null then plan will be re-scanned by
* first ExecProcNode.
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
}
...@@ -165,6 +165,22 @@ _copyResult(const Result *from) ...@@ -165,6 +165,22 @@ _copyResult(const Result *from)
return newnode; return newnode;
} }
/*
* _copyProjectSet
*/
static ProjectSet *
_copyProjectSet(const ProjectSet *from)
{
ProjectSet *newnode = makeNode(ProjectSet);
/*
* copy node superclass fields
*/
CopyPlanFields((const Plan *) from, (Plan *) newnode);
return newnode;
}
/* /*
* _copyModifyTable * _copyModifyTable
*/ */
...@@ -4415,6 +4431,9 @@ copyObject(const void *from) ...@@ -4415,6 +4431,9 @@ copyObject(const void *from)
case T_Result: case T_Result:
retval = _copyResult(from); retval = _copyResult(from);
break; break;
case T_ProjectSet:
retval = _copyProjectSet(from);
break;
case T_ModifyTable: case T_ModifyTable:
retval = _copyModifyTable(from); retval = _copyModifyTable(from);
break; break;
......
...@@ -326,6 +326,14 @@ _outResult(StringInfo str, const Result *node) ...@@ -326,6 +326,14 @@ _outResult(StringInfo str, const Result *node)
WRITE_NODE_FIELD(resconstantqual); WRITE_NODE_FIELD(resconstantqual);
} }
static void
_outProjectSet(StringInfo str, const ProjectSet *node)
{
WRITE_NODE_TYPE("PROJECTSET");
_outPlanInfo(str, (const Plan *) node);
}
static void static void
_outModifyTable(StringInfo str, const ModifyTable *node) _outModifyTable(StringInfo str, const ModifyTable *node)
{ {
...@@ -1807,6 +1815,16 @@ _outProjectionPath(StringInfo str, const ProjectionPath *node) ...@@ -1807,6 +1815,16 @@ _outProjectionPath(StringInfo str, const ProjectionPath *node)
WRITE_BOOL_FIELD(dummypp); WRITE_BOOL_FIELD(dummypp);
} }
static void
_outProjectSetPath(StringInfo str, const ProjectSetPath *node)
{
WRITE_NODE_TYPE("PROJECTSETPATH");
_outPathInfo(str, (const Path *) node);
WRITE_NODE_FIELD(subpath);
}
static void static void
_outSortPath(StringInfo str, const SortPath *node) _outSortPath(StringInfo str, const SortPath *node)
{ {
...@@ -3367,6 +3385,9 @@ outNode(StringInfo str, const void *obj) ...@@ -3367,6 +3385,9 @@ outNode(StringInfo str, const void *obj)
case T_Result: case T_Result:
_outResult(str, obj); _outResult(str, obj);
break; break;
case T_ProjectSet:
_outProjectSet(str, obj);
break;
case T_ModifyTable: case T_ModifyTable:
_outModifyTable(str, obj); _outModifyTable(str, obj);
break; break;
...@@ -3679,6 +3700,9 @@ outNode(StringInfo str, const void *obj) ...@@ -3679,6 +3700,9 @@ outNode(StringInfo str, const void *obj)
case T_ProjectionPath: case T_ProjectionPath:
_outProjectionPath(str, obj); _outProjectionPath(str, obj);
break; break;
case T_ProjectSetPath:
_outProjectSetPath(str, obj);
break;
case T_SortPath: case T_SortPath:
_outSortPath(str, obj); _outSortPath(str, obj);
break; break;
......
...@@ -1483,6 +1483,19 @@ _readResult(void) ...@@ -1483,6 +1483,19 @@ _readResult(void)
READ_DONE(); READ_DONE();
} }
/*
* _readProjectSet
*/
static ProjectSet *
_readProjectSet(void)
{
READ_LOCALS_NO_FIELDS(ProjectSet);
ReadCommonPlan(&local_node->plan);
READ_DONE();
}
/* /*
* _readModifyTable * _readModifyTable
*/ */
...@@ -2450,6 +2463,8 @@ parseNodeString(void) ...@@ -2450,6 +2463,8 @@ parseNodeString(void)
return_value = _readPlan(); return_value = _readPlan();
else if (MATCH("RESULT", 6)) else if (MATCH("RESULT", 6))
return_value = _readResult(); return_value = _readResult();
else if (MATCH("PROJECTSET", 10))
return_value = _readProjectSet();
else if (MATCH("MODIFYTABLE", 11)) else if (MATCH("MODIFYTABLE", 11))
return_value = _readModifyTable(); return_value = _readModifyTable();
else if (MATCH("APPEND", 6)) else if (MATCH("APPEND", 6))
......
...@@ -375,6 +375,7 @@ RelOptInfo - a relation or joined relations ...@@ -375,6 +375,7 @@ RelOptInfo - a relation or joined relations
UniquePath - remove duplicate rows (either by hashing or sorting) UniquePath - remove duplicate rows (either by hashing or sorting)
GatherPath - collect the results of parallel workers GatherPath - collect the results of parallel workers
ProjectionPath - a Result plan node with child (used for projection) ProjectionPath - a Result plan node with child (used for projection)
ProjectSetPath - a ProjectSet plan node applied to some sub-path
SortPath - a Sort plan node applied to some sub-path SortPath - a Sort plan node applied to some sub-path
GroupPath - a Group plan node applied to some sub-path GroupPath - a Group plan node applied to some sub-path
UpperUniquePath - a Unique plan node applied to some sub-path UpperUniquePath - a Unique plan node applied to some sub-path
......
...@@ -3051,6 +3051,10 @@ print_path(PlannerInfo *root, Path *path, int indent) ...@@ -3051,6 +3051,10 @@ print_path(PlannerInfo *root, Path *path, int indent)
ptype = "Projection"; ptype = "Projection";
subpath = ((ProjectionPath *) path)->subpath; subpath = ((ProjectionPath *) path)->subpath;
break; break;
case T_ProjectSetPath:
ptype = "ProjectSet";
subpath = ((ProjectSetPath *) path)->subpath;
break;
case T_SortPath: case T_SortPath:
ptype = "Sort"; ptype = "Sort";
subpath = ((SortPath *) path)->subpath; subpath = ((SortPath *) path)->subpath;
......
...@@ -81,6 +81,7 @@ static Plan *create_join_plan(PlannerInfo *root, JoinPath *best_path); ...@@ -81,6 +81,7 @@ static Plan *create_join_plan(PlannerInfo *root, JoinPath *best_path);
static Plan *create_append_plan(PlannerInfo *root, AppendPath *best_path); static Plan *create_append_plan(PlannerInfo *root, AppendPath *best_path);
static Plan *create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path); static Plan *create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path);
static Result *create_result_plan(PlannerInfo *root, ResultPath *best_path); static Result *create_result_plan(PlannerInfo *root, ResultPath *best_path);
static ProjectSet *create_project_set_plan(PlannerInfo *root, ProjectSetPath *best_path);
static Material *create_material_plan(PlannerInfo *root, MaterialPath *best_path, static Material *create_material_plan(PlannerInfo *root, MaterialPath *best_path,
int flags); int flags);
static Plan *create_unique_plan(PlannerInfo *root, UniquePath *best_path, static Plan *create_unique_plan(PlannerInfo *root, UniquePath *best_path,
...@@ -264,6 +265,7 @@ static SetOp *make_setop(SetOpCmd cmd, SetOpStrategy strategy, Plan *lefttree, ...@@ -264,6 +265,7 @@ static SetOp *make_setop(SetOpCmd cmd, SetOpStrategy strategy, Plan *lefttree,
long numGroups); long numGroups);
static LockRows *make_lockrows(Plan *lefttree, List *rowMarks, int epqParam); static LockRows *make_lockrows(Plan *lefttree, List *rowMarks, int epqParam);
static Result *make_result(List *tlist, Node *resconstantqual, Plan *subplan); static Result *make_result(List *tlist, Node *resconstantqual, Plan *subplan);
static ProjectSet *make_project_set(List *tlist, Plan *subplan);
static ModifyTable *make_modifytable(PlannerInfo *root, static ModifyTable *make_modifytable(PlannerInfo *root,
CmdType operation, bool canSetTag, CmdType operation, bool canSetTag,
Index nominalRelation, Index nominalRelation,
...@@ -392,6 +394,10 @@ create_plan_recurse(PlannerInfo *root, Path *best_path, int flags) ...@@ -392,6 +394,10 @@ create_plan_recurse(PlannerInfo *root, Path *best_path, int flags)
(ResultPath *) best_path); (ResultPath *) best_path);
} }
break; break;
case T_ProjectSet:
plan = (Plan *) create_project_set_plan(root,
(ProjectSetPath *) best_path);
break;
case T_Material: case T_Material:
plan = (Plan *) create_material_plan(root, plan = (Plan *) create_material_plan(root,
(MaterialPath *) best_path, (MaterialPath *) best_path,
...@@ -1141,6 +1147,31 @@ create_result_plan(PlannerInfo *root, ResultPath *best_path) ...@@ -1141,6 +1147,31 @@ create_result_plan(PlannerInfo *root, ResultPath *best_path)
return plan; return plan;
} }
/*
* create_project_set_plan
* Create a ProjectSet plan for 'best_path'.
*
* Returns a Plan node.
*/
static ProjectSet *
create_project_set_plan(PlannerInfo *root, ProjectSetPath *best_path)
{
ProjectSet *plan;
Plan *subplan;
List *tlist;
/* Since we intend to project, we don't need to constrain child tlist */
subplan = create_plan_recurse(root, best_path->subpath, 0);
tlist = build_path_tlist(root, &best_path->path);
plan = make_project_set(tlist, subplan);
copy_generic_path_info(&plan->plan, (Path *) best_path);
return plan;
}
/* /*
* create_material_plan * create_material_plan
* Create a Material plan for 'best_path' and (recursively) plans * Create a Material plan for 'best_path' and (recursively) plans
...@@ -6063,6 +6094,25 @@ make_result(List *tlist, ...@@ -6063,6 +6094,25 @@ make_result(List *tlist,
return node; return node;
} }
/*
* make_project_set
* Build a ProjectSet plan node
*/
static ProjectSet *
make_project_set(List *tlist,
Plan *subplan)
{
ProjectSet *node = makeNode(ProjectSet);
Plan *plan = &node->plan;
plan->targetlist = tlist;
plan->qual = NIL;
plan->lefttree = subplan;
plan->righttree = NULL;
return node;
}
/* /*
* make_modifytable * make_modifytable
* Build a ModifyTable plan node * Build a ModifyTable plan node
...@@ -6229,6 +6279,15 @@ is_projection_capable_path(Path *path) ...@@ -6229,6 +6279,15 @@ is_projection_capable_path(Path *path)
* projection to its dummy path. * projection to its dummy path.
*/ */
return IS_DUMMY_PATH(path); return IS_DUMMY_PATH(path);
case T_ProjectSet:
/*
* Although ProjectSet certainly projects, say "no" because we
* don't want the planner to randomly replace its tlist with
* something else; the SRFs have to stay at top level. This might
* get relaxed later.
*/
return false;
default: default:
break; break;
} }
...@@ -6257,6 +6316,15 @@ is_projection_capable_plan(Plan *plan) ...@@ -6257,6 +6316,15 @@ is_projection_capable_plan(Plan *plan)
case T_MergeAppend: case T_MergeAppend:
case T_RecursiveUnion: case T_RecursiveUnion:
return false; return false;
case T_ProjectSet:
/*
* Although ProjectSet certainly projects, say "no" because we
* don't want the planner to randomly replace its tlist with
* something else; the SRFs have to stay at top level. This might
* get relaxed later.
*/
return false;
default: default:
break; break;
} }
......
This diff is collapsed.
...@@ -733,6 +733,9 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset) ...@@ -733,6 +733,9 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
fix_scan_expr(root, splan->resconstantqual, rtoffset); fix_scan_expr(root, splan->resconstantqual, rtoffset);
} }
break; break;
case T_ProjectSet:
set_upper_references(root, plan, rtoffset);
break;
case T_ModifyTable: case T_ModifyTable:
{ {
ModifyTable *splan = (ModifyTable *) plan; ModifyTable *splan = (ModifyTable *) plan;
......
...@@ -2680,6 +2680,7 @@ finalize_plan(PlannerInfo *root, Plan *plan, Bitmapset *valid_params, ...@@ -2680,6 +2680,7 @@ finalize_plan(PlannerInfo *root, Plan *plan, Bitmapset *valid_params,
&context); &context);
break; break;
case T_ProjectSet:
case T_Hash: case T_Hash:
case T_Material: case T_Material:
case T_Sort: case T_Sort:
...@@ -2687,6 +2688,7 @@ finalize_plan(PlannerInfo *root, Plan *plan, Bitmapset *valid_params, ...@@ -2687,6 +2688,7 @@ finalize_plan(PlannerInfo *root, Plan *plan, Bitmapset *valid_params,
case T_Gather: case T_Gather:
case T_SetOp: case T_SetOp:
case T_Group: case T_Group:
/* no node-type-specific fields need fixing */
break; break;
default: default:
......
...@@ -99,7 +99,6 @@ static bool contain_agg_clause_walker(Node *node, void *context); ...@@ -99,7 +99,6 @@ static bool contain_agg_clause_walker(Node *node, void *context);
static bool get_agg_clause_costs_walker(Node *node, static bool get_agg_clause_costs_walker(Node *node,
get_agg_clause_costs_context *context); get_agg_clause_costs_context *context);
static bool find_window_functions_walker(Node *node, WindowFuncLists *lists); static bool find_window_functions_walker(Node *node, WindowFuncLists *lists);
static bool expression_returns_set_rows_walker(Node *node, double *count);
static bool contain_subplans_walker(Node *node, void *context); static bool contain_subplans_walker(Node *node, void *context);
static bool contain_mutable_functions_walker(Node *node, void *context); static bool contain_mutable_functions_walker(Node *node, void *context);
static bool contain_volatile_functions_walker(Node *node, void *context); static bool contain_volatile_functions_walker(Node *node, void *context);
...@@ -790,114 +789,37 @@ find_window_functions_walker(Node *node, WindowFuncLists *lists) ...@@ -790,114 +789,37 @@ find_window_functions_walker(Node *node, WindowFuncLists *lists)
/* /*
* expression_returns_set_rows * expression_returns_set_rows
* Estimate the number of rows returned by a set-returning expression. * Estimate the number of rows returned by a set-returning expression.
* The result is 1 if there are no set-returning functions. * The result is 1 if it's not a set-returning expression.
* *
* We use the product of the rowcount estimates of all the functions in * We should only examine the top-level function or operator; it used to be
* the given tree (this corresponds to the behavior of ExecMakeFunctionResult * appropriate to recurse, but not anymore. (Even if there are more SRFs in
* for nested set-returning functions). * the function's inputs, their multipliers are accounted for separately.)
* *
* Note: keep this in sync with expression_returns_set() in nodes/nodeFuncs.c. * Note: keep this in sync with expression_returns_set() in nodes/nodeFuncs.c.
*/ */
double double
expression_returns_set_rows(Node *clause) expression_returns_set_rows(Node *clause)
{ {
double result = 1; if (clause == NULL)
return 1.0;
(void) expression_returns_set_rows_walker(clause, &result); if (IsA(clause, FuncExpr))
return clamp_row_est(result);
}
static bool
expression_returns_set_rows_walker(Node *node, double *count)
{
if (node == NULL)
return false;
if (IsA(node, FuncExpr))
{ {
FuncExpr *expr = (FuncExpr *) node; FuncExpr *expr = (FuncExpr *) clause;
if (expr->funcretset) if (expr->funcretset)
*count *= get_func_rows(expr->funcid); return clamp_row_est(get_func_rows(expr->funcid));
} }
if (IsA(node, OpExpr)) if (IsA(clause, OpExpr))
{ {
OpExpr *expr = (OpExpr *) node; OpExpr *expr = (OpExpr *) clause;
if (expr->opretset) if (expr->opretset)
{ {
set_opfuncid(expr); set_opfuncid(expr);
*count *= get_func_rows(expr->opfuncid); return clamp_row_est(get_func_rows(expr->opfuncid));
}
} }
/* Avoid recursion for some cases that can't return a set */
if (IsA(node, Aggref))
return false;
if (IsA(node, WindowFunc))
return false;
if (IsA(node, DistinctExpr))
return false;
if (IsA(node, NullIfExpr))
return false;
if (IsA(node, ScalarArrayOpExpr))
return false;
if (IsA(node, BoolExpr))
return false;
if (IsA(node, SubLink))
return false;
if (IsA(node, SubPlan))
return false;
if (IsA(node, AlternativeSubPlan))
return false;
if (IsA(node, ArrayExpr))
return false;
if (IsA(node, RowExpr))
return false;
if (IsA(node, RowCompareExpr))
return false;
if (IsA(node, CoalesceExpr))
return false;
if (IsA(node, MinMaxExpr))
return false;
if (IsA(node, XmlExpr))
return false;
return expression_tree_walker(node, expression_returns_set_rows_walker,
(void *) count);
}
/*
* tlist_returns_set_rows
* Estimate the number of rows returned by a set-returning targetlist.
* The result is 1 if there are no set-returning functions.
*
* Here, the result is the largest rowcount estimate of any of the tlist's
* expressions, not the product as you would get from naively applying
* expression_returns_set_rows() to the whole tlist. The behavior actually
* implemented by ExecTargetList produces a number of rows equal to the least
* common multiple of the expression rowcounts, so that the product would be
* a worst-case estimate that is typically not realistic. Taking the max as
* we do here is a best-case estimate that might not be realistic either,
* but it's probably closer for typical usages. We don't try to compute the
* actual LCM because we're working with very approximate estimates, so their
* LCM would be unduly noisy.
*/
double
tlist_returns_set_rows(List *tlist)
{
double result = 1;
ListCell *lc;
foreach(lc, tlist)
{
TargetEntry *tle = (TargetEntry *) lfirst(lc);
double colresult;
colresult = expression_returns_set_rows((Node *) tle->expr);
if (result < colresult)
result = colresult;
} }
return result; return 1.0;
} }
......
...@@ -2319,6 +2319,72 @@ apply_projection_to_path(PlannerInfo *root, ...@@ -2319,6 +2319,72 @@ apply_projection_to_path(PlannerInfo *root,
return path; return path;
} }
/*
* create_set_projection_path
* Creates a pathnode that represents performing a projection that
* includes set-returning functions.
*
* 'rel' is the parent relation associated with the result
* 'subpath' is the path representing the source of data
* 'target' is the PathTarget to be computed
*/
ProjectSetPath *
create_set_projection_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
PathTarget *target)
{
ProjectSetPath *pathnode = makeNode(ProjectSetPath);
double tlist_rows;
ListCell *lc;
pathnode->path.pathtype = T_ProjectSet;
pathnode->path.parent = rel;
pathnode->path.pathtarget = target;
/* For now, assume we are above any joins, so no parameterization */
pathnode->path.param_info = NULL;
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe &&
is_parallel_safe(root, (Node *) target->exprs);
pathnode->path.parallel_workers = subpath->parallel_workers;
/* Projection does not change the sort order XXX? */
pathnode->path.pathkeys = subpath->pathkeys;
pathnode->subpath = subpath;
/*
* Estimate number of rows produced by SRFs for each row of input; if
* there's more than one in this node, use the maximum.
*/
tlist_rows = 1;
foreach(lc, target->exprs)
{
Node *node = (Node *) lfirst(lc);
double itemrows;
itemrows = expression_returns_set_rows(node);
if (tlist_rows < itemrows)
tlist_rows = itemrows;
}
/*
* In addition to the cost of evaluating the tlist, charge cpu_tuple_cost
* per input row, and half of cpu_tuple_cost for each added output row.
* This is slightly bizarre maybe, but it's what 9.6 did; we may revisit
* this estimate later.
*/
pathnode->path.rows = subpath->rows * tlist_rows;
pathnode->path.startup_cost = subpath->startup_cost +
target->cost.startup;
pathnode->path.total_cost = subpath->total_cost +
target->cost.startup +
(cpu_tuple_cost + target->cost.per_tuple) * subpath->rows +
(pathnode->path.rows - subpath->rows) * cpu_tuple_cost / 2;
return pathnode;
}
/* /*
* create_sort_path * create_sort_path
* Creates a pathnode that represents performing an explicit sort. * Creates a pathnode that represents performing an explicit sort.
......
...@@ -16,9 +16,20 @@ ...@@ -16,9 +16,20 @@
#include "nodes/makefuncs.h" #include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h" #include "nodes/nodeFuncs.h"
#include "optimizer/cost.h"
#include "optimizer/tlist.h" #include "optimizer/tlist.h"
typedef struct
{
List *nextlevel_tlist;
bool nextlevel_contains_srfs;
} split_pathtarget_context;
static bool split_pathtarget_walker(Node *node,
split_pathtarget_context *context);
/***************************************************************************** /*****************************************************************************
* Target list creation and searching utilities * Target list creation and searching utilities
*****************************************************************************/ *****************************************************************************/
...@@ -759,3 +770,191 @@ apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target) ...@@ -759,3 +770,191 @@ apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target)
i++; i++;
} }
} }
/*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
* The executor can only handle set-returning functions that appear at the
* top level of the targetlist of a ProjectSet plan node. If we have any SRFs
* that are not at top level, we need to split up the evaluation into multiple
* plan levels in which each level satisfies this constraint. This function
* creates appropriate PathTarget(s) for each level.
*
* As an example, consider the tlist expression
* x + srf1(srf2(y + z))
* This expression should appear as-is in the top PathTarget, but below that
* we must have a PathTarget containing
* x, srf1(srf2(y + z))
* and below that, another PathTarget containing
* x, srf2(y + z)
* and below that, another PathTarget containing
* x, y, z
* When these tlists are processed by setrefs.c, subexpressions that match
* output expressions of the next lower tlist will be replaced by Vars,
* so that what the executor gets are tlists looking like
* Var1 + Var2
* Var1, srf1(Var2)
* Var1, srf2(Var2 + Var3)
* x, y, z
* which satisfy the desired property.
*
* In some cases, a SRF has already been evaluated in some previous plan level
* and we shouldn't expand it again (that is, what we see in the target is
* already meant as a reference to a lower subexpression). So, don't expand
* any tlist expressions that appear in input_target, if that's not NULL.
* In principle we might need to consider matching subexpressions to
* input_target, but for now it's not necessary because only ORDER BY and
* GROUP BY expressions are at issue and those will look the same at both
* plan levels.
*
* The outputs of this function are two parallel lists, one a list of
* PathTargets and the other an integer list of bool flags indicating
* whether the corresponding PathTarget contains any top-level SRFs.
* The lists are given in the order they'd need to be evaluated in, with
* the "lowest" PathTarget first. So the last list entry is always the
* originally given PathTarget, and any entries before it indicate evaluation
* levels that must be inserted below it. The first list entry must not
* contain any SRFs, since it will typically be attached to a plan node
* that cannot evaluate SRFs.
*
* Note: using a list for the flags may seem like overkill, since there
* are only a few possible patterns for which levels contain SRFs.
* But this representation decouples callers from that knowledge.
*/
void
split_pathtarget_at_srfs(PlannerInfo *root,
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs)
{
/* Initialize output lists to empty; we prepend to them within loop */
*targets = *targets_contain_srfs = NIL;
/* Loop to consider each level of PathTarget we need */
for (;;)
{
bool target_contains_srfs = false;
split_pathtarget_context context;
ListCell *lc;
context.nextlevel_tlist = NIL;
context.nextlevel_contains_srfs = false;
/*
* Scan the PathTarget looking for SRFs. Top-level SRFs are handled
* in this loop, ones lower down are found by split_pathtarget_walker.
*/
foreach(lc, target->exprs)
{
Node *node = (Node *) lfirst(lc);
/*
* A tlist item that is just a reference to an expression already
* computed in input_target need not be evaluated here, so just
* make sure it's included in the next PathTarget.
*/
if (input_target && list_member(input_target->exprs, node))
{
context.nextlevel_tlist = lappend(context.nextlevel_tlist, node);
continue;
}
/* Else, we need to compute this expression. */
if (IsA(node, FuncExpr) &&
((FuncExpr *) node)->funcretset)
{
/* Top-level SRF: it can be evaluated here */
target_contains_srfs = true;
/* Recursively examine SRF's inputs */
split_pathtarget_walker((Node *) ((FuncExpr *) node)->args,
&context);
}
else if (IsA(node, OpExpr) &&
((OpExpr *) node)->opretset)
{
/* Same as above, but for set-returning operator */
target_contains_srfs = true;
split_pathtarget_walker((Node *) ((OpExpr *) node)->args,
&context);
}
else
{
/* Not a top-level SRF, so recursively examine expression */
split_pathtarget_walker(node, &context);
}
}
/*
* Prepend current target and associated flag to output lists.
*/
*targets = lcons(target, *targets);
*targets_contain_srfs = lcons_int(target_contains_srfs,
*targets_contain_srfs);
/*
* Done if we found no SRFs anywhere in this target; the tentative
* tlist we built for the next level can be discarded.
*/
if (!target_contains_srfs && !context.nextlevel_contains_srfs)
break;
/*
* Else build the next PathTarget down, and loop back to process it.
* Copy the subexpressions to make sure PathTargets don't share
* substructure (might be unnecessary, but be safe); and drop any
* duplicate entries in the sub-targetlist.
*/
target = create_empty_pathtarget();
add_new_columns_to_pathtarget(target,
(List *) copyObject(context.nextlevel_tlist));
set_pathtarget_cost_width(root, target);
}
}
/* Recursively examine expressions for split_pathtarget_at_srfs */
static bool
split_pathtarget_walker(Node *node, split_pathtarget_context *context)
{
if (node == NULL)
return false;
if (IsA(node, Var) ||
IsA(node, PlaceHolderVar) ||
IsA(node, Aggref) ||
IsA(node, GroupingFunc) ||
IsA(node, WindowFunc))
{
/*
* Pass these items down to the child plan level for evaluation.
*
* We assume that these constructs cannot contain any SRFs (if one
* does, there will be an executor failure from a misplaced SRF).
*/
context->nextlevel_tlist = lappend(context->nextlevel_tlist, node);
/* Having done that, we need not examine their sub-structure */
return false;
}
else if ((IsA(node, FuncExpr) &&
((FuncExpr *) node)->funcretset) ||
(IsA(node, OpExpr) &&
((OpExpr *) node)->opretset))
{
/*
* Pass SRFs down to the child plan level for evaluation, and mark
* that it contains SRFs. (We are not at top level of our own tlist,
* else this would have been picked up by split_pathtarget_at_srfs.)
*/
context->nextlevel_tlist = lappend(context->nextlevel_tlist, node);
context->nextlevel_contains_srfs = true;
/* Inputs to the SRF need not be considered here, so we're done */
return false;
}
/*
* Otherwise, the node is evaluatable within the current PathTarget, so
* recurse to examine its inputs.
*/
return expression_tree_walker(node, split_pathtarget_walker,
(void *) context);
}
...@@ -253,6 +253,10 @@ extern Tuplestorestate *ExecMakeTableFunctionResult(ExprState *funcexpr, ...@@ -253,6 +253,10 @@ extern Tuplestorestate *ExecMakeTableFunctionResult(ExprState *funcexpr,
MemoryContext argContext, MemoryContext argContext,
TupleDesc expectedDesc, TupleDesc expectedDesc,
bool randomAccess); bool randomAccess);
extern Datum ExecMakeFunctionResultSet(FuncExprState *fcache,
ExprContext *econtext,
bool *isNull,
ExprDoneCond *isDone);
extern Datum ExecEvalExprSwitchContext(ExprState *expression, ExprContext *econtext, extern Datum ExecEvalExprSwitchContext(ExprState *expression, ExprContext *econtext,
bool *isNull, ExprDoneCond *isDone); bool *isNull, ExprDoneCond *isDone);
extern ExprState *ExecInitExpr(Expr *node, PlanState *parent); extern ExprState *ExecInitExpr(Expr *node, PlanState *parent);
......
/*-------------------------------------------------------------------------
*
* nodeProjectSet.h
*
*
*
* Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* src/include/executor/nodeProjectSet.h
*
*-------------------------------------------------------------------------
*/
#ifndef NODEPROJECTSET_H
#define NODEPROJECTSET_H
#include "nodes/execnodes.h"
extern ProjectSetState *ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags);
extern TupleTableSlot *ExecProjectSet(ProjectSetState *node);
extern void ExecEndProjectSet(ProjectSetState *node);
extern void ExecReScanProjectSet(ProjectSetState *node);
#endif /* NODEPROJECTSET_H */
...@@ -696,7 +696,7 @@ typedef struct FuncExprState ...@@ -696,7 +696,7 @@ typedef struct FuncExprState
/* /*
* Function manager's lookup info for the target function. If func.fn_oid * Function manager's lookup info for the target function. If func.fn_oid
* is InvalidOid, we haven't initialized it yet (nor any of the following * is InvalidOid, we haven't initialized it yet (nor any of the following
* fields). * fields, except funcReturnsSet).
*/ */
FmgrInfo func; FmgrInfo func;
...@@ -716,6 +716,12 @@ typedef struct FuncExprState ...@@ -716,6 +716,12 @@ typedef struct FuncExprState
bool funcReturnsTuple; /* valid when funcResultDesc isn't bool funcReturnsTuple; /* valid when funcResultDesc isn't
* NULL */ * NULL */
/*
* Remember whether the function is declared to return a set. This is set
* by ExecInitExpr, and is valid even before the FmgrInfo is set up.
*/
bool funcReturnsSet;
/* /*
* setArgsValid is true when we are evaluating a set-returning function * setArgsValid is true when we are evaluating a set-returning function
* that uses value-per-call mode and we are in the middle of a call * that uses value-per-call mode and we are in the middle of a call
...@@ -1129,6 +1135,18 @@ typedef struct ResultState ...@@ -1129,6 +1135,18 @@ typedef struct ResultState
bool rs_checkqual; /* do we need to check the qual? */ bool rs_checkqual; /* do we need to check the qual? */
} ResultState; } ResultState;
/* ----------------
* ProjectSetState information
* ----------------
*/
typedef struct ProjectSetState
{
PlanState ps; /* its first field is NodeTag */
ExprDoneCond *elemdone; /* array of per-SRF is-done states */
int nelems; /* length of elemdone[] array */
bool pending_srf_tuples; /* still evaluating srfs in tlist? */
} ProjectSetState;
/* ---------------- /* ----------------
* ModifyTableState information * ModifyTableState information
* ---------------- * ----------------
......
...@@ -43,6 +43,7 @@ typedef enum NodeTag ...@@ -43,6 +43,7 @@ typedef enum NodeTag
*/ */
T_Plan, T_Plan,
T_Result, T_Result,
T_ProjectSet,
T_ModifyTable, T_ModifyTable,
T_Append, T_Append,
T_MergeAppend, T_MergeAppend,
...@@ -91,6 +92,7 @@ typedef enum NodeTag ...@@ -91,6 +92,7 @@ typedef enum NodeTag
*/ */
T_PlanState, T_PlanState,
T_ResultState, T_ResultState,
T_ProjectSetState,
T_ModifyTableState, T_ModifyTableState,
T_AppendState, T_AppendState,
T_MergeAppendState, T_MergeAppendState,
...@@ -245,6 +247,7 @@ typedef enum NodeTag ...@@ -245,6 +247,7 @@ typedef enum NodeTag
T_UniquePath, T_UniquePath,
T_GatherPath, T_GatherPath,
T_ProjectionPath, T_ProjectionPath,
T_ProjectSetPath,
T_SortPath, T_SortPath,
T_GroupPath, T_GroupPath,
T_UpperUniquePath, T_UpperUniquePath,
......
...@@ -176,6 +176,17 @@ typedef struct Result ...@@ -176,6 +176,17 @@ typedef struct Result
Node *resconstantqual; Node *resconstantqual;
} Result; } Result;
/* ----------------
* ProjectSet node -
* Apply a projection that includes set-returning functions to the
* output tuples of the outer plan.
* ----------------
*/
typedef struct ProjectSet
{
Plan plan;
} ProjectSet;
/* ---------------- /* ----------------
* ModifyTable node - * ModifyTable node -
* Apply rows produced by subplan(s) to result table(s), * Apply rows produced by subplan(s) to result table(s),
......
...@@ -1304,6 +1304,17 @@ typedef struct ProjectionPath ...@@ -1304,6 +1304,17 @@ typedef struct ProjectionPath
bool dummypp; /* true if no separate Result is needed */ bool dummypp; /* true if no separate Result is needed */
} ProjectionPath; } ProjectionPath;
/*
* ProjectSetPath represents evaluation of a targetlist that includes
* set-returning function(s), which will need to be implemented by a
* ProjectSet plan node.
*/
typedef struct ProjectSetPath
{
Path path;
Path *subpath; /* path representing input source */
} ProjectSetPath;
/* /*
* SortPath represents an explicit sort step * SortPath represents an explicit sort step
* *
......
...@@ -54,7 +54,6 @@ extern bool contain_window_function(Node *clause); ...@@ -54,7 +54,6 @@ extern bool contain_window_function(Node *clause);
extern WindowFuncLists *find_window_functions(Node *clause, Index maxWinRef); extern WindowFuncLists *find_window_functions(Node *clause, Index maxWinRef);
extern double expression_returns_set_rows(Node *clause); extern double expression_returns_set_rows(Node *clause);
extern double tlist_returns_set_rows(List *tlist);
extern bool contain_subplans(Node *clause); extern bool contain_subplans(Node *clause);
......
...@@ -144,6 +144,10 @@ extern Path *apply_projection_to_path(PlannerInfo *root, ...@@ -144,6 +144,10 @@ extern Path *apply_projection_to_path(PlannerInfo *root,
RelOptInfo *rel, RelOptInfo *rel,
Path *path, Path *path,
PathTarget *target); PathTarget *target);
extern ProjectSetPath *create_set_projection_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
PathTarget *target);
extern SortPath *create_sort_path(PlannerInfo *root, extern SortPath *create_sort_path(PlannerInfo *root,
RelOptInfo *rel, RelOptInfo *rel,
Path *subpath, Path *subpath,
......
...@@ -61,6 +61,9 @@ extern void add_column_to_pathtarget(PathTarget *target, ...@@ -61,6 +61,9 @@ extern void add_column_to_pathtarget(PathTarget *target,
extern void add_new_column_to_pathtarget(PathTarget *target, Expr *expr); extern void add_new_column_to_pathtarget(PathTarget *target, Expr *expr);
extern void add_new_columns_to_pathtarget(PathTarget *target, List *exprs); extern void add_new_columns_to_pathtarget(PathTarget *target, List *exprs);
extern void apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target); extern void apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target);
extern void split_pathtarget_at_srfs(PlannerInfo *root,
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
/* Convenience macro to get a PathTarget with valid cost/width fields */ /* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \ #define create_pathtarget(root, tlist) \
......
...@@ -822,8 +822,9 @@ explain (costs off) ...@@ -822,8 +822,9 @@ explain (costs off)
-> Limit -> Limit
-> Index Only Scan Backward using tenk1_unique2 on tenk1 -> Index Only Scan Backward using tenk1_unique2 on tenk1
Index Cond: (unique2 IS NOT NULL) Index Cond: (unique2 IS NOT NULL)
-> ProjectSet
-> Result -> Result
(7 rows) (8 rows)
select max(unique2), generate_series(1,3) as g from tenk1 order by g desc; select max(unique2), generate_series(1,3) as g from tenk1 order by g desc;
max | g max | g
......
...@@ -209,12 +209,14 @@ explain (verbose, costs off) ...@@ -209,12 +209,14 @@ explain (verbose, costs off)
select unique1, unique2, generate_series(1,10) select unique1, unique2, generate_series(1,10)
from tenk1 order by unique2 limit 7; from tenk1 order by unique2 limit 7;
QUERY PLAN QUERY PLAN
---------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit Limit
Output: unique1, unique2, (generate_series(1, 10)) Output: unique1, unique2, (generate_series(1, 10))
-> Index Scan using tenk1_unique2 on public.tenk1 -> ProjectSet
Output: unique1, unique2, generate_series(1, 10) Output: unique1, unique2, generate_series(1, 10)
(4 rows) -> Index Scan using tenk1_unique2 on public.tenk1
Output: unique1, unique2, two, four, ten, twenty, hundred, thousand, twothousand, fivethous, tenthous, odd, even, stringu1, stringu2, string4
(6 rows)
select unique1, unique2, generate_series(1,10) select unique1, unique2, generate_series(1,10)
from tenk1 order by unique2 limit 7; from tenk1 order by unique2 limit 7;
...@@ -236,7 +238,7 @@ select unique1, unique2, generate_series(1,10) ...@@ -236,7 +238,7 @@ select unique1, unique2, generate_series(1,10)
-------------------------------------------------------------------- --------------------------------------------------------------------
Limit Limit
Output: unique1, unique2, (generate_series(1, 10)), tenthous Output: unique1, unique2, (generate_series(1, 10)), tenthous
-> Result -> ProjectSet
Output: unique1, unique2, generate_series(1, 10), tenthous Output: unique1, unique2, generate_series(1, 10), tenthous
-> Sort -> Sort
Output: unique1, unique2, tenthous Output: unique1, unique2, tenthous
...@@ -263,9 +265,10 @@ explain (verbose, costs off) ...@@ -263,9 +265,10 @@ explain (verbose, costs off)
select generate_series(0,2) as s1, generate_series((random()*.1)::int,2) as s2; select generate_series(0,2) as s1, generate_series((random()*.1)::int,2) as s2;
QUERY PLAN QUERY PLAN
------------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------------
Result ProjectSet
Output: generate_series(0, 2), generate_series(((random() * '0.1'::double precision))::integer, 2) Output: generate_series(0, 2), generate_series(((random() * '0.1'::double precision))::integer, 2)
(2 rows) -> Result
(3 rows)
select generate_series(0,2) as s1, generate_series((random()*.1)::int,2) as s2; select generate_series(0,2) as s1, generate_series((random()*.1)::int,2) as s2;
s1 | s2 s1 | s2
...@@ -283,9 +286,10 @@ order by s2 desc; ...@@ -283,9 +286,10 @@ order by s2 desc;
Sort Sort
Output: (generate_series(0, 2)), (generate_series(((random() * '0.1'::double precision))::integer, 2)) Output: (generate_series(0, 2)), (generate_series(((random() * '0.1'::double precision))::integer, 2))
Sort Key: (generate_series(((random() * '0.1'::double precision))::integer, 2)) DESC Sort Key: (generate_series(((random() * '0.1'::double precision))::integer, 2)) DESC
-> Result -> ProjectSet
Output: generate_series(0, 2), generate_series(((random() * '0.1'::double precision))::integer, 2) Output: generate_series(0, 2), generate_series(((random() * '0.1'::double precision))::integer, 2)
(5 rows) -> Result
(6 rows)
select generate_series(0,2) as s1, generate_series((random()*.1)::int,2) as s2 select generate_series(0,2) as s1, generate_series((random()*.1)::int,2) as s2
order by s2 desc; order by s2 desc;
......
...@@ -1321,16 +1321,18 @@ rollback; ...@@ -1321,16 +1321,18 @@ rollback;
begin; begin;
explain (costs off) declare c2 cursor for select generate_series(1,3) as g; explain (costs off) declare c2 cursor for select generate_series(1,3) as g;
QUERY PLAN QUERY PLAN
------------ --------------
Result ProjectSet
(1 row) -> Result
(2 rows)
explain (costs off) declare c2 scroll cursor for select generate_series(1,3) as g; explain (costs off) declare c2 scroll cursor for select generate_series(1,3) as g;
QUERY PLAN QUERY PLAN
-------------- --------------------
Materialize Materialize
-> ProjectSet
-> Result -> Result
(2 rows) (3 rows)
declare c2 scroll cursor for select generate_series(1,3) as g; declare c2 scroll cursor for select generate_series(1,3) as g;
fetch all in c2; fetch all in c2;
......
...@@ -1996,11 +1996,9 @@ SELECT *, ...@@ -1996,11 +1996,9 @@ SELECT *,
FROM FROM
(VALUES (1,''), (2,'0000000049404'), (3,'FROM 10000000876')) v(id, str); (VALUES (1,''), (2,'0000000049404'), (3,'FROM 10000000876')) v(id, str);
id | str | lower id | str | lower
----+------------------+------------------ ----+---------------+-------
1 | |
2 | 0000000049404 | 49404 2 | 0000000049404 | 49404
3 | FROM 10000000876 | from 10000000876 (1 row)
(3 rows)
-- check whole-row-Var handling in nested lateral functions (bug #11703) -- check whole-row-Var handling in nested lateral functions (bug #11703)
create function extractq2(t int8_tbl) returns int8 as $$ create function extractq2(t int8_tbl) returns int8 as $$
......
...@@ -808,23 +808,27 @@ explain (verbose, costs off) ...@@ -808,23 +808,27 @@ explain (verbose, costs off)
select * from int4_tbl o where (f1, f1) in select * from int4_tbl o where (f1, f1) in
(select f1, generate_series(1,2) / 10 g from int4_tbl i group by f1); (select f1, generate_series(1,2) / 10 g from int4_tbl i group by f1);
QUERY PLAN QUERY PLAN
---------------------------------------------------------------- -------------------------------------------------------------------
Hash Semi Join Nested Loop Semi Join
Output: o.f1 Output: o.f1
Hash Cond: (o.f1 = "ANY_subquery".f1) Join Filter: (o.f1 = "ANY_subquery".f1)
-> Seq Scan on public.int4_tbl o -> Seq Scan on public.int4_tbl o
Output: o.f1 Output: o.f1
-> Hash -> Materialize
Output: "ANY_subquery".f1, "ANY_subquery".g Output: "ANY_subquery".f1, "ANY_subquery".g
-> Subquery Scan on "ANY_subquery" -> Subquery Scan on "ANY_subquery"
Output: "ANY_subquery".f1, "ANY_subquery".g Output: "ANY_subquery".f1, "ANY_subquery".g
Filter: ("ANY_subquery".f1 = "ANY_subquery".g) Filter: ("ANY_subquery".f1 = "ANY_subquery".g)
-> Result
Output: i.f1, ((generate_series(1, 2)) / 10)
-> ProjectSet
Output: i.f1, generate_series(1, 2)
-> HashAggregate -> HashAggregate
Output: i.f1, (generate_series(1, 2) / 10) Output: i.f1
Group Key: i.f1 Group Key: i.f1
-> Seq Scan on public.int4_tbl i -> Seq Scan on public.int4_tbl i
Output: i.f1 Output: i.f1
(15 rows) (19 rows)
select * from int4_tbl o where (f1, f1) in select * from int4_tbl o where (f1, f1) in
(select f1, generate_series(1,2) / 10 g from int4_tbl i group by f1); (select f1, generate_series(1,2) / 10 g from int4_tbl i group by f1);
...@@ -899,9 +903,10 @@ select * from ...@@ -899,9 +903,10 @@ select * from
Subquery Scan on ss Subquery Scan on ss
Output: x, u Output: x, u
Filter: tattle(ss.x, 8) Filter: tattle(ss.x, 8)
-> Result -> ProjectSet
Output: 9, unnest('{1,2,3,11,12,13}'::integer[]) Output: 9, unnest('{1,2,3,11,12,13}'::integer[])
(5 rows) -> Result
(6 rows)
select * from select * from
(select 9 as x, unnest(array[1,2,3,11,12,13]) as u) ss (select 9 as x, unnest(array[1,2,3,11,12,13]) as u) ss
...@@ -930,10 +935,11 @@ select * from ...@@ -930,10 +935,11 @@ select * from
where tattle(x, 8); where tattle(x, 8);
QUERY PLAN QUERY PLAN
---------------------------------------------------- ----------------------------------------------------
Result ProjectSet
Output: 9, unnest('{1,2,3,11,12,13}'::integer[]) Output: 9, unnest('{1,2,3,11,12,13}'::integer[])
-> Result
One-Time Filter: tattle(9, 8) One-Time Filter: tattle(9, 8)
(3 rows) (4 rows)
select * from select * from
(select 9 as x, unnest(array[1,2,3,11,12,13]) as u) ss (select 9 as x, unnest(array[1,2,3,11,12,13]) as u) ss
...@@ -959,9 +965,10 @@ select * from ...@@ -959,9 +965,10 @@ select * from
Subquery Scan on ss Subquery Scan on ss
Output: x, u Output: x, u
Filter: tattle(ss.x, ss.u) Filter: tattle(ss.x, ss.u)
-> Result -> ProjectSet
Output: 9, unnest('{1,2,3,11,12,13}'::integer[]) Output: 9, unnest('{1,2,3,11,12,13}'::integer[])
(5 rows) -> Result
(6 rows)
select * from select * from
(select 9 as x, unnest(array[1,2,3,11,12,13]) as u) ss (select 9 as x, unnest(array[1,2,3,11,12,13]) as u) ss
......
...@@ -25,8 +25,8 @@ SELECT generate_series(1, 2), generate_series(1,4); ...@@ -25,8 +25,8 @@ SELECT generate_series(1, 2), generate_series(1,4);
-----------------+----------------- -----------------+-----------------
1 | 1 1 | 1
2 | 2 2 | 2
1 | 3 | 3
2 | 4 | 4
(4 rows) (4 rows)
-- srf, with SRF argument -- srf, with SRF argument
...@@ -43,7 +43,16 @@ SELECT generate_series(1, generate_series(1, 3)); ...@@ -43,7 +43,16 @@ SELECT generate_series(1, generate_series(1, 3));
-- srf, with two SRF arguments -- srf, with two SRF arguments
SELECT generate_series(generate_series(1,3), generate_series(2, 4)); SELECT generate_series(generate_series(1,3), generate_series(2, 4));
ERROR: functions and operators can take at most one set argument generate_series
-----------------
1
2
2
3
3
4
(6 rows)
CREATE TABLE few(id int, dataa text, datab text); CREATE TABLE few(id int, dataa text, datab text);
INSERT INTO few VALUES(1, 'a', 'foo'),(2, 'a', 'bar'),(3, 'b', 'bar'); INSERT INTO few VALUES(1, 'a', 'foo'),(2, 'a', 'bar'),(3, 'b', 'bar');
-- SRF output order of sorting is maintained, if SRF is not referenced -- SRF output order of sorting is maintained, if SRF is not referenced
...@@ -118,15 +127,15 @@ SELECT few.dataa, count(*), min(id), max(id), unnest('{1,1,3}'::int[]) FROM few ...@@ -118,15 +127,15 @@ SELECT few.dataa, count(*), min(id), max(id), unnest('{1,1,3}'::int[]) FROM few
SELECT few.dataa, count(*), min(id), max(id), unnest('{1,1,3}'::int[]) FROM few WHERE few.id = 1 GROUP BY few.dataa, unnest('{1,1,3}'::int[]); SELECT few.dataa, count(*), min(id), max(id), unnest('{1,1,3}'::int[]) FROM few WHERE few.id = 1 GROUP BY few.dataa, unnest('{1,1,3}'::int[]);
dataa | count | min | max | unnest dataa | count | min | max | unnest
-------+-------+-----+-----+-------- -------+-------+-----+-----+--------
a | 2 | 1 | 1 | 1
a | 1 | 1 | 1 | 3 a | 1 | 1 | 1 | 3
a | 2 | 1 | 1 | 1
(2 rows) (2 rows)
SELECT few.dataa, count(*), min(id), max(id), unnest('{1,1,3}'::int[]) FROM few WHERE few.id = 1 GROUP BY few.dataa, 5; SELECT few.dataa, count(*), min(id), max(id), unnest('{1,1,3}'::int[]) FROM few WHERE few.id = 1 GROUP BY few.dataa, 5;
dataa | count | min | max | unnest dataa | count | min | max | unnest
-------+-------+-----+-----+-------- -------+-------+-----+-----+--------
a | 2 | 1 | 1 | 1
a | 1 | 1 | 1 | 3 a | 1 | 1 | 1 | 3
a | 2 | 1 | 1 | 1
(2 rows) (2 rows)
-- check HAVING works when GROUP BY does [not] reference SRF output -- check HAVING works when GROUP BY does [not] reference SRF output
......
...@@ -636,9 +636,10 @@ ORDER BY x; ...@@ -636,9 +636,10 @@ ORDER BY x;
-> HashAggregate -> HashAggregate
Group Key: (1), (generate_series(1, 10)) Group Key: (1), (generate_series(1, 10))
-> Append -> Append
-> ProjectSet
-> Result -> Result
-> Result -> Result
(9 rows) (10 rows)
SELECT * FROM SELECT * FROM
(SELECT 1 AS t, generate_series(1,10) AS x (SELECT 1 AS t, generate_series(1,10) AS x
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment