Commit 6719b238 authored by Tom Lane's avatar Tom Lane

Rearrange execution of PARAM_EXTERN Params for plpgsql's benefit.

This patch does three interrelated things:

* Create a new expression execution step type EEOP_PARAM_CALLBACK
and add the infrastructure needed for add-on modules to generate that.
As discussed, the best control mechanism for that seems to be to add
another hook function to ParamListInfo, which will be called by
ExecInitExpr if it's supplied and a PARAM_EXTERN Param is found.
For stand-alone expressions, we add a new entry point to allow the
ParamListInfo to be specified directly, since it can't be retrieved
from the parent plan node's EState.

* Redesign the API for the ParamListInfo paramFetch hook so that the
ParamExternData array can be entirely virtual.  This also lets us get rid
of ParamListInfo.paramMask, instead leaving it to the paramFetch hook to
decide which param IDs should be accessible or not.  plpgsql_param_fetch
was already doing the identical masking check, so having callers do it too
seemed redundant.  While I was at it, I added a "speculative" flag to
paramFetch that the planner can specify as TRUE to avoid unwanted failures.
This solves an ancient problem for plpgsql that it couldn't provide values
of non-DTYPE_VAR variables to the planner for fear of triggering premature
"record not assigned yet" or "field not found" errors during planning.

* Rework plpgsql to get rid of the need for "unshared" parameter lists,
by dint of turning the single ParamListInfo per estate into a nearly
read-only data structure that doesn't instantiate any per-variable data.
Instead, the paramFetch hook controls access to per-variable data and can
make the right decisions on the fly, replacing the cases that we used to
need multiple ParamListInfos for.  This might perhaps have been a
performance loss on its own, but by using a paramCompile hook we can
bypass plpgsql_param_fetch entirely during normal query execution.
(It's now only called when, eg, we copy the ParamListInfo into a cursor
portal.  copyParamList() or SerializeParamList() effectively instantiate
the virtual parameter array as a simple physical array without a
paramFetch hook, which is what we want in those cases.)  This allows
reverting most of commit 6c82d8d1, though I kept the cosmetic
code-consolidation aspects of that (eg the assign_simple_var function).

Performance testing shows this to be at worst a break-even change,
and it can provide wins ranging up to 20% in test cases involving
accesses to fields of "record" variables.  The fact that values of
such variables can now be exposed to the planner might produce wins
in some situations, too, but I've not pursued that angle.

In passing, remove the "parent" pointer from the arguments to
ExecInitExprRec and related functions, instead storing that pointer in a
transient field in ExprState.  The ParamListInfo pointer for a stand-alone
expression is handled the same way; we'd otherwise have had to add
yet another recursively-passed-down argument in expression compilation.

Discussion: https://postgr.es/m/32589.1513706441@sss.pgh.pa.us
parent 8a0596cb
......@@ -399,10 +399,11 @@ EvaluateParams(PreparedStatement *pstmt, List *params,
/* we have static list of params, so no hooks needed */
paramLI->paramFetch = NULL;
paramLI->paramFetchArg = NULL;
paramLI->paramCompile = NULL;
paramLI->paramCompileArg = NULL;
paramLI->parserSetup = NULL;
paramLI->parserSetupArg = NULL;
paramLI->numParams = num_params;
paramLI->paramMask = NULL;
i = 0;
foreach(l, exprstates)
......
......@@ -216,11 +216,14 @@ fetch_cursor_param_value(ExprContext *econtext, int paramId)
if (paramInfo &&
paramId > 0 && paramId <= paramInfo->numParams)
{
ParamExternData *prm = &paramInfo->params[paramId - 1];
ParamExternData *prm;
ParamExternData prmdata;
/* give hook a chance in case parameter is dynamic */
if (!OidIsValid(prm->ptype) && paramInfo->paramFetch != NULL)
paramInfo->paramFetch(paramInfo, paramId);
if (paramInfo->paramFetch != NULL)
prm = paramInfo->paramFetch(paramInfo, paramId, false, &prmdata);
else
prm = &paramInfo->params[paramId - 1];
if (OidIsValid(prm->ptype) && !prm->isnull)
{
......
This diff is collapsed.
......@@ -335,6 +335,7 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
&&CASE_EEOP_BOOLTEST_IS_NOT_FALSE,
&&CASE_EEOP_PARAM_EXEC,
&&CASE_EEOP_PARAM_EXTERN,
&&CASE_EEOP_PARAM_CALLBACK,
&&CASE_EEOP_CASE_TESTVAL,
&&CASE_EEOP_MAKE_READONLY,
&&CASE_EEOP_IOCOERCE,
......@@ -1047,6 +1048,13 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
EEO_NEXT();
}
EEO_CASE(EEOP_PARAM_CALLBACK)
{
/* allow an extension module to supply a PARAM_EXTERN value */
op->d.cparam.paramfunc(state, op, econtext);
EEO_NEXT();
}
EEO_CASE(EEOP_CASE_TESTVAL)
{
/*
......@@ -1967,11 +1975,14 @@ ExecEvalParamExtern(ExprState *state, ExprEvalStep *op, ExprContext *econtext)
if (likely(paramInfo &&
paramId > 0 && paramId <= paramInfo->numParams))
{
ParamExternData *prm = &paramInfo->params[paramId - 1];
ParamExternData *prm;
ParamExternData prmdata;
/* give hook a chance in case parameter is dynamic */
if (!OidIsValid(prm->ptype) && paramInfo->paramFetch != NULL)
paramInfo->paramFetch(paramInfo, paramId);
if (paramInfo->paramFetch != NULL)
prm = paramInfo->paramFetch(paramInfo, paramId, false, &prmdata);
else
prm = &paramInfo->params[paramId - 1];
if (likely(OidIsValid(prm->ptype)))
{
......
......@@ -914,10 +914,11 @@ postquel_sub_params(SQLFunctionCachePtr fcache,
/* we have static list of params, so no hooks needed */
paramLI->paramFetch = NULL;
paramLI->paramFetchArg = NULL;
paramLI->paramCompile = NULL;
paramLI->paramCompileArg = NULL;
paramLI->parserSetup = NULL;
paramLI->parserSetupArg = NULL;
paramLI->numParams = nargs;
paramLI->paramMask = NULL;
fcache->paramLI = paramLI;
}
else
......
......@@ -2259,10 +2259,11 @@ _SPI_convert_params(int nargs, Oid *argtypes,
/* we have static list of params, so no hooks needed */
paramLI->paramFetch = NULL;
paramLI->paramFetchArg = NULL;
paramLI->paramCompile = NULL;
paramLI->paramCompileArg = NULL;
paramLI->parserSetup = NULL;
paramLI->parserSetupArg = NULL;
paramLI->numParams = nargs;
paramLI->paramMask = NULL;
for (i = 0; i < nargs; i++)
{
......
......@@ -48,32 +48,25 @@ copyParamList(ParamListInfo from)
retval = (ParamListInfo) palloc(size);
retval->paramFetch = NULL;
retval->paramFetchArg = NULL;
retval->paramCompile = NULL;
retval->paramCompileArg = NULL;
retval->parserSetup = NULL;
retval->parserSetupArg = NULL;
retval->numParams = from->numParams;
retval->paramMask = NULL;
for (i = 0; i < from->numParams; i++)
{
ParamExternData *oprm = &from->params[i];
ParamExternData *oprm;
ParamExternData *nprm = &retval->params[i];
ParamExternData prmdata;
int16 typLen;
bool typByVal;
/* Ignore parameters we don't need, to save cycles and space. */
if (from->paramMask != NULL &&
!bms_is_member(i, from->paramMask))
{
nprm->value = (Datum) 0;
nprm->isnull = true;
nprm->pflags = 0;
nprm->ptype = InvalidOid;
continue;
}
/* give hook a chance in case parameter is dynamic */
if (!OidIsValid(oprm->ptype) && from->paramFetch != NULL)
from->paramFetch(from, i + 1);
if (from->paramFetch != NULL)
oprm = from->paramFetch(from, i + 1, false, &prmdata);
else
oprm = &from->params[i];
/* flat-copy the parameter info */
*nprm = *oprm;
......@@ -102,22 +95,19 @@ EstimateParamListSpace(ParamListInfo paramLI)
for (i = 0; i < paramLI->numParams; i++)
{
ParamExternData *prm = &paramLI->params[i];
ParamExternData *prm;
ParamExternData prmdata;
Oid typeOid;
int16 typLen;
bool typByVal;
/* Ignore parameters we don't need, to save cycles and space. */
if (paramLI->paramMask != NULL &&
!bms_is_member(i, paramLI->paramMask))
typeOid = InvalidOid;
/* give hook a chance in case parameter is dynamic */
if (paramLI->paramFetch != NULL)
prm = paramLI->paramFetch(paramLI, i + 1, false, &prmdata);
else
{
/* give hook a chance in case parameter is dynamic */
if (!OidIsValid(prm->ptype) && paramLI->paramFetch != NULL)
paramLI->paramFetch(paramLI, i + 1);
typeOid = prm->ptype;
}
prm = &paramLI->params[i];
typeOid = prm->ptype;
sz = add_size(sz, sizeof(Oid)); /* space for type OID */
sz = add_size(sz, sizeof(uint16)); /* space for pflags */
......@@ -171,22 +161,19 @@ SerializeParamList(ParamListInfo paramLI, char **start_address)
/* Write each parameter in turn. */
for (i = 0; i < nparams; i++)
{
ParamExternData *prm = &paramLI->params[i];
ParamExternData *prm;
ParamExternData prmdata;
Oid typeOid;
int16 typLen;
bool typByVal;
/* Ignore parameters we don't need, to save cycles and space. */
if (paramLI->paramMask != NULL &&
!bms_is_member(i, paramLI->paramMask))
typeOid = InvalidOid;
/* give hook a chance in case parameter is dynamic */
if (paramLI->paramFetch != NULL)
prm = paramLI->paramFetch(paramLI, i + 1, false, &prmdata);
else
{
/* give hook a chance in case parameter is dynamic */
if (!OidIsValid(prm->ptype) && paramLI->paramFetch != NULL)
paramLI->paramFetch(paramLI, i + 1);
typeOid = prm->ptype;
}
prm = &paramLI->params[i];
typeOid = prm->ptype;
/* Write type OID. */
memcpy(*start_address, &typeOid, sizeof(Oid));
......@@ -237,10 +224,11 @@ RestoreParamList(char **start_address)
paramLI = (ParamListInfo) palloc(size);
paramLI->paramFetch = NULL;
paramLI->paramFetchArg = NULL;
paramLI->paramCompile = NULL;
paramLI->paramCompileArg = NULL;
paramLI->parserSetup = NULL;
paramLI->parserSetupArg = NULL;
paramLI->numParams = nparams;
paramLI->paramMask = NULL;
for (i = 0; i < nparams; i++)
{
......
......@@ -2513,14 +2513,27 @@ eval_const_expressions_mutator(Node *node,
case T_Param:
{
Param *param = (Param *) node;
ParamListInfo paramLI = context->boundParams;
/* Look to see if we've been given a value for this Param */
if (param->paramkind == PARAM_EXTERN &&
context->boundParams != NULL &&
paramLI != NULL &&
param->paramid > 0 &&
param->paramid <= context->boundParams->numParams)
param->paramid <= paramLI->numParams)
{
ParamExternData *prm = &context->boundParams->params[param->paramid - 1];
ParamExternData *prm;
ParamExternData prmdata;
/*
* Give hook a chance in case parameter is dynamic. Tell
* it that this fetch is speculative, so it should avoid
* erroring out if parameter is unavailable.
*/
if (paramLI->paramFetch != NULL)
prm = paramLI->paramFetch(paramLI, param->paramid,
true, &prmdata);
else
prm = &paramLI->params[param->paramid - 1];
if (OidIsValid(prm->ptype))
{
......
......@@ -1646,10 +1646,11 @@ exec_bind_message(StringInfo input_message)
/* we have static list of params, so no hooks needed */
params->paramFetch = NULL;
params->paramFetchArg = NULL;
params->paramCompile = NULL;
params->paramCompileArg = NULL;
params->parserSetup = NULL;
params->parserSetupArg = NULL;
params->numParams = numParams;
params->paramMask = NULL;
for (paramno = 0; paramno < numParams; paramno++)
{
......@@ -2211,6 +2212,9 @@ errdetail_params(ParamListInfo params)
MemoryContext oldcontext;
int paramno;
/* This code doesn't support dynamic param lists */
Assert(params->paramFetch == NULL);
/* Make sure any trash is generated in MessageContext */
oldcontext = MemoryContextSwitchTo(MessageContext);
......
......@@ -16,7 +16,8 @@
#include "nodes/execnodes.h"
/* forward reference to avoid circularity */
/* forward references to avoid circularity */
struct ExprEvalStep;
struct ArrayRefState;
/* Bits in ExprState->flags (see also execnodes.h for public flag bits): */
......@@ -25,6 +26,11 @@ struct ArrayRefState;
/* jump-threading is in use */
#define EEO_FLAG_DIRECT_THREADED (1 << 2)
/* Typical API for out-of-line evaluation subroutines */
typedef void (*ExecEvalSubroutine) (ExprState *state,
struct ExprEvalStep *op,
ExprContext *econtext);
/*
* Discriminator for ExprEvalSteps.
*
......@@ -131,6 +137,7 @@ typedef enum ExprEvalOp
/* evaluate PARAM_EXEC/EXTERN parameters */
EEOP_PARAM_EXEC,
EEOP_PARAM_EXTERN,
EEOP_PARAM_CALLBACK,
/* return CaseTestExpr value */
EEOP_CASE_TESTVAL,
......@@ -331,6 +338,15 @@ typedef struct ExprEvalStep
Oid paramtype; /* OID of parameter's datatype */
} param;
/* for EEOP_PARAM_CALLBACK */
struct
{
ExecEvalSubroutine paramfunc; /* add-on evaluation subroutine */
void *paramarg; /* private data for same */
int paramid; /* numeric ID for parameter */
Oid paramtype; /* OID of parameter's datatype */
} cparam;
/* for EEOP_CASE_TESTVAL/DOMAIN_TESTVAL */
struct
{
......@@ -598,8 +614,11 @@ typedef struct ArrayRefState
} ArrayRefState;
extern void ExecReadyInterpretedExpr(ExprState *state);
/* functions in execExpr.c */
extern void ExprEvalPushStep(ExprState *es, const ExprEvalStep *s);
/* functions in execExprInterp.c */
extern void ExecReadyInterpretedExpr(ExprState *state);
extern ExprEvalOp ExecEvalStepOp(ExprState *state, ExprEvalStep *op);
/*
......
......@@ -247,6 +247,7 @@ ExecProcNode(PlanState *node)
* prototypes from functions in execExpr.c
*/
extern ExprState *ExecInitExpr(Expr *node, PlanState *parent);
extern ExprState *ExecInitExprWithParams(Expr *node, ParamListInfo ext_params);
extern ExprState *ExecInitQual(List *qual, PlanState *parent);
extern ExprState *ExecInitCheck(List *qual, PlanState *parent);
extern List *ExecInitExprList(List *nodes, PlanState *parent);
......
......@@ -33,6 +33,13 @@
#include "storage/condition_variable.h"
struct PlanState; /* forward references in this file */
struct ParallelHashJoinState;
struct ExprState;
struct ExprContext;
struct ExprEvalStep; /* avoid including execExpr.h everywhere */
/* ----------------
* ExprState node
*
......@@ -40,12 +47,6 @@
* It contains instructions (in ->steps) to evaluate the expression.
* ----------------
*/
struct ExprState; /* forward references in this file */
struct ExprContext;
struct ExprEvalStep; /* avoid including execExpr.h everywhere */
struct ParallelHashJoinState;
typedef Datum (*ExprStateEvalFunc) (struct ExprState *expression,
struct ExprContext *econtext,
bool *isNull);
......@@ -87,12 +88,16 @@ typedef struct ExprState
Expr *expr;
/*
* XXX: following only needed during "compilation", could be thrown away.
* XXX: following fields only needed during "compilation" (ExecInitExpr);
* could be thrown away afterwards.
*/
int steps_len; /* number of steps currently */
int steps_alloc; /* allocated length of steps array */
struct PlanState *parent; /* parent PlanState node, if any */
ParamListInfo ext_params; /* for compiling PARAM_EXTERN nodes */
Datum *innermost_caseval;
bool *innermost_casenull;
......@@ -827,8 +832,6 @@ typedef struct DomainConstraintState
* ----------------------------------------------------------------
*/
struct PlanState;
/* ----------------
* ExecProcNodeMtd
*
......
......@@ -16,16 +16,23 @@
/* Forward declarations, to avoid including other headers */
struct Bitmapset;
struct ExprState;
struct Param;
struct ParseState;
/* ----------------
/*
* ParamListInfo
*
* ParamListInfo arrays are used to pass parameters into the executor
* for parameterized plans. Each entry in the array defines the value
* to be substituted for a PARAM_EXTERN parameter. The "paramid"
* of a PARAM_EXTERN Param can range from 1 to numParams.
* ParamListInfo structures are used to pass parameters into the executor
* for parameterized plans. We support two basic approaches to supplying
* parameter values, the "static" way and the "dynamic" way.
*
* In the static approach, per-parameter data is stored in an array of
* ParamExternData structs appended to the ParamListInfo struct.
* Each entry in the array defines the value to be substituted for a
* PARAM_EXTERN parameter. The "paramid" of a PARAM_EXTERN Param
* can range from 1 to numParams.
*
* Although parameter numbers are normally consecutive, we allow
* ptype == InvalidOid to signal an unused array entry.
......@@ -35,18 +42,47 @@ struct ParseState;
* as a constant (i.e., generate a plan that works only for this value
* of the parameter).
*
* There are two hook functions that can be associated with a ParamListInfo
* array to support dynamic parameter handling. First, if paramFetch
* isn't null and the executor requires a value for an invalid parameter
* (one with ptype == InvalidOid), the paramFetch hook is called to give
* it a chance to fill in the parameter value. Second, a parserSetup
* hook can be supplied to re-instantiate the original parsing hooks if
* a query needs to be re-parsed/planned (as a substitute for supposing
* that the current ptype values represent a fixed set of parameter types).
* In the dynamic approach, all access to parameter values is done through
* hook functions found in the ParamListInfo struct. In this case,
* the ParamExternData array is typically unused and not allocated;
* but the legal range of paramid is still 1 to numParams.
*
* Although the data structure is really an array, not a list, we keep
* the old typedef name to avoid unnecessary code changes.
* ----------------
*
* There are 3 hook functions that can be associated with a ParamListInfo
* structure:
*
* If paramFetch isn't null, it is called to fetch the ParamExternData
* for a particular param ID, rather than accessing the relevant element
* of the ParamExternData array. This supports the case where the array
* isn't there at all, as well as cases where the data in the array
* might be obsolete or lazily evaluated. paramFetch must return the
* address of a ParamExternData struct describing the specified param ID;
* the convention above about ptype == InvalidOid signaling an invalid
* param ID still applies. The returned struct can either be placed in
* the "workspace" supplied by the caller, or it can be in storage
* controlled by the paramFetch hook if that's more convenient.
* (In either case, the struct is not expected to be long-lived.)
* If "speculative" is true, the paramFetch hook should not risk errors
* in trying to fetch the parameter value, and should report an invalid
* parameter instead.
*
* If paramCompile isn't null, then it controls what execExpr.c compiles
* for PARAM_EXTERN Param nodes --- typically, this hook would emit a
* EEOP_PARAM_CALLBACK step. This allows unnecessary work to be
* optimized away in compiled expressions.
*
* If parserSetup isn't null, then it is called to re-instantiate the
* original parsing hooks when a query needs to be re-parsed/planned.
* This is especially useful if the types of parameters might change
* from time to time, since it can replace the need to supply a fixed
* list of parameter types to the parser.
*
* Notice that the paramFetch and paramCompile hooks are actually passed
* the ParamListInfo struct's address; they can therefore access all
* three of the "arg" fields, and the distinction between paramFetchArg
* and paramCompileArg is rather arbitrary.
*/
#define PARAM_FLAG_CONST 0x0001 /* parameter is constant */
......@@ -61,7 +97,13 @@ typedef struct ParamExternData
typedef struct ParamListInfoData *ParamListInfo;
typedef void (*ParamFetchHook) (ParamListInfo params, int paramid);
typedef ParamExternData *(*ParamFetchHook) (ParamListInfo params,
int paramid, bool speculative,
ParamExternData *workspace);
typedef void (*ParamCompileHook) (ParamListInfo params, struct Param *param,
struct ExprState *state,
Datum *resv, bool *resnull);
typedef void (*ParserSetupHook) (struct ParseState *pstate, void *arg);
......@@ -69,10 +111,16 @@ typedef struct ParamListInfoData
{
ParamFetchHook paramFetch; /* parameter fetch hook */
void *paramFetchArg;
ParamCompileHook paramCompile; /* parameter compile hook */
void *paramCompileArg;
ParserSetupHook parserSetup; /* parser setup hook */
void *parserSetupArg;
int numParams; /* number of ParamExternDatas following */
struct Bitmapset *paramMask; /* if non-NULL, can ignore omitted params */
int numParams; /* nominal/maximum # of Params represented */
/*
* params[] may be of length zero if paramFetch is supplied; otherwise it
* must be of length numParams.
*/
ParamExternData params[FLEXIBLE_ARRAY_MEMBER];
} ParamListInfoData;
......
......@@ -2350,35 +2350,17 @@ plpgsql_adddatum(PLpgSQL_datum *new)
/* ----------
* plpgsql_finish_datums Copy completed datum info into function struct.
*
* This is also responsible for building resettable_datums, a bitmapset
* of the dnos of all ROW, REC, and RECFIELD datums in the function.
* ----------
*/
static void
plpgsql_finish_datums(PLpgSQL_function *function)
{
Bitmapset *resettable_datums = NULL;
int i;
function->ndatums = plpgsql_nDatums;
function->datums = palloc(sizeof(PLpgSQL_datum *) * plpgsql_nDatums);
for (i = 0; i < plpgsql_nDatums; i++)
{
function->datums[i] = plpgsql_Datums[i];
switch (function->datums[i]->dtype)
{
case PLPGSQL_DTYPE_ROW:
case PLPGSQL_DTYPE_REC:
case PLPGSQL_DTYPE_RECFIELD:
resettable_datums = bms_add_member(resettable_datums, i);
break;
default:
break;
}
}
function->resettable_datums = resettable_datums;
}
......
This diff is collapsed.
......@@ -857,7 +857,6 @@ typedef struct PLpgSQL_function
/* the datums representing the function's local variables */
int ndatums;
PLpgSQL_datum **datums;
Bitmapset *resettable_datums; /* dnos of non-simple vars */
/* function body parsetree */
PLpgSQL_stmt_block *action;
......@@ -899,9 +898,13 @@ typedef struct PLpgSQL_execstate
int ndatums;
PLpgSQL_datum **datums;
/* we pass datums[i] to the executor, when needed, in paramLI->params[i] */
/*
* paramLI is what we use to pass local variable values to the executor.
* It does not have a ParamExternData array; we just dynamically
* instantiate parameter data as needed. By convention, PARAM_EXTERN
* Params have paramid equal to the dno of the referenced local variable.
*/
ParamListInfo paramLI;
bool params_dirty; /* T if any resettable datum has been passed */
/* EState to use for "simple" expression evaluation */
EState *simple_eval_estate;
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment