1. 21 Oct, 2008 1 commit
    • Tom Lane's avatar
      Add a concept of "placeholder" variables to the planner. These are variables · e6ae3b5d
      Tom Lane authored
      that represent some expression that we desire to compute below the top level
      of the plan, and then let that value "bubble up" as though it were a plain
      Var (ie, a column value).
      
      The immediate application is to allow sub-selects to be flattened even when
      they are below an outer join and have non-nullable output expressions.
      Formerly we couldn't flatten because such an expression wouldn't properly
      go to NULL when evaluated above the outer join.  Now, we wrap it in a
      PlaceHolderVar and arrange for the actual evaluation to occur below the outer
      join.  When the resulting Var bubbles up through the join, it will be set to
      NULL if necessary, yielding the correct results.  This fixes a planner
      limitation that's existed since 7.1.
      
      In future we might want to use this mechanism to re-introduce some form of
      Hellerstein's "expensive functions" optimization, ie place the evaluation of
      an expensive function at the most suitable point in the plan tree.
      e6ae3b5d
  2. 07 Oct, 2008 1 commit
  3. 06 Oct, 2008 1 commit
    • Tom Lane's avatar
      When expanding a whole-row Var into a RowExpr during ResolveNew(), attach · bf461538
      Tom Lane authored
      the column alias names of the RTE referenced by the Var to the RowExpr.
      This is needed to allow ruleutils.c to correctly deparse FieldSelect nodes
      referencing such a construct.  Per my recent bug report.
      
      Adding a field to RowExpr forces initdb (because of stored rules changes)
      so this solution is not back-patchable; which is unfortunate because 8.2
      and 8.3 have this issue.  But it only affects EXPLAIN for some pretty odd
      corner cases, so we can probably live without a solution for the back
      branches.
      bf461538
  4. 04 Oct, 2008 1 commit
    • Tom Lane's avatar
      Implement SQL-standard WITH clauses, including WITH RECURSIVE. · 44d5be0e
      Tom Lane authored
      There are some unimplemented aspects: recursive queries must use UNION ALL
      (should allow UNION too), and we don't have SEARCH or CYCLE clauses.
      These might or might not get done for 8.4, but even without them it's a
      pretty useful feature.
      
      There are also a couple of small loose ends and definitional quibbles,
      which I'll send a memo about to pgsql-hackers shortly.  But let's land
      the patch now so we can get on with other development.
      
      Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
      44d5be0e
  5. 28 Aug, 2008 1 commit
    • Tom Lane's avatar
      Extend the parser location infrastructure to include a location field in · a2794623
      Tom Lane authored
      most node types used in expression trees (both before and after parse
      analysis).  This allows us to place an error cursor in many situations
      where we formerly could not, because the information wasn't available
      beyond the very first level of parse analysis.  There's a fair amount
      of work still to be done to persuade individual ereport() calls to actually
      include an error location, but this gets the initdb-forcing part of the
      work out of the way; and the situation is already markedly better than
      before for complaints about unimplementable implicit casts, such as
      CASE and UNION constructs with incompatible alternative data types.
      Per my proposal of a few days ago.
      a2794623
  6. 25 Aug, 2008 1 commit
  7. 14 Aug, 2008 1 commit
    • Tom Lane's avatar
      Implement SEMI and ANTI joins in the planner and executor. (Semijoins replace · e006a24a
      Tom Lane authored
      the old JOIN_IN code, but antijoins are new functionality.)  Teach the planner
      to convert appropriate EXISTS and NOT EXISTS subqueries into semi and anti
      joins respectively.  Also, LEFT JOINs with suitable upper-level IS NULL
      filters are recognized as being anti joins.  Unify the InClauseInfo and
      OuterJoinInfo infrastructure into "SpecialJoinInfo".  With that change,
      it becomes possible to associate a SpecialJoinInfo with every join attempt,
      which permits some cleanup of join selectivity estimation.  That needs to be
      taken much further than this patch does, but the next step is to change the
      API for oprjoin selectivity functions, which seems like material for a
      separate patch.  So for the moment the output size estimates for semi and
      especially anti joins are quite bogus.
      e006a24a
  8. 07 Aug, 2008 3 commits
    • Tom Lane's avatar
      Improve INTERSECT/EXCEPT hashing by realizing that we don't need to make any · af95d7aa
      Tom Lane authored
      hashtable entries for tuples that are found only in the second input: they
      can never contribute to the output.  Furthermore, this implies that the
      planner should endeavor to put first the smaller (in number of groups) input
      relation for an INTERSECT.  Implement that, and upgrade prepunion's estimation
      of the number of rows returned by setops so that there's some amount of sanity
      in the estimate of which one is smaller.
      af95d7aa
    • Tom Lane's avatar
      Support hashing for duplicate-elimination in INTERSECT and EXCEPT queries. · 368df304
      Tom Lane authored
      This completes my project of improving usage of hashing for duplicate
      elimination (aggregate functions with DISTINCT remain undone, but that's
      for some other day).
      
      As with the previous patches, this means we can INTERSECT/EXCEPT on datatypes
      that can hash but not sort, and it means that INTERSECT/EXCEPT without ORDER
      BY are no longer certain to produce sorted output.
      368df304
    • Tom Lane's avatar
      Teach the system how to use hashing for UNION. (INTERSECT/EXCEPT will follow, · 2d1d96b1
      Tom Lane authored
      but seem like a separate patch since most of the remaining work is on the
      executor side.)  I took the opportunity to push selection of the grouping
      operators for set operations into the parser where it belongs.  Otherwise this
      is just a small exercise in making prepunion.c consider both alternatives.
      
      As with the recent DISTINCT patch, this means we can UNION on datatypes that
      can hash but not sort, and it means that UNION without ORDER BY is no longer
      certain to produce sorted output.
      2d1d96b1
  9. 02 Aug, 2008 1 commit
    • Tom Lane's avatar
      Rearrange the querytree representation of ORDER BY/GROUP BY/DISTINCT items · 95113047
      Tom Lane authored
      as per my recent proposal:
      
      1. Fold SortClause and GroupClause into a single node type SortGroupClause.
      We were already relying on them to be struct-equivalent, so using two node
      tags wasn't accomplishing much except to get in the way of comparing items
      with equal().
      
      2. Add an "eqop" field to SortGroupClause to carry the associated equality
      operator.  This is cheap for the parser to get at the same time it's looking
      up the sort operator, and storing it eliminates the need for repeated
      not-so-cheap lookups during planning.  In future this will also let us
      represent GROUP/DISTINCT operations on datatypes that have hash opclasses
      but no btree opclasses (ie, they have equality but no natural sort order).
      The previous representation simply didn't work for that, since its only
      indicator of comparison semantics was a sort operator.
      
      3. Add a hasDistinctOn boolean to struct Query to explicitly record whether
      the distinctClause came from DISTINCT or DISTINCT ON.  This allows removing
      some complicated and not 100% bulletproof code that attempted to figure
      that out from the distinctClause alone.
      
      This patch doesn't in itself create any new capability, but it's necessary
      infrastructure for future attempts to use hash-based grouping for DISTINCT
      and UNION/INTERSECT/EXCEPT.
      95113047
  10. 31 Jul, 2008 1 commit
    • Tom Lane's avatar
      Fix parser so that we don't modify the user-written ORDER BY list in order · 63247bec
      Tom Lane authored
      to represent DISTINCT or DISTINCT ON.  This gets rid of a longstanding
      annoyance that a view or rule using SELECT DISTINCT will be dumped out
      with an overspecified ORDER BY list, and is one small step along the way
      to decoupling DISTINCT and ORDER BY enough so that hash-based implementation
      of DISTINCT will be possible.  In passing, improve transformDistinctClause
      so that it doesn't reject duplicate DISTINCT ON items, as was reported by
      Steve Midgley a couple weeks ago.
      63247bec
  11. 19 Jun, 2008 1 commit
  12. 01 Jan, 2008 1 commit
  13. 15 Nov, 2007 1 commit
  14. 22 Oct, 2007 1 commit
  15. 12 Jul, 2007 1 commit
  16. 11 Jun, 2007 1 commit
    • Tom Lane's avatar
      Support UPDATE/DELETE WHERE CURRENT OF cursor_name, per SQL standard. · 6808f1b1
      Tom Lane authored
      Along the way, allow FOR UPDATE in non-WITH-HOLD cursors; there may once
      have been a reason to disallow that, but it seems to work now, and it's
      really rather necessary if you want to select a row via a cursor and then
      update it in a concurrent-safe fashion.
      
      Original patch by Arul Shaji, rather heavily editorialized by Tom Lane.
      6808f1b1
  17. 21 Apr, 2007 1 commit
  18. 17 Mar, 2007 1 commit
    • Tom Lane's avatar
      Fix up the remaining places where the expression node structure would lose · 0f4ff460
      Tom Lane authored
      available information about the typmod of an expression; namely, Const,
      ArrayRef, ArrayExpr, and EXPR and ARRAY SubLinks.  In the ArrayExpr and
      SubLink cases it wasn't really the data structure's fault, but exprTypmod()
      being lazy.  This seems like a good idea in view of the expected increase in
      typmod usage from Teodor's work to allow user-defined types to have typmods.
      In particular this responds to the concerns we had about eliminating the
      special-purpose hack that exprTypmod() used to have for BPCHAR Consts.
      We can now tell whether or not such a Const has been cast to a specific
      length, and report or display properly if so.
      
      initdb forced due to changes in stored rules.
      0f4ff460
  19. 22 Feb, 2007 1 commit
    • Tom Lane's avatar
      Turn the rangetable used by the executor into a flat list, and avoid storing · eab6b8b2
      Tom Lane authored
      useless substructure for its RangeTblEntry nodes.  (I chose to keep using the
      same struct node type and just zero out the link fields for unneeded info,
      rather than making a separate ExecRangeTblEntry type --- it seemed too
      fragile to have two different rangetable representations.)
      
      Along the way, put subplans into a list in the toplevel PlannedStmt node,
      and have SubPlan nodes refer to them by list index instead of direct pointers.
      Vadim wanted to do that years ago, but I never understood what he was on about
      until now.  It makes things a *whole* lot more robust, because we can stop
      worrying about duplicate processing of subplans during expression tree
      traversals.  That's been a constant source of bugs, and it's finally gone.
      
      There are some consequent simplifications yet to be made, like not using
      a separate EState for subplans in the executor, but I'll tackle that later.
      eab6b8b2
  20. 19 Feb, 2007 1 commit
    • Tom Lane's avatar
      Get rid of some old and crufty global variables in the planner. When · 7c5e5439
      Tom Lane authored
      this code was last gone over, there wasn't really any alternative to
      globals because we didn't have the PlannerInfo struct being passed all
      through the planner code.  Now that we do, we can restructure things
      to avoid non-reentrancy.  I'm fooling with this because otherwise I'd
      have had to add another global variable for the planned compact
      range table list.
      7c5e5439
  21. 22 Jan, 2007 1 commit
    • Tom Lane's avatar
      Put back planner's ability to cache the results of mergejoinscansel(), · 4f06c688
      Tom Lane authored
      which I had removed in the first cut of the EquivalenceClass rewrite to
      simplify that patch a little.  But it's still important --- in a four-way
      join problem mergejoinscansel() was eating about 40% of the planning time
      according to gprof.  Also, improve the EquivalenceClass code to re-use
      join RestrictInfos rather than generating fresh ones for each join
      considered.  This saves some memory space but more importantly improves
      the effectiveness of caching planning info in RestrictInfos.
      4f06c688
  22. 20 Jan, 2007 1 commit
    • Tom Lane's avatar
      Refactor planner's pathkeys data structure to create a separate, explicit · f41803bb
      Tom Lane authored
      representation of equivalence classes of variables.  This is an extensive
      rewrite, but it brings a number of benefits:
      * planner no longer fails in the presence of "incomplete" operator families
      that don't offer operators for every possible combination of datatypes.
      * avoid generating and then discarding redundant equality clauses.
      * remove bogus assumption that derived equalities always use operators
      named "=".
      * mergejoins can work with a variety of sort orders (e.g., descending) now,
      instead of tying each mergejoinable operator to exactly one sort order.
      * better recognition of redundant sort columns.
      * can make use of equalities appearing underneath an outer join.
      f41803bb
  23. 05 Jan, 2007 1 commit
  24. 04 Oct, 2006 1 commit
  25. 10 Aug, 2006 1 commit
  26. 30 Apr, 2006 1 commit
  27. 05 Mar, 2006 1 commit
  28. 03 Feb, 2006 1 commit
    • Tom Lane's avatar
      Teach planner to convert simple UNION ALL subqueries into append relations, · 8b109ebf
      Tom Lane authored
      thereby sharing code with the inheritance case.  This puts the UNION-ALL-view
      approach to partitioned tables on par with inheritance, so far as constraint
      exclusion is concerned: it works either way.  (Still need to update the docs
      to say so.)  The definition of "simple UNION ALL" is a little simpler than
      I would like --- basically the union arms can only be SELECT * FROM foo
      --- but it's good enough for partitioned-table cases.
      8b109ebf
  29. 31 Jan, 2006 1 commit
    • Tom Lane's avatar
      Restructure planner's handling of inheritance. Rather than processing · 8a1468af
      Tom Lane authored
      inheritance trees on-the-fly, which pretty well constrained us to considering
      only one way of planning inheritance, expand inheritance sets during the
      planner prep phase, and build a side data structure that can be consulted
      later to find which RTEs are members of which inheritance sets.  As proof of
      concept, use the data structure to plan joins against inheritance sets more
      efficiently: we can now use indexes on the set members in inner-indexscan
      joins.  (The generated plans could be improved further, but it'll take some
      executor changes.)  This data structure will also support handling UNION ALL
      subqueries in the same way as inheritance sets, but that aspect of it isn't
      finished yet.
      8a1468af
  30. 22 Nov, 2005 1 commit
  31. 15 Oct, 2005 1 commit
  32. 02 Aug, 2005 1 commit
  33. 28 Jul, 2005 1 commit
  34. 10 Jun, 2005 1 commit
    • Tom Lane's avatar
      If a LIMIT is applied to a UNION ALL query, plan each UNION arm as · 3b167a40
      Tom Lane authored
      if the limit were directly applied to it.  This does not actually
      add a LIMIT plan node to the generated subqueries --- that would be
      useless overhead --- but it does cause the planner to prefer fast-
      start plans when the limit is small.  After an idea from Phil Endecott.
      3b167a40
  35. 09 Jun, 2005 1 commit
    • Tom Lane's avatar
      Simplify the planner's join clause management by storing join clauses · a31ad27f
      Tom Lane authored
      of a relation in a flat 'joininfo' list.  The former arrangement grouped
      the join clauses according to the set of unjoined relids used in each;
      however, profiling on test cases involving lots of joins proves that
      that data structure is a net loss.  It takes more time to group the
      join clauses together than is saved by avoiding duplicate tests later.
      It doesn't help any that there are usually not more than one or two
      clauses per group ...
      a31ad27f
  36. 05 Jun, 2005 1 commit
    • Tom Lane's avatar
      Remove planner's private fields from Query struct, and put them into · 9ab4d981
      Tom Lane authored
      a new PlannerInfo struct, which is passed around instead of the bare
      Query in all the planning code.  This commit is essentially just a
      code-beautification exercise, but it does open the door to making
      larger changes to the planner data structures without having to muck
      with the widely-known Query struct.
      9ab4d981
  37. 22 May, 2005 1 commit
    • Tom Lane's avatar
      Teach the planner to remove SubqueryScan nodes from the plan if they · e2159f38
      Tom Lane authored
      aren't doing anything useful (ie, neither selection nor projection).
      Also, extend to SubqueryScan the hacks already in place to avoid
      unnecessary ExecProject calls when the result would just be the same
      tuple the subquery already delivered.  This saves some overhead in
      UNION and other set operations, as well as avoiding overhead for
      unflatten-able subqueries.  Per example from Sokolov Yura.
      e2159f38
  38. 06 Apr, 2005 1 commit
    • Tom Lane's avatar
      Merge Resdom nodes into TargetEntry nodes to simplify code and save a · ad161bcc
      Tom Lane authored
      few palloc's.  I also chose to eliminate the restype and restypmod fields
      entirely, since they are redundant with information stored in the node's
      contained expression; re-examining the expression at need seems simpler
      and more reliable than trying to keep restype/restypmod up to date.
      
      initdb forced due to change in contents of stored rules.
      ad161bcc