1. 24 Sep, 2014 5 commits
    • Stephen Frost's avatar
      Copy-editing of row security · afd1d95f
      Stephen Frost authored
      Address a few typos in the row security update, pointed out
      off-list by Adam Brightwell.  Also include 'ALL' in the list
      of commands supported, for completeness.
      afd1d95f
    • Stephen Frost's avatar
      Code review for row security. · 6550b901
      Stephen Frost authored
      Buildfarm member tick identified an issue where the policies in the
      relcache for a relation were were being replaced underneath a running
      query, leading to segfaults while processing the policies to be added
      to a query.  Similar to how TupleDesc RuleLocks are handled, add in a
      equalRSDesc() function to check if the policies have actually changed
      and, if not, swap back the rsdesc field (using the original instead of
      the temporairly built one; the whole structure is swapped and then
      specific fields swapped back).  This now passes a CLOBBER_CACHE_ALWAYS
      for me and should resolve the buildfarm error.
      
      In addition to addressing this, add a new chapter in Data Definition
      under Privileges which explains row security and provides examples of
      its usage, change \d to always list policies (even if row security is
      disabled- but note that it is disabled, or enabled with no policies),
      rework check_role_for_policy (it really didn't need the entire policy,
      but it did need to be using has_privs_of_role()), and change the field
      in pg_class to relrowsecurity from relhasrowsecurity, based on
      Heikki's suggestion.  Also from Heikki, only issue SET ROW_SECURITY in
      pg_restore when talking to a 9.5+ server, list Bypass RLS in \du, and
      document --enable-row-security options for pg_dump and pg_restore.
      
      Lastly, fix a number of minor whitespace and typo issues from Heikki,
      Dimitri, add a missing #include, per Peter E, fix a few minor
      variable-assigned-but-not-used and resource leak issues from Coverity
      and add tab completion for role attribute bypassrls as well.
      6550b901
    • Tom Lane's avatar
      Fix bogus variable-mangling in security_barrier_replace_vars(). · 3f6f9260
      Tom Lane authored
      This function created new Vars with varno different from varnoold, which
      is a condition that should never prevail before setrefs.c does the final
      variable-renumbering pass.  The created Vars could not be seen as equal()
      to normal Vars, which among other things broke equivalence-class processing
      for them.  The consequences of this were indeed visible in the regression
      tests, in the form of failure to propagate constants as one would expect.
      I stumbled across it while poking at bug #11457 --- after intentionally
      disabling join equivalence processing, the security-barrier regression
      tests started falling over with fun errors like "could not find pathkey
      item to sort", because of failure to match the corrupted Vars to normal
      ones.
      3f6f9260
    • Andrew Dunstan's avatar
      b1a52872
    • Tom Lane's avatar
      Fix incorrect search for "x?" style matches in creviterdissect(). · 3694b4d7
      Tom Lane authored
      When the number of allowed iterations is limited (either a "?" quantifier
      or a bound expression), the last sub-match has to reach to the end of the
      target string.  The previous coding here first tried the shortest possible
      match (one character, usually) and then gave up and back-tracked if that
      didn't work, typically leading to failure to match overall, as shown in
      bug #11478 from Christoph Berg.  The minimum change to fix that would be to
      not decrement k before "goto backtrack"; but that would be a pretty stupid
      solution, because we'd laboriously try each possible sub-match length
      before finally discovering that only ending at the end can work.  Instead,
      force the sub-match endpoint limit up to the end for even the first
      shortest() call if we cannot have any more sub-matches after this one.
      
      Bug introduced in my rewrite that added the iterdissect logic, commit
      173e29aa.  The shortest-first search code
      was too closely modeled on the longest-first code, which hasn't got this
      issue since it tries a match reaching to the end to start with anyway.
      Back-patch to all affected branches.
      3694b4d7
  2. 23 Sep, 2014 3 commits
    • Stephen Frost's avatar
      Add unicode_*_linestyle to \? variables · a5643073
      Stephen Frost authored
      In a2dabf0e we added the ability to have single or double unicode
      linestyle for the border, column, or header.  Unfortunately, the
      \? variables output was not updated for these new psql variables.
      
      This corrects that oversight.
      
      Patch by Pavel Stehule.
      a5643073
    • Stephen Frost's avatar
      Log ALTER SYSTEM statements as DDL · 43bed84c
      Stephen Frost authored
      Per discussion in bug #11350, log ALTER SYSTEM commands at the
      log_statement=ddl level, rather than at the log_statement=all level.
      
      Pointed out by Tomonari Katsumata.
      
      Back-patch to 9.4 where ALTER SYSTEM was introduced.
      43bed84c
    • Stephen Frost's avatar
      Process withCheckOption exprs in setrefs.c · 6ef8c658
      Stephen Frost authored
      While withCheckOption exprs had been handled in many cases by
      happenstance, they need to be handled during set_plan_references and
      more specifically down in set_plan_refs for ModifyTable plan nodes.
      This is to ensure that the opfuncid's are set for operators referenced
      in the withCheckOption exprs.
      
      Identified as an issue by Thom Brown
      
      Patch by Dean Rasheed
      
      Back-patch to 9.4, where withCheckOption was introduced.
      6ef8c658
  3. 22 Sep, 2014 6 commits
    • Andres Freund's avatar
      Remove most volatile qualifiers from xlog.c · 6ba4ecbf
      Andres Freund authored
      For the reason outlined in df4077cd also remove volatile qualifiers
      from xlog.c. Some of these uses of volatile have been added after
      noticing problems back when spinlocks didn't imply compiler
      barriers. So they are a good test - in fact removing the volatiles
      breaks when done without the barriers in spinlocks present.
      
      Several uses of volatile remain where they are explicitly used to
      access shared memory without locks. These locations are ok with
      slightly out of date data, but removing the volatile might lead to the
      variables never being reread from memory. These uses could also be
      replaced by barriers, but that's a separate change of doubtful value.
      6ba4ecbf
    • Robert Haas's avatar
      Remove volatile qualifiers from lwlock.c. · df4077cd
      Robert Haas authored
      Now that spinlocks (hopefully!) act as compiler barriers, as of commit
      0709b7ee, this should be safe.  This
      serves as a demonstration of the new coding style, and may be optimized
      better on some machines as well.
      df4077cd
    • Robert Haas's avatar
      Fix compiler warning. · e38da8d6
      Robert Haas authored
      It is meaningless to declare a pass-by-value return type const.
      e38da8d6
    • Robert Haas's avatar
      Fix mishandling of CreateEventTrigStmt's eventname field. · 763ba1b0
      Robert Haas authored
      It's a string, not a scalar.
      
      Petr Jelinek
      763ba1b0
    • Andres Freund's avatar
      Remove postgres --help blurb about the removed -A option. · 0926ef43
      Andres Freund authored
      I missed this in 3bdcf6a5.
      
      Noticed by Merlin Moncure
      Discussion: CAHyXU0yC7uPeeVzQROwtnrOP9dxTEUPYjB0og4qUnbipMEV57w@mail.gmail.com
      0926ef43
    • Andres Freund's avatar
      Improve code around the recently added rm_identify rmgr callback. · 604f7956
      Andres Freund authored
      There are four weaknesses in728f152e:
      
      * append_init() in heapdesc.c was ugly and required that rm_identify
        return values are only valid till the next call. Instead just add a
        couple more switch() cases for the INIT_PAGE cases. Now the returned
        value will always be valid.
      * a couple rm_identify() callbacks missed masking xl_info with
        ~XLR_INFO_MASK.
      * pg_xlogdump didn't map a NULL rm_identify to UNKNOWN or a similar
        string.
      * append_init() was called when id=NULL - which should never actually
        happen. But it's better to be careful.
      604f7956
  4. 19 Sep, 2014 8 commits
    • Tom Lane's avatar
      Fix failure of contrib/auto_explain to print per-node timing information. · 898f8a96
      Tom Lane authored
      This has been broken since commit af7914c6,
      which added the EXPLAIN (TIMING) option.  Although that commit included
      updates to auto_explain, they evidently weren't tested very carefully,
      because the code failed to print node timings even when it should, due to
      failure to set es.timing in the ExplainState struct.  Reported off-list by
      Neelakanth Nadgir of Salesforce.
      
      In passing, clean up the documentation for auto_explain's options a
      little bit, including re-ordering them into what seems to me a more
      logical order.
      898f8a96
    • Robert Haas's avatar
      doc: Use <literal> and all-caps for READ COMMITTED isolation level. · a92b5f96
      Robert Haas authored
      The documentation overall is not entirely consistent about how we do
      this, but this is consistent with other usages within lock.sgml.
      
      Etsuro Fujita
      a92b5f96
    • Robert Haas's avatar
      Add a fast pre-check for equality of equal-length strings. · e246b3d6
      Robert Haas authored
      Testing reveals that that doing a memcmp() before the strcoll() costs
      practically nothing, at least on the systems we tested, and it speeds
      up sorts containing many equal strings significatly.
      
      Peter Geoghegan.  Review by myself and Heikki Linnakangas.  Comments
      rewritten by me.
      e246b3d6
    • Stephen Frost's avatar
      Row-Level Security Policies (RLS) · 491c029d
      Stephen Frost authored
      Building on the updatable security-barrier views work, add the
      ability to define policies on tables to limit the set of rows
      which are returned from a query and which are allowed to be added
      to a table.  Expressions defined by the policy for filtering are
      added to the security barrier quals of the query, while expressions
      defined to check records being added to a table are added to the
      with-check options of the query.
      
      New top-level commands are CREATE/ALTER/DROP POLICY and are
      controlled by the table owner.  Row Security is able to be enabled
      and disabled by the owner on a per-table basis using
      ALTER TABLE .. ENABLE/DISABLE ROW SECURITY.
      
      Per discussion, ROW SECURITY is disabled on tables by default and
      must be enabled for policies on the table to be used.  If no
      policies exist on a table with ROW SECURITY enabled, a default-deny
      policy is used and no records will be visible.
      
      By default, row security is applied at all times except for the
      table owner and the superuser.  A new GUC, row_security, is added
      which can be set to ON, OFF, or FORCE.  When set to FORCE, row
      security will be applied even for the table owner and superusers.
      When set to OFF, row security will be disabled when allowed and an
      error will be thrown if the user does not have rights to bypass row
      security.
      
      Per discussion, pg_dump sets row_security = OFF by default to ensure
      that exports and backups will have all data in the table or will
      error if there are insufficient privileges to bypass row security.
      A new option has been added to pg_dump, --enable-row-security, to
      ask pg_dump to export with row security enabled.
      
      A new role capability, BYPASSRLS, which can only be set by the
      superuser, is added to allow other users to be able to bypass row
      security using row_security = OFF.
      
      Many thanks to the various individuals who have helped with the
      design, particularly Robert Haas for his feedback.
      
      Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean
      Rasheed, with additional changes and rework by me.
      
      Reviewers have included all of the above, Greg Smith,
      Jeff McCormick, and Robert Haas.
      491c029d
    • Andres Freund's avatar
      Mark x86's memory barrier inline assembly as clobbering the cpu flags. · e5603a2f
      Andres Freund authored
      x86's memory barrier assembly was marked as clobbering "memory" but
      not "cc" even though 'addl' sets various flags. As it turns out gcc on
      x86 implicitly assumes "cc" on every inline assembler statement, so
      it's not a bug. But as that's poorly documented and might get copied
      to architectures or compilers where that's not the case, it seems
      better to be precise.
      
      Discussion: 20140919100016.GH4277@alap3.anarazel.de
      
      To keep the code common, backpatch to 9.2 where explicit memory
      barriers were introduced.
      e5603a2f
    • Andres Freund's avatar
      Avoid 'clobbered by longjmp' warning in psql/copy.c. · afaefa1b
      Andres Freund authored
      This was introduced in 51bb7956.
      afaefa1b
    • Andres Freund's avatar
      Add the capability to display summary statistics to pg_xlogdump. · bdd5726c
      Andres Freund authored
      The new --stats/--stats=record options to pg_xlogdump display per
      rmgr/per record statistics about the parsed WAL. This is useful to
      understand what the WAL primarily consists of, to allow targeted
      optimizations on application, configuration, and core code level.
      
      It is likely that we will want to fine tune the statistics further,
      but the feature already is quite helpful.
      
      Author: Abhijit Menon-Sen, slightly editorialized by me
      Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu
      Discussion: 20140604104716.GA3989@toroid.org
      bdd5726c
    • Andres Freund's avatar
      Add rmgr callback to name xlog record types for display purposes. · 728f152e
      Andres Freund authored
      This is primarily useful for the upcoming pg_xlogdump --stats feature,
      but also allows to remove some duplicated code in the rmgr_desc
      routines.
      
      Due to the separation and harmonization, the output of dipsplayed
      records changes somewhat. But since this isn't enduser oriented
      content that's ok.
      
      It's potentially desirable to further change pg_xlogdump's display of
      records. It previously wasn't possible to show the record type
      separately from the description forcing it to be in the last
      column. But that's better done in a separate commit.
      
      Author: Abhijit Menon-Sen, slightly editorialized by me
      Reviewed-By: Álvaro Herrera, Andres Freund, and Heikki Linnakangas
      Discussion: 20140604104716.GA3989@toroid.org
      728f152e
  5. 18 Sep, 2014 1 commit
    • Andres Freund's avatar
      Fix configure check for %z printf support after INT64_MODIFIER changes. · 7e3f7283
      Andres Freund authored
      The PGAC_FUNC_SNPRINTF_SIZE_T_SUPPORT test was broken by
      ce486056. Among others it made the UINT64_FORMAT macro to be
      defined in c.h, instead of directly being defined by configure.
      
      This lead to the replacement printf being used on all platforms for a
      while. Which seems to work, because this was only used due to
      different profiles ;)
      
      Fix by relying on INT64_MODIFIER instead.
      7e3f7283
  6. 17 Sep, 2014 1 commit
  7. 16 Sep, 2014 1 commit
    • Heikki Linnakangas's avatar
      Fix the return type of GIN triConsistent support functions to "char". · 77e65bf3
      Heikki Linnakangas authored
      They were marked to return a boolean, but they actually return a
      GinTernaryValue, which is more like a "char". It makes no practical
      difference, as the triConsistent functions cannot be called directly from
      SQL because they have "internal" arguments, but this nevertheless seems
      more correct.
      
      Also fix the GinTernaryValue name in the documentation. I renamed the enum
      earlier, but neglected the docs.
      
      Alexander Korotkov. This is new in 9.4, so backpatch there.
      77e65bf3
  8. 15 Sep, 2014 1 commit
  9. 14 Sep, 2014 3 commits
    • Heikki Linnakangas's avatar
      Fix pointer type in size passed to memset. · 2df465e6
      Heikki Linnakangas authored
      Pointers are all the same size, so it makes no practical difference, but
      let's be tidy.
      
      Found by Coverity, noted off-list by Tom Lane.
      2df465e6
    • Tom Lane's avatar
      Invent PGC_SU_BACKEND and mark log_connections/log_disconnections that way. · fe550b2a
      Tom Lane authored
      This new GUC context option allows GUC parameters to have the combined
      properties of PGC_BACKEND and PGC_SUSET, ie, they don't change after
      session start and non-superusers can't change them.  This is a more
      appropriate choice for log_connections and log_disconnections than their
      previous context of PGC_BACKEND, because we don't want non-superusers
      to be able to affect whether their sessions get logged.
      
      Note: the behavior for log_connections is still a bit odd, in that when
      a superuser attempts to set it from PGOPTIONS, the setting takes effect
      but it's too late to enable or suppress connection startup logging.
      It's debatable whether that's worth fixing, and in any case there is
      a reasonable argument for PGC_SU_BACKEND to exist.
      
      In passing, re-pgindent the files touched by this commit.
      
      Fujii Masao, reviewed by Joe Conway and Amit Kapila
      fe550b2a
    • Peter Eisentraut's avatar
      Run missing documentation tools through "missing" · c2a01439
      Peter Eisentraut authored
      Instead of just erroring out when a tool is missing, wrap the call with
      the "missing" script that we are already using for bison, flex, and
      perl, so that the users get a useful error message.
      c2a01439
  10. 13 Sep, 2014 3 commits
  11. 12 Sep, 2014 8 commits
    • Robert Haas's avatar
      Change NTUP_PER_BUCKET to 1 to improve hash join lookup speed. · 8cce08f1
      Robert Haas authored
      Since this makes the bucket headers use ~10x as much memory, properly
      account for that memory when we figure out whether everything fits
      in work_mem.  This might result in some cases that previously used
      only a single batch getting split into multiple batches, but it's
      unclear as yet whether we need defenses against that case, and if so,
      what the shape of those defenses should be.
      
      It's worth noting that even in these edge cases, users should still be
      no worse off than they would have been last week, because commit
      45f6240a saved a big pile of memory
      on exactly the same workloads.
      
      Tomas Vondra, reviewed and somewhat revised by me.
      8cce08f1
    • Fujii Masao's avatar
      Add GUC to enable logging of replication commands. · 4ad2a548
      Fujii Masao authored
      Previously replication commands like IDENTIFY_COMMAND were not logged
      even when log_statements is set to all. Some users who want to audit
      all types of statements were not satisfied with this situation. To
      address the problem, this commit adds new GUC log_replication_commands.
      If it's enabled, all replication commands are logged in the server log.
      
      There are many ways to allow us to enable that logging. For example,
      we can extend log_statement so that replication commands are logged
      when it's set to all. But per discussion in the community, we reached
      the consensus to add separate GUC for that.
      
      Reviewed by Ian Barwick, Robert Haas and Heikki Linnakangas.
      4ad2a548
    • Stephen Frost's avatar
      Add unicode_{column|header|border}_style to psql · a2dabf0e
      Stephen Frost authored
      With the unicode linestyle, this adds support to control if the
      column, header, or border style should be single or double line
      unicode characters.  The default remains 'single'.
      
      In passing, clean up the border documentation and address some
      minor formatting/spelling issues.
      
      Pavel Stehule, with some additional changes by me.
      a2dabf0e
    • Stephen Frost's avatar
      Handle border = 3 in expanded mode · 82962838
      Stephen Frost authored
      In psql, expanded mode was not being displayed correctly when using
      the normal ascii or unicode linestyles and border set to '3'.  Now,
      per the documentation, border '3' is really only sensible for HTML
      and LaTeX formats, however, that's no excuse for ascii/unicode to
      break in that case, and provisions had been made for psql to cleanly
      handle this case (and it did, in non-expanded mode).
      
      This was broken when ascii/unicode was initially added a good five
      years ago because print_aligned_vertical_line wasn't passed in the
      border setting being used by print_aligned_vertical but instead was
      given the whole printTableContent.  There really isn't a good reason
      for vertical_line to have the entire printTableContent structure, so
      just pass in the printTextFormat and border setting (similar to how
      this is handled in horizontal_line).
      
      Pointed out by Pavel Stehule, fix by me.
      
      Back-patch to all currently-supported versions.
      82962838
    • Heikki Linnakangas's avatar
      Support Subject Alternative Names in SSL server certificates. · acd08d76
      Heikki Linnakangas authored
      This patch makes libpq check the server's hostname against DNS names listed
      in the X509 subjectAltName extension field in the server certificate. This
      allows the same certificate to be used for multiple domain names. If there
      are no SANs in the certificate, the Common Name field is used, like before
      this patch. If both are given, the Common Name is ignored. That is a bit
      surprising, but that's the behavior mandated by the relevant RFCs, and it's
      also what the common web browsers do.
      
      This also adds a libpq_ngettext helper macro to allow plural messages to be
      translated in libpq. Apparently this happened to be the first plural message
      in libpq, so it was not needed before.
      
      Alexey Klyukin, with some kibitzing by me.
      acd08d76
    • Heikki Linnakangas's avatar
      Fix GIN data page split ratio calculation. · 774a78ff
      Heikki Linnakangas authored
      The code that tried to split a page at 75/25 ratio, when appending to the
      end of an index, was buggy in two ways. First, there was a silly typo that
      caused it to just fill the left page as full as possible. But the logic as
      it was intended wasn't correct either, and would actually have given a ratio
      closer to 60/40 than 75/25.
      
      Gaetano Mendola spotted the typo. Backpatch to 9.4, where this code was added.
      774a78ff
    • Tom Lane's avatar
      Fix power_var_int() for large integer exponents. · 1d352325
      Tom Lane authored
      The code for raising a NUMERIC value to an integer power wasn't very
      careful about large powers.  It got an outright wrong answer for an
      exponent of INT_MIN, due to failure to consider overflow of the Abs(exp)
      operation; which is fixable by using an unsigned rather than signed
      exponent value after that point.  Also, even though the number of
      iterations of the power-computation loop is pretty limited, it's easy for
      the repeated squarings to result in ridiculously enormous intermediate
      values, which can take unreasonable amounts of time/memory to process,
      or even overflow the internal "weight" field and so produce a wrong answer.
      We can forestall misbehaviors of that sort by bailing out as soon as the
      weight value exceeds what will fit in int16, since then the final answer
      must overflow (if exp > 0) or underflow (if exp < 0) the packed numeric
      format.
      
      Per off-list report from Pavel Stehule.  Back-patch to all supported
      branches.
      1d352325
    • Tom Lane's avatar
      Fix JSON regression tests. · e3ec0728
      Tom Lane authored
      Commit 95d737ff neglected to update
      expected/json_1.out.  Per buildfarm.
      e3ec0728