1. 20 Sep, 2021 1 commit
  2. 09 Aug, 2021 1 commit
  3. 24 Jul, 2021 1 commit
  4. 12 Jul, 2021 1 commit
  5. 10 Jul, 2021 1 commit
    • Tom Lane's avatar
      Fix busted test for ldap_initialize. · ebc346e5
      Tom Lane authored
      Sigh ... I was expecting AC_CHECK_LIB to do something it didn't,
      namely update LIBS.  This led to not finding ldap_initialize.
      Fix by moving the probe for ldap_initialize.  In some sense this
      is more correct anyway, since (at least for now) we care about
      whether ldap_initialize exists in libldap not libldap_r.
      
      Per buildfarm member elver and local testing.
      
      Discussion: https://postgr.es/m/17083-a19190d9591946a7@postgresql.org
      ebc346e5
  6. 09 Jul, 2021 3 commits
    • Tom Lane's avatar
      Un-break AIX build, take 2. · 9ffad7ae
      Tom Lane authored
      I incorrectly diagnosed the reason why hoverfly is unhappy.
      Looking closer, it appears that it fails to link libldap
      unless libssl is also present; so the problem was my
      idea of clearing LIBS before making the check.  Revert
      to essentially the original coding, except that instead
      of failing when libldap_r isn't there, use libldap.
      
      Per buildfarm member hoverfly.
      
      Discussion: https://postgr.es/m/17083-a19190d9591946a7@postgresql.org
      9ffad7ae
    • Tom Lane's avatar
      Un-break AIX build. · 7f2eca6f
      Tom Lane authored
      In commit d0a02bdb8, I'd supposed that uniformly probing for
      ldap_bind would make the intent clearer.  However, that seems
      not to work on AIX, for obscure reasons (maybe it's a macro
      there?).  Revert to the former behavior of probing
      ldap_simple_bind for thread-safe cases and ldap_bind otherwise.
      
      Per buildfarm member hoverfly.
      
      Discussion: https://postgr.es/m/17083-a19190d9591946a7@postgresql.org
      7f2eca6f
    • Tom Lane's avatar
      Update configure's probe for libldap to work with OpenLDAP 2.5. · 5620ec83
      Tom Lane authored
      The separate libldap_r is gone and libldap itself is now always
      thread-safe.  Unfortunately there seems no easy way to tell by
      inspection whether libldap is thread-safe, so we have to take
      it on faith that libldap is thread-safe if there's no libldap_r.
      That should be okay, as it appears that libldap_r was a standard
      part of the installation going back at least 20 years.
      
      Report and patch by Adrian Ho.  Back-patch to all supported
      branches, since people might try to build any of them with
      a newer OpenLDAP.
      
      Discussion: https://postgr.es/m/17083-a19190d9591946a7@postgresql.org
      5620ec83
  7. 21 Jun, 2021 1 commit
  8. 17 May, 2021 1 commit
  9. 03 Apr, 2021 1 commit
    • Michael Paquier's avatar
      Refactor HMAC implementations · e6bdfd97
      Michael Paquier authored
      Similarly to the cryptohash implementations, this refactors the existing
      HMAC code into a single set of APIs that can be plugged with any crypto
      libraries PostgreSQL is built with (only OpenSSL currently).  If there
      is no such libraries, a fallback implementation is available.  Those new
      APIs are designed similarly to the existing cryptohash layer, so there
      is no real new design here, with the same logic around buffer bound
      checks and memory handling.
      
      HMAC has a dependency on cryptohashes, so all the cryptohash types
      supported by cryptohash{_openssl}.c can be used with HMAC.  This
      refactoring is an advantage mainly for SCRAM, that included its own
      implementation of HMAC with SHA256 without relying on the existing
      crypto libraries even if PostgreSQL was built with their support.
      
      This code has been tested on Windows and Linux, with and without
      OpenSSL, across all the versions supported on HEAD from 1.1.1 down to
      1.0.1.  I have also checked that the implementations are working fine
      using some sample results, a custom extension of my own, and doing
      cross-checks across different major versions with SCRAM with the client
      and the backend.
      
      Author: Michael Paquier
      Reviewed-by: Bruce Momjian
      Discussion: https://postgr.es/m/X9m0nkEJEzIPXjeZ@paquier.xyz
      e6bdfd97
  10. 22 Mar, 2021 1 commit
    • Tom Lane's avatar
      Remove useless configure probe for <lz4/lz4.h>. · 2c75f8a6
      Tom Lane authored
      This seems to have been just copied-and-pasted from some other
      header checks.  But our C code is entirely unprepared to support
      such a header name, so it's only wasting cycles to look for it.
      If we did need to support it, some #ifdefs would be required.
      
      (A quick trawl at codesearch.debian.net finds some packages that
      reference lz4/lz4.h; but they use *only* that spelling, and
      appear to be intending to reference their own copy rather than
      a system-level installation of liblz4.  There's no evidence of
      freestanding installations that require this spelling.)
      
      Discussion: https://postgr.es/m/457962.1616362509@sss.pgh.pa.us
      2c75f8a6
  11. 21 Mar, 2021 1 commit
    • Tom Lane's avatar
      Bring configure support for LZ4 up to snuff. · 4d399a6f
      Tom Lane authored
      It's not okay to just shove the pkg_config results right into our
      build flags, for a couple different reasons:
      
      * This fails to maintain the separation between CPPFLAGS and CFLAGS,
      as well as that between LDFLAGS and LIBS.  (The CPPFLAGS angle is,
      I believe, the reason for warning messages reported when building
      with MacPorts' liblz4.)
      
      * If pkg_config emits anything other than -I/-D/-L/-l switches,
      it's highly unlikely that we want to absorb those.  That'd be more
      likely to break the build than do anything helpful.  (Even the -D
      case is questionable; but we're doing that for libxml2, so I kept it.)
      
      Also, it's not okay to skip doing an AC_CHECK_LIB probe, as
      evidenced by recent build failure on topminnow; that should
      have been caught at configure time.
      
      Model fixes for this on configure's libxml2 support.
      
      It appears that somebody overlooked an autoheader run, too.
      
      Discussion: https://postgr.es/m/20210119190720.GL8560@telsasoft.com
      4d399a6f
  12. 19 Mar, 2021 2 commits
    • Thomas Munro's avatar
      Provide recovery_init_sync_method=syncfs. · 61752afb
      Thomas Munro authored
      Since commit 2ce439f3 we have opened every file in the data directory
      and called fsync() at the start of crash recovery.  This can be very
      slow if there are many files, leading to field complaints of systems
      taking minutes or even hours to begin crash recovery.
      
      Provide an alternative method, for Linux only, where we call syncfs() on
      every possibly different filesystem under the data directory.  This is
      equivalent, but avoids faulting in potentially many inodes from
      potentially slow storage.
      
      The new mode comes with some caveats, described in the documentation, so
      the default value for the new setting is "fsync", preserving the older
      behavior.
      Reported-by: default avatarMichael Brown <michael.brown@discourse.org>
      Reviewed-by: default avatarFujii Masao <masao.fujii@oss.nttdata.com>
      Reviewed-by: default avatarPaul Guo <guopa@vmware.com>
      Reviewed-by: default avatarBruce Momjian <bruce@momjian.us>
      Reviewed-by: default avatarJustin Pryzby <pryzby@telsasoft.com>
      Reviewed-by: default avatarDavid Steele <david@pgmasters.net>
      Discussion: https://postgr.es/m/11bc2bb7-ecb5-3ad0-b39f-df632734cd81%40discourse.org
      Discussion: https://postgr.es/m/CAEET0ZHGnbXmi8yF3ywsDZvb3m9CbdsGZgfTXscQ6agcbzcZAw%40mail.gmail.com
      61752afb
    • Robert Haas's avatar
      Allow configurable LZ4 TOAST compression. · bbe0a81d
      Robert Haas authored
      There is now a per-column COMPRESSION option which can be set to pglz
      (the default, and the only option in up until now) or lz4. Or, if you
      like, you can set the new default_toast_compression GUC to lz4, and
      then that will be the default for new table columns for which no value
      is specified. We don't have lz4 support in the PostgreSQL code, so
      to use lz4 compression, PostgreSQL must be built --with-lz4.
      
      In general, TOAST compression means compression of individual column
      values, not the whole tuple, and those values can either be compressed
      inline within the tuple or compressed and then stored externally in
      the TOAST table, so those properties also apply to this feature.
      
      Prior to this commit, a TOAST pointer has two unused bits as part of
      the va_extsize field, and a compessed datum has two unused bits as
      part of the va_rawsize field. These bits are unused because the length
      of a varlena is limited to 1GB; we now use them to indicate the
      compression type that was used. This means we only have bit space for
      2 more built-in compresison types, but we could work around that
      problem, if necessary, by introducing a new vartag_external value for
      any further types we end up wanting to add. Hopefully, it won't be
      too important to offer a wide selection of algorithms here, since
      each one we add not only takes more coding but also adds a build
      dependency for every packager. Nevertheless, it seems worth doing
      at least this much, because LZ4 gets better compression than PGLZ
      with less CPU usage.
      
      It's possible for LZ4-compressed datums to leak into composite type
      values stored on disk, just as it is for PGLZ. It's also possible for
      LZ4-compressed attributes to be copied into a different table via SQL
      commands such as CREATE TABLE AS or INSERT .. SELECT.  It would be
      expensive to force such values to be decompressed, so PostgreSQL has
      never done so. For the same reasons, we also don't force recompression
      of already-compressed values even if the target table prefers a
      different compression method than was used for the source data.  These
      architectural decisions are perhaps arguable but revisiting them is
      well beyond the scope of what seemed possible to do as part of this
      project.  However, it's relatively cheap to recompress as part of
      VACUUM FULL or CLUSTER, so this commit adjusts those commands to do
      so, if the configured compression method of the table happens not to
      match what was used for some column value stored therein.
      
      Dilip Kumar. The original patches on which this work was based were
      written by Ildus Kurbangaliev, and those were patches were based on
      even earlier work by Nikita Glukhov, but the design has since changed
      very substantially, since allow a potentially large number of
      compression methods that could be added and dropped on a running
      system proved too problematic given some of the architectural issues
      mentioned above; the choice of which specific compression method to
      add first is now different; and a lot of the code has been heavily
      refactored.  More recently, Justin Przyby helped quite a bit with
      testing and reviewing and this version also includes some code
      contributions from him. Other design input and review from Tomas
      Vondra, Álvaro Herrera, Andres Freund, Oleg Bartunov, Alexander
      Korotkov, and me.
      
      Discussion: http://postgr.es/m/20170907194236.4cefce96%40wp.localdomain
      Discussion: http://postgr.es/m/CAFiTN-uUpX3ck%3DK0mLEk-G_kUQY%3DSNOTeqdaNRR9FMdQrHKebw%40mail.gmail.com
      bbe0a81d
  13. 13 Mar, 2021 1 commit
  14. 10 Mar, 2021 1 commit
  15. 20 Feb, 2021 1 commit
  16. 01 Feb, 2021 1 commit
  17. 23 Jan, 2021 1 commit
    • Michael Paquier's avatar
      Introduce SHA1 implementations in the cryptohash infrastructure · a8ed6bb8
      Michael Paquier authored
      With this commit, SHA1 goes through the implementation provided by
      OpenSSL via EVP when building the backend with it, and uses as fallback
      implementation KAME which was located in pgcrypto and already shaped for
      an integration with a set of init, update and final routines.
      Structures and routines have been renamed to make things consistent with
      the fallback implementations of MD5 and SHA2.
      
      uuid-ossp has used for ages a shortcut with pgcrypto to fetch a copy of
      SHA1 if needed.  This was built depending on the build options within
      ./configure, so this cleans up some code and removes the build
      dependency between pgcrypto and uuid-ossp.
      
      Note that this will help with the refactoring of HMAC, as pgcrypto
      offers the option to use MD5, SHA1 or SHA2, so only the second option
      was missing to make that possible.
      
      Author: Michael Paquier
      Reviewed-by: Heikki Linnakangas
      Discussion: https://postgr.es/m/X9HXKTgrvJvYO7Oh@paquier.xyz
      a8ed6bb8
  18. 13 Jan, 2021 1 commit
  19. 11 Jan, 2021 1 commit
  20. 02 Jan, 2021 1 commit
  21. 30 Dec, 2020 1 commit
    • Tom Lane's avatar
      Use setenv() in preference to putenv(). · 7ca37fb0
      Tom Lane authored
      Since at least 2001 we've used putenv() and avoided setenv(), on the
      grounds that the latter was unportable and not in POSIX.  However,
      POSIX added it that same year, and by now the situation has reversed:
      setenv() is probably more portable than putenv(), since POSIX now
      treats the latter as not being a core function.  And setenv() has
      cleaner semantics too.  So, let's reverse that old policy.
      
      This commit adds a simple src/port/ implementation of setenv() for
      any stragglers (we have one in the buildfarm, but I'd not be surprised
      if that code is never used in the field).  More importantly, extend
      win32env.c to also support setenv().  Then, replace usages of putenv()
      with setenv(), and get rid of some ad-hoc implementations of setenv()
      wannabees.
      
      Also, adjust our src/port/ implementation of unsetenv() to follow the
      POSIX spec that it returns an error indicator, rather than returning
      void as per the ancient BSD convention.  I don't feel a need to make
      all the call sites check for errors, but the portability stub ought
      to match real-world practice.
      
      Discussion: https://postgr.es/m/2065122.1609212051@sss.pgh.pa.us
      7ca37fb0
  22. 10 Dec, 2020 1 commit
    • Michael Paquier's avatar
      Fix compilation of uuid-ossp · 525e60b7
      Michael Paquier authored
      This module had a dependency on pgcrypto's md5.c that got removed by
      b67b57a9.  Instead of the code from pgcrypto, this code can just use the
      new cryptohash routines for MD5 as a drop-in replacement, so let's just
      do this switch.  This has also the merit to simplify a bit the
      compilation of uuid-ossp.
      
      This requires --with-uuid to be reproduced, and I have used e2fs as a
      way to reproduce the failure, then test this commit.
      
      Per reports from buildfarm members longfin, florican and sifaka.
      
      Discussion: https://postgr.es/m/X9GToVd3QmWeNvj8@paquier.xyz
      525e60b7
  23. 30 Nov, 2020 1 commit
    • Tom Lane's avatar
      Remove configure-time probe for DocBook DTD. · 4823c4f6
      Tom Lane authored
      Checking for DocBook being installed was valuable when we were on the
      OpenSP docs toolchain, because that was rather hard to get installed
      fully.  Nowadays, as long as you have xmllint and xsltproc installed,
      you're good, because those programs will fetch the DocBook files off
      the net at need.  Moreover, testing this at configure time means that
      a network access may well occur whether or not you have any interest
      in building the docs later.  That can be slow (typically 2 or 3
      seconds, though much higher delays have been reported), and it seems
      not very nice to be doing an off-machine access without warning, too.
      
      Hence, drop the PGAC_CHECK_DOCBOOK probe, and adjust related
      documentation.  Without that macro, there's not much left of
      config/docbook.m4 at all, so I just removed it.
      
      Back-patch to v11, where we started to use xmllint in the
      PGAC_CHECK_DOCBOOK probe.
      
      Discussion: https://postgr.es/m/E2EE6B76-2D96-408A-B961-CAE47D1A86F0@yesql.se
      Discussion: https://postgr.es/m/A55A7FC9-FA60-47FE-98B5-139CDC57CE6E@gmail.com
      4823c4f6
  24. 20 Nov, 2020 2 commits
    • Magnus Hagander's avatar
      Remove ability to independently select random number generator · 16f96c74
      Magnus Hagander authored
      Remove the ability to select random number generator independently from
      SSL library. Instead, use the random number generator from the SSL
      library (today only OpenSSL supported) if one is configured. If no SSL
      library is configured, use the platform default (which means use
      CryptoAPI on Win32 and /dev/urandom on Linux).
      
      This also restructures pg_strong_random.c to have three clearly separate
      sections, one for each implementation, with two functions in each,
      instead of a scattered set of ifdefs throughout the whole file.
      
      Author: Daniel Gustafsson, Magnus Hagander, Michael Paquier
      Discussion: https://postgr.es/m/632623.1605460616@sss.pgh.pa.us
      16f96c74
    • Tom Lane's avatar
      On macOS, use -isysroot in link steps as well as compile steps. · 49407dc3
      Tom Lane authored
      We previously put the -isysroot switch only into CPPFLAGS, theorizing
      that it was only needed to find the right copies of include files.
      However, it seems that we also need to use it while linking programs,
      to find the right stub ".tbd" files for libraries.  We got away
      without that up to now, but apparently that was mostly luck.  It may
      also be that failures are only observed when the Xcode version is
      noticeably out of sync with the host macOS version; the case that's
      prompting action right now is that builds fail when using latest Xcode
      (12.2) on macOS Catalina, even though it's fine on Big Sur.
      
      Hence, add -isysroot to LDFLAGS as well.  (It seems that the more
      common practice is to put it in CFLAGS, whence it'd be included at
      both compile and link steps.  However, we can't mess with CFLAGS in
      the platform template file without confusing configure's logic for
      choosing default CFLAGS.)
      
      This should be back-patched, but first let's see if the buildfarm
      likes it on HEAD.
      
      Report and patch by James Hilliard (some cosmetic mods by me)
      
      Discussion: https://postgr.es/m/20201120003314.20560-1-james.hilliard1@gmail.com
      49407dc3
  25. 21 Oct, 2020 1 commit
    • Tom Lane's avatar
      Remove the option to build thread_test.c outside configure. · 8a212118
      Tom Lane authored
      Theoretically one could go into src/test/thread and build/run this
      program there.  In practice, that hasn't worked since 96bf88d5,
      and probably much longer on some platforms (likely including just
      the sort of hoary leftovers where this test might be of interest).
      While it wouldn't be too hard to repair the breakage, the fact that
      nobody has noticed for two years shows that there is zero usefulness
      in maintaining this build pathway.  Let's get rid of it and decree
      that thread_test.c is *only* meant to be built/used in configure.
      
      Given that decision, it makes sense to put thread_test.c under config/
      and get rid of src/test/thread altogether, so that's what I did.
      
      In passing, update src/test/README, which had been ignored by some
      not-so-recent additions of subdirectories.
      
      Discussion: https://postgr.es/m/227659.1603041612@sss.pgh.pa.us
      8a212118
  26. 09 Oct, 2020 1 commit
  27. 07 Sep, 2020 1 commit
  28. 24 Jul, 2020 1 commit
  29. 19 Jul, 2020 1 commit
  30. 14 Jul, 2020 1 commit
    • Peter Eisentraut's avatar
      Fix -Wcast-function-type warnings · de8feb1f
      Peter Eisentraut authored
      Three groups of issues needed to be addressed:
      
      load_external_function() and related functions returned PGFunction,
      even though not necessarily all callers are looking for a function of
      type PGFunction.  Since these functions are really just wrappers
      around dlsym(), change to return void * just like dlsym().
      
      In dynahash.c, we are using strlcpy() where a function with a
      signature like memcpy() is expected.  This should be safe, as the new
      comment there explains, but the cast needs to be augmented to avoid
      the warning.
      
      In PL/Python, methods all need to be cast to PyCFunction, per Python
      API, but this now runs afoul of these warnings.  (This issue also
      exists in core CPython.)
      
      To fix the second and third case, we add a new type pg_funcptr_t that
      is defined specifically so that gcc accepts it as a special function
      pointer that can be cast to any other function pointer without the
      warning.
      
      Also add -Wcast-function-type to the standard warning flags, subject
      to configure check.
      Reviewed-by: default avatarTom Lane <tgl@sss.pgh.pa.us>
      Discussion: https://www.postgresql.org/message-id/flat/1e97628e-6447-b4fd-e230-d109cec2d584%402ndquadrant.com
      de8feb1f
  31. 07 Jun, 2020 1 commit
  32. 18 May, 2020 1 commit
  33. 13 May, 2020 1 commit
  34. 12 May, 2020 1 commit
  35. 25 Mar, 2020 1 commit
  36. 17 Mar, 2020 1 commit
    • Tom Lane's avatar
      Use pkg-config, if available, to locate libxml2 during configure. · 0bc8cebd
      Tom Lane authored
      If pkg-config is installed and knows about libxml2, use its information
      rather than asking xml2-config.  Otherwise proceed as before.  This
      patch allows "configure --with-libxml" to succeed on platforms that
      have pkg-config but not xml2-config, which is likely to soon become
      a typical situation.
      
      The old mechanism can be forced by setting XML2_CONFIG explicitly
      (hence, build processes that were already doing so will certainly
      not need adjustment).  Also, it's now possible to set XML2_CFLAGS
      and XML2_LIBS explicitly to override both programs.
      
      There is a small risk of this breaking existing build processes,
      if there are multiple libxml2 installations on the machine and
      pkg-config disagrees with xml2-config about which to use.  The
      only case where that seems really likely is if a builder has tried
      to select a non-default xml2-config by putting it early in his PATH
      rather than setting XML2_CONFIG.  Plan to warn against that in the
      minor release notes.
      
      Back-patch to v10; before that we had no pkg-config infrastructure,
      and it doesn't seem worth adding it for this.
      
      Hugh McMaster and Tom Lane; Peter Eisentraut also made an earlier
      attempt at this, from which I lifted most of the docs changes.
      
      Discussion: https://postgr.es/m/CAN9BcdvfUwc9Yx5015bLH2TOiQ-M+t_NADBSPhMF7dZ=pLa_iw@mail.gmail.com
      0bc8cebd