1. 14 Sep, 2021 1 commit
    • Tom Lane's avatar
      Send NOTIFY signals during CommitTransaction. · 0eff10a0
      Tom Lane authored
      Formerly, we sent signals for outgoing NOTIFY messages within
      ProcessCompletedNotifies, which was also responsible for sending
      relevant ones of those messages to our connected client.  It therefore
      had to run during the main-loop processing that occurs just before
      going idle.  This arrangement had two big disadvantages:
      
      * Now that procedures allow intra-command COMMITs, it would be
      useful to send NOTIFYs to other sessions immediately at COMMIT
      (though, for reasons of wire-protocol stability, we still shouldn't
      forward them to our client until end of command).
      
      * Background processes such as replication workers would not send
      NOTIFYs at all, since they never execute the client communication
      loop.  We've had requests to allow triggers running in replication
      workers to send NOTIFYs, so that's a problem.
      
      To fix these things, move transmission of outgoing NOTIFY signals
      into AtCommit_Notify, where it will happen during CommitTransaction.
      Also move the possible call of asyncQueueAdvanceTail there, to
      ensure we don't bloat the async SLRU if a background worker sends
      many NOTIFYs with no one listening.
      
      We can also drop the call of asyncQueueReadAllNotifications,
      allowing ProcessCompletedNotifies to go away entirely.  That's
      because commit 79002697 added a call of ProcessNotifyInterrupt
      adjacent to PostgresMain's call of ProcessCompletedNotifies,
      and that does its own call of asyncQueueReadAllNotifications,
      meaning that we were uselessly doing two such calls (inside two
      separate transactions) whenever inbound notify signals coincided
      with an outbound notify.  We need only set notifyInterruptPending
      to ensure that ProcessNotifyInterrupt runs, and we're done.
      
      The existing documentation suggests that custom background workers
      should call ProcessCompletedNotifies if they want to send NOTIFY
      messages.  To avoid an ABI break in the back branches, reduce it
      to an empty routine rather than removing it entirely.  Removal
      will occur in v15.
      
      Although the problems mentioned above have existed for awhile,
      I don't feel comfortable back-patching this any further than v13.
      There was quite a bit of churn in adjacent code between 12 and 13.
      At minimum we'd have to also backpatch 51004c71, and a good deal
      of other adjustment would also be needed, so the benefit-to-risk
      ratio doesn't look attractive.
      
      Per bug #15293 from Michael Powers (and similar gripes from others).
      
      Artur Zakirov and Tom Lane
      
      Discussion: https://postgr.es/m/153243441449.1404.2274116228506175596@wrigleys.postgresql.org
      0eff10a0
  2. 27 Aug, 2021 1 commit
  3. 09 Sep, 2020 1 commit
  4. 10 May, 2020 1 commit
    • Tom Lane's avatar
      Doc: marginal hacking to remove some PDF build warnings. · 336aa51b
      Tom Lane authored
      This patch eliminates a few more "exceed the available area" warnings
      whose causes aren't particularly connected to anything else.
      
      The only one really worthy of comment is that I increased the space
      allowed for an <orderedlist>'s numbers, because the default of 1em
      doesn't quite work for more than one digit.  The rest are one-off
      insertions of &zwsp; and suchlike tweaks, in places where they
      shouldn't do any damage to the material.  (In particular, although
      I split some long identifiers with zwsp's, there are other nearby
      occurrences of each one; so those changes shouldn't hurt greppability
      of the document sources.)
      336aa51b
  5. 06 Nov, 2019 1 commit
  6. 22 Apr, 2018 1 commit
  7. 03 Mar, 2018 1 commit
  8. 01 Mar, 2018 1 commit
  9. 24 Jan, 2018 1 commit
  10. 23 Nov, 2017 1 commit
    • Peter Eisentraut's avatar
      Convert documentation to DocBook XML · 3c49c6fa
      Peter Eisentraut authored
      Since some preparation work had already been done, the only source
      changes left were changing empty-element tags like <xref linkend="foo">
      to <xref linkend="foo"/>, and changing the DOCTYPE.
      
      The source files are still named *.sgml, but they are actually XML files
      now.  Renaming could be considered later.
      
      In the build system, the intermediate step to convert from SGML to XML
      is removed.  Everything is build straight from the source files again.
      The OpenSP (or the old SP) package is no longer needed.
      
      The documentation toolchain instructions are updated and are much
      simpler now.
      
      Peter Eisentraut, Alexander Lakhin, Jürgen Purtz
      3c49c6fa
  11. 17 Oct, 2017 1 commit
    • Peter Eisentraut's avatar
      Don't use SGML empty tags · c29c5789
      Peter Eisentraut authored
      For DocBook XML compatibility, don't use SGML empty tags (</>) anymore,
      replace by the full tag name.  Add a warning option to catch future
      occurrences.
      
      Alexander Lakhin, Jürgen Purtz
      c29c5789
  12. 29 Sep, 2017 1 commit
    • Peter Eisentraut's avatar
      Add background worker type · 5373bc2a
      Peter Eisentraut authored
      Add bgw_type field to background worker structure.  It is intended to be
      set to the same value for all workers of the same type, so they can be
      grouped in pg_stat_activity, for example.
      
      The backend_type column in pg_stat_activity now shows bgw_type for a
      background worker.  The ps listing also no longer calls out that a
      process is a background worker but just show the bgw_type.  That way,
      being a background worker is more of an implementation detail now that
      is not shown to the user.  However, most log messages still refer to
      'background worker "%s"'; otherwise constructing sensible and
      translatable log messages would become tricky.
      Reviewed-by: default avatarMichael Paquier <michael.paquier@gmail.com>
      Reviewed-by: default avatarDaniel Gustafsson <daniel@yesql.se>
      5373bc2a
  13. 01 Apr, 2017 1 commit
  14. 28 Jan, 2016 1 commit
  15. 06 Nov, 2015 1 commit
  16. 05 Nov, 2015 1 commit
    • Robert Haas's avatar
      Pass extra data to bgworkers, and use this to fix parallel contexts. · 64b2e7ad
      Robert Haas authored
      Up until now, the total amount of data that could be passed to a
      background worker at startup was one datum, which can be a small as
      4 bytes on some systems.  That's enough to pass a dsm_handle or an
      array index, but not much else.  Add a bgw_extra flag to the
      BackgroundWorker struct, allowing up to 128 bytes to be passed to
      a new worker on any platform.
      
      Use this to fix a problem I recently discovered with the parallel
      context machinery added in 9.5: the master assigns each worker an
      array index, and each worker subsequently assigns itself an array
      index, and there's nothing to guarantee that the two sets of indexes
      match, leading to chaos.
      
      Normally, I would not back-patch the change to add bgw_extra, since it
      is basically a feature addition.  However, since 9.5 is still in beta
      and there seems to be no other sensible way to repair the broken
      parallel context machinery, back-patch to 9.5.  Existing background
      worker code can ignore the bgw_extra field without a problem, but
      might need to be recompiled since the structure size has changed.
      
      Report and patch by me.  Review by Amit Kapila.
      64b2e7ad
  17. 29 Jul, 2015 1 commit
    • Robert Haas's avatar
      Flesh out the background worker documentation. · 38d4ce6b
      Robert Haas authored
      Make it more clear that bgw_main is usually not what you want.  Put the
      background worker flags in a variablelist rather than having them as
      part of a paragraph.  Explain important limits on how bgw_main_arg can
      be used.
      
      Craig Ringer, substantially revised by me.
      38d4ce6b
  18. 02 Feb, 2015 1 commit
  19. 14 Oct, 2014 1 commit
  20. 17 Jul, 2014 1 commit
  21. 07 May, 2014 1 commit
  22. 27 Feb, 2014 1 commit
  23. 18 Oct, 2013 1 commit
    • Robert Haas's avatar
      Provide a reliable mechanism for terminating a background worker. · 523beaa1
      Robert Haas authored
      Although previously-introduced APIs allow the process that registers a
      background worker to obtain the worker's PID, there's no way to prevent
      a worker that is not currently running from being restarted.  This
      patch introduces a new API TerminateBackgroundWorker() that prevents
      the background worker from being restarted, terminates it if it is
      currently running, and causes it to be unregistered if or when it is
      not running.
      
      Patch by me.  Review by Michael Paquier and KaiGai Kohei.
      523beaa1
  24. 04 Oct, 2013 1 commit
  25. 28 Aug, 2013 1 commit
    • Robert Haas's avatar
      Allow discovery of whether a dynamic background worker is running. · 090d0f20
      Robert Haas authored
      Using the infrastructure provided by this patch, it's possible either
      to wait for the startup of a dynamically-registered background worker,
      or to poll the status of such a worker without waiting.  In either
      case, the current PID of the worker process can also be obtained.
      As usual, worker_spi is updated to demonstrate the new functionality.
      
      Patch by me.  Review by Andres Freund.
      090d0f20
  26. 22 Jul, 2013 1 commit
    • Robert Haas's avatar
      Remove bgw_sighup and bgw_sigterm. · f40a318e
      Robert Haas authored
      Per discussion on pgsql-hackers, these aren't really needed.  Interim
      versions of the background worker patch had the worker starting with
      signals already unblocked, which would have made this necessary.
      But the final version does not, so we don't really need it; and it
      doesn't work well with the new facility for starting dynamic background
      workers, so just rip it out.
      
      Also per discussion on pgsql-hackers, back-patch this change to 9.3.
      It's best to get the API break out of the way before we do an
      official release of this facility, to avoid more pain for extension
      authors later.
      f40a318e
  27. 16 Jul, 2013 1 commit
    • Robert Haas's avatar
      Allow background workers to be started dynamically. · 7f7485a0
      Robert Haas authored
      There is a new API, RegisterDynamicBackgroundWorker, which allows
      an ordinary user backend to register a new background writer during
      normal running.  This means that it's no longer necessary for all
      background workers to be registered during processing of
      shared_preload_libraries, although the option of registering workers
      at that time remains available.
      
      When a background worker exits and will not be restarted, the
      slot previously used by that background worker is automatically
      released and becomes available for reuse.  Slots used by background
      workers that are configured for automatic restart can't (yet) be
      released without shutting down the system.
      
      This commit adds a new source file, bgworker.c, and moves some
      of the existing control logic for background workers there.
      Previously, there was little enough logic that it made sense to
      keep everything in postmaster.c, but not any more.
      
      This commit also makes the worker_spi contrib module into an
      extension and adds a new function, worker_spi_launch, which can
      be used to demonstrate the new facility.
      7f7485a0
  28. 04 Jul, 2013 2 commits
    • Robert Haas's avatar
      Add new GUC, max_worker_processes, limiting number of bgworkers. · 6bc8ef0b
      Robert Haas authored
      In 9.3, there's no particular limit on the number of bgworkers;
      instead, we just count up the number that are actually registered,
      and use that to set MaxBackends.  However, that approach causes
      problems for Hot Standby, which needs both MaxBackends and the
      size of the lock table to be the same on the standby as on the
      master, yet it may not be desirable to run the same bgworkers in
      both places.  9.3 handles that by failing to notice the problem,
      which will probably work fine in nearly all cases anyway, but is
      not theoretically sound.
      
      A further problem with simply counting the number of registered
      workers is that new workers can't be registered without a
      postmaster restart.  This is inconvenient for administrators,
      since bouncing the postmaster causes an interruption of service.
      Moreover, there are a number of applications for background
      processes where, by necessity, the background process must be
      started on the fly (e.g. parallel query).  While this patch
      doesn't actually make it possible to register new background
      workers after startup time, it's a necessary prerequisite.
      
      Patch by me.  Review by Michael Paquier.
      6bc8ef0b
    • Robert Haas's avatar
      docs: Clarify flag dependencies for background workers. · 5cbe935c
      Robert Haas authored
      BGWORKER_BACKEND_DATABASE_CONNECTION can only be used if
      BGWORKER_SHMEM_ACCESS is also used.
      
      Michael Paquier, with some tweaks by me.
      5cbe935c
  29. 24 Jan, 2013 1 commit
  30. 06 Dec, 2012 1 commit
    • Alvaro Herrera's avatar
      Background worker processes · da07a1e8
      Alvaro Herrera authored
      Background workers are postmaster subprocesses that run arbitrary
      user-specified code.  They can request shared memory access as well as
      backend database connections; or they can just use plain libpq frontend
      database connections.
      
      Modules listed in shared_preload_libraries can register background
      workers in their _PG_init() function; this is early enough that it's not
      necessary to provide an extra GUC option, because the necessary extra
      resources can be allocated early on.  Modules can install more than one
      bgworker, if necessary.
      
      Care is taken that these extra processes do not interfere with other
      postmaster tasks: only one such process is started on each ServerLoop
      iteration.  This means a large number of them could be waiting to be
      started up and postmaster is still able to quickly service external
      connection requests.  Also, shutdown sequence should not be impacted by
      a worker process that's reasonably well behaved (i.e. promptly responds
      to termination signals.)
      
      The current implementation lets worker processes specify their start
      time, i.e. at what point in the server startup process they are to be
      started: right after postmaster start (in which case they mustn't ask
      for shared memory access), when consistent state has been reached
      (useful during recovery in a HOT standby server), or when recovery has
      terminated (i.e. when normal backends are allowed).
      
      In case of a bgworker crash, actions to take depend on registration
      data: if shared memory was requested, then all other connections are
      taken down (as well as other bgworkers), just like it were a regular
      backend crashing.  The bgworker itself is restarted, too, within a
      configurable timeframe (which can be configured to be never).
      
      More features to add to this framework can be imagined without much
      effort, and have been discussed, but this seems good enough as a useful
      unit already.
      
      An elementary sample module is supplied.
      
      Author: Álvaro Herrera
      
      This patch is loosely based on prior patches submitted by KaiGai Kohei,
      and unsubmitted code by Simon Riggs.
      
      Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund,
      Heikki Linnakangas, Simon Riggs, Amit Kapila
      da07a1e8