From pgsql-hackers-owner+M49851=pgman=candle.pha.pa.us@postgresql.org Sat Feb  7 10:33:51 2004
Return-path: <pgsql-hackers-owner+M49851=pgman=candle.pha.pa.us@postgresql.org>
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id i17HXbr05526
	for <pgman@candle.pha.pa.us>; Sat, 7 Feb 2004 12:33:50 -0500 (EST)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.2])
	by svr1.postgresql.org (Postfix) with ESMTP id 0FDCBD1D8A0
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Wed,  4 Feb 2004 20:22:23 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
	with ESMTP id 35031-10
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
	Wed,  4 Feb 2004 16:22:21 -0400 (AST)
Received: from main.gmane.org (main.gmane.org [80.91.224.249])
	by svr1.postgresql.org (Postfix) with ESMTP id D9726D1D85F
	for <pgsql-hackers@postgresql.org>; Wed,  4 Feb 2004 16:22:20 -0400 (AST)
Received: from list by main.gmane.org with local (Exim 3.35 #1 (Debian))
	id 1AoTXQ-0006tA-00
	for <pgsql-hackers@postgresql.org>; Wed, 04 Feb 2004 21:22:20 +0100
Received: from kosh.hut.fi ([130.233.228.10])
        by main.gmane.org with esmtp (Gmexim 0.1 (Debian))
        id 1AlnuQ-0007hv-00
        for <pgsql-hackers@postgresql.org>; Wed Feb  4 20:22:20 2004
Received: from hlinnaka by kosh.hut.fi with local (Gmexim 0.1 (Debian))
        id 1AlnuQ-0007hv-00
        for <pgsql-hackers@postgresql.org>; Wed Feb  4 20:22:20 2004
X-Injected-Via-Gmane: http://gmane.org/
To: pgsql-hackers@postgresql.org
From: Heikki Linnakangas <hlinnaka@iki.fi>
Subject: [HACKERS] Two-phase commit
Date: Wed, 4 Feb 2004 22:22:16 +0200
Lines: 30
Message-ID: <Pine.OSF.4.58.0402042200330.238747@kosh.hut.fi>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Complaints-To: usenet@sea.gmane.org
X-Gmane-NNTP-Posting-Host: kosh.hut.fi
X-X-Sender: hlinnaka@kosh.hut.fi
X-Virus-Scanned: by amavisd-new at postgresql.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on 
	candle.pha.pa.us
X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham 
	version=2.61
Status: ORr

I've been very slowly continuing my work on two-phase commits for a couple
months now, and I now have my original patch updated so that it applies to
the current CVS tip, with some improvements.

The patch introduces three new commands, PREPCOMMIT, COMMITPREPARED and
ABORTPREPARED.

To start a 2PC transaction, you first do a BEGIN and your updates as
usual. At the end of the transaction, you call PREPCOMMIT 'foobar' instead
of COMMIT. Now the transaction is in prepared state, ready to commit at a
later time. 'foobar' is the global transaction identifier assigned for the
transaction.

Later, when you want to finish the second phase, you call
COMMITPREPARED 'foobar';

There is a system view pg_prepared_xacts that gives you all transactions
that are in prepared state waiting for COMMITPREPARED or ABORTPREPARED.

I have also done some work on XA-enabling the JDBC drivers, now that we
have what it takes in the server side. I have succesfully executed
2PC transactions with JBossMQ and Postgres, using JBoss as the
transaction manager, so the basic stuff seems to be working.

Please have a look and comment, the patches can be found here:
http://www.iki.fi/hlinnaka/pgsql/

What is the schedule for 7.5? Any chance of getting this in?

- Heikki


---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
      subscribe-nomail command to majordomo@postgresql.org so that your
      message can get through to the mailing list cleanly

From pgsql-hackers-owner+M50166=pgman=candle.pha.pa.us@postgresql.org Sun Feb 15 14:10:46 2004
Return-path: <pgsql-hackers-owner+M50166=pgman=candle.pha.pa.us@postgresql.org>
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id i1FLAir14581
	for <pgman@candle.pha.pa.us>; Sun, 15 Feb 2004 16:10:45 -0500 (EST)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.2])
	by svr1.postgresql.org (Postfix) with ESMTP id 297AAD1CAF3
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon,  9 Feb 2004 20:09:53 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
	with ESMTP id 07202-01
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
	Mon,  9 Feb 2004 16:09:37 -0400 (AST)
Received: from smtp-4.hut.fi (smtp-4.hut.fi [130.233.228.94])
	by svr1.postgresql.org (Postfix) with ESMTP id 90D81D1C4BC
	for <pgsql-hackers@postgresql.org>; Mon,  9 Feb 2004 16:09:34 -0400 (AST)
Received: from kosh.hut.fi (kosh.hut.fi [130.233.228.10])
	by smtp-4.hut.fi (8.12.10/8.12.10) with ESMTP id i19K9Y66012417;
	Mon, 9 Feb 2004 22:09:34 +0200
Date: Mon, 9 Feb 2004 22:09:34 +0200 (EET)
From: Heikki Linnakangas <hlinnaka@iki.fi>
X-X-Sender: hlinnaka@kosh.hut.fi
To: "Jeroen T. Vermeulen" <jtv@xs4all.nl>
cc: PostgreSQL Development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Two-phase commit
In-Reply-To: <20040208195332.GE25140@xs4all.nl>
Message-ID: <Pine.OSF.4.58.0402092201460.226205@kosh.hut.fi>
References: <Pine.OSF.4.58.0402042200330.238747@kosh.hut.fi>
	<20040208195332.GE25140@xs4all.nl>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-RAVMilter-Version: 8.4.3(snapshot 20030212) (smtp-4.hut.fi)
X-Virus-Scanned: by amavisd-new at postgresql.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on 
	candle.pha.pa.us
X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham 
	version=2.61
Status: OR

On Sun, 8 Feb 2004, Jeroen T. Vermeulen wrote:

> On Wed, Feb 04, 2004 at 10:22:16PM +0200, Heikki Linnakangas wrote:
>
> > There is a system view pg_prepared_xacts that gives you all transactions
> > that are in prepared state waiting for COMMITPREPARED or ABORTPREPARED.
>
> Great to hear that you've gotten so far with this...  One question: can I
> check for this view to see if 2PC is supported before issuing the new
> kind of commit?  I'm interested in supporting 2PC even for some regular
> transactions to reduce their in-doubt window, but I don't want to issue a
> command at the last moment that may fail (and thereby abort) because the
> backend version I'm connected to doesn't support the new command!

Yes, I suppose that would work. Though you would have to use a query that
wouldn't fail in case the view doesn't exist, otherwise you end up
aborting the transaction anyway. This should work:

SELECT COUNT(*) FROM pg_views WHERE schemanem='pg_catalog' AND viewname
='pg_prepared_xacts'

If it returns 1, you can do 2PC, if it returns 0, you have to regular
commit.

However, if this gets into 7.5, I guess you could just check for the
version of the backend instead with "SELECT version()".

- Heikki

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

From pgsql-hackers-owner+M49946=pgman=candle.pha.pa.us@postgresql.org Mon Feb  9 14:18:20 2004
Return-path: <pgsql-hackers-owner+M49946=pgman=candle.pha.pa.us@postgresql.org>
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id i19LIHr27947
	for <pgman@candle.pha.pa.us>; Mon, 9 Feb 2004 16:18:18 -0500 (EST)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.2])
	by svr1.postgresql.org (Postfix) with ESMTP id 5B8CBD1C9E1
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon,  9 Feb 2004 21:00:06 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
	with ESMTP id 25216-09
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
	Mon,  9 Feb 2004 16:59:56 -0400 (AST)
Received: from smtp-out4.xs4all.nl (smtp-out4.xs4all.nl [194.109.24.5])
	by svr1.postgresql.org (Postfix) with ESMTP id 0381AD1C4BC
	for <pgsql-hackers@postgresql.org>; Mon,  9 Feb 2004 16:59:53 -0400 (AST)
Received: from xs1.xs4all.nl (xs1.xs4all.nl [194.109.21.2])
	by smtp-out4.xs4all.nl (8.12.10/8.12.10) with ESMTP id i19Kw1Oh062440;
	Mon, 9 Feb 2004 21:59:41 +0100 (CET)
Received: from xs1.xs4all.nl (jtv@localhost.xs4all.nl [127.0.0.1])
	by xs1.xs4all.nl (8.12.10/8.12.9) with ESMTP id i19KuUBT033906;
	Mon, 9 Feb 2004 21:56:30 +0100 (CET)
	(envelope-from jtv@xs4all.nl)
Received: (from jtv@localhost)
	by xs1.xs4all.nl (8.12.10/8.12.9/Submit) id i19KuUgt033905;
	Mon, 9 Feb 2004 21:56:30 +0100 (CET)
	(envelope-from jtv)
Date: Mon, 9 Feb 2004 21:56:30 +0100
From: "Jeroen T. Vermeulen" <jtv@xs4all.nl>
To: Heikki Linnakangas <hlinnaka@iki.fi>
cc: PostgreSQL Development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Two-phase commit
Message-ID: <20040209205629.GB13454@xs4all.nl>
Mail-Followup-To: Heikki Linnakangas <hlinnaka@iki.fi>,
	PostgreSQL Development <pgsql-hackers@postgresql.org>
References: <Pine.OSF.4.58.0402042200330.238747@kosh.hut.fi> <20040208195332.GE25140@xs4all.nl> <Pine.OSF.4.58.0402092201460.226205@kosh.hut.fi>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Pine.OSF.4.58.0402092201460.226205@kosh.hut.fi>
User-Agent: Mutt/1.4.1i
X-Virus-Scanned: by amavisd-new at postgresql.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on 
	candle.pha.pa.us
X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham 
	version=2.61
Status: OR

On Mon, Feb 09, 2004 at 10:09:34PM +0200, Heikki Linnakangas wrote:
> 
> However, if this gets into 7.5, I guess you could just check for the
> version of the backend instead with "SELECT version()".

Hey, that works?  That's very good news, because I was getting a bit
worried about all the things I want to do in libpqxx that may depend on
the Postgres version...

Thanks!


---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
      subscribe-nomail command to majordomo@postgresql.org so that your
      message can get through to the mailing list cleanly

From pgsql-hackers-owner+M50540=pgman=candle.pha.pa.us@postgresql.org Sat Feb 28 14:40:52 2004
Return-path: <pgsql-hackers-owner+M50540=pgman=candle.pha.pa.us@postgresql.org>
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id i1SLeoD13679
	for <pgman@candle.pha.pa.us>; Sat, 28 Feb 2004 16:40:50 -0500 (EST)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.2])
	by svr1.postgresql.org (Postfix) with ESMTP id A0F5BD1E1AA
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Sun, 22 Feb 2004 02:27:19 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
	with ESMTP id 92481-04
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
	Sat, 21 Feb 2004 22:27:18 -0400 (AST)
Received: from main.gmane.org (main.gmane.org [80.91.224.249])
	by svr1.postgresql.org (Postfix) with ESMTP id 54A72D1E197
	for <pgsql-hackers@postgresql.org>; Sat, 21 Feb 2004 22:27:11 -0400 (AST)
Received: from list by main.gmane.org with local (Exim 3.35 #1 (Debian))
	id 1AujKe-0004i5-00
	for <pgsql-hackers@postgresql.org>; Sun, 22 Feb 2004 03:27:00 +0100
Received: from kosh.hut.fi ([130.233.228.10])
        by main.gmane.org with esmtp (Gmexim 0.1 (Debian))
        id 1AlnuQ-0007hv-00
        for <pgsql-hackers@postgresql.org>; Sun Feb 22 02:27:00 2004
Received: from hlinnaka by kosh.hut.fi with local (Gmexim 0.1 (Debian))
        id 1AlnuQ-0007hv-00
        for <pgsql-hackers@postgresql.org>; Sun Feb 22 02:27:00 2004
X-Injected-Via-Gmane: http://gmane.org/
To: pgsql-hackers@postgresql.org
From: Heikki Linnakangas <hlinnaka@iki.fi>
Subject: Re: [HACKERS] Two-phase commit
Date: Sun, 22 Feb 2004 04:26:56 +0200
Lines: 70
Message-ID: <Pine.OSF.4.58.0402220324250.126984@kosh.hut.fi>
References: <Pine.OSF.4.58.0402042200330.238747@kosh.hut.fi>
	<200402080138.i181cPl15259@candle.pha.pa.us>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Complaints-To: usenet@sea.gmane.org
X-Gmane-NNTP-Posting-Host: kosh.hut.fi
X-X-Sender: hlinnaka@kosh.hut.fi
In-Reply-To: <200402080138.i181cPl15259@candle.pha.pa.us>
X-Virus-Scanned: by amavisd-new at postgresql.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on 
	candle.pha.pa.us
X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham 
	version=2.61
Status: OR

On Sat, 7 Feb 2004, Bruce Momjian wrote:

> > Please have a look and comment, the patches can be found here:
> > http://www.iki.fi/hlinnaka/pgsql/
> >
> > What is the schedule for 7.5? Any chance of getting this in?
>
> 7.5 is certainly possible.  We are months away from beta on 7.5 and I
> would like ot see two-phase commit included.  One item that has come up
> in past discussions is a way of recording two-phase commit failures to
> the administrator in cases where you precommit, get a reply, commit,
> then the remote machine disappears.

You would resolve this by opening a new session, and checking if the gid
you specified in PREPARE TRANSACTION is still present in the
pg_prepared_xacts view. It could be done manually by the administrator, or
it could be done automatically by an external transaction manager if
there is one.

The XA interface specifies a function called "recover", that gives you a
list of pending transactions. If we some day have an XA implementation,
the recover call would map directly to "SELECT gid FROM
pg_prepared_xacts". The JDBC XA implementation that I'm working on does
that already.

I have updated my patches, see the URL above. I renamed the commands to
PREPARE TRANSACTION, COMMIT PREPARED and ROLLBACK PREPARED. I think it's
more coherent that way.

I also added documentation entries for the commands, and a basic
regression test.

I went through all the AtCommit_* and AtEOXact* hooks in xact.c to find
any possible problem areas. The following items have not yet been
implemented and throw an error if you try to do 2PC in the same
transaction.

* Notifications (NOTIFY/LISTEN). All pending notifications should be
stored in persistent storage in the prepare phase, and sent in the commit
phase.

* Creation/deletion of relations. I couldn't figure out how the relation
cache invalidation stuff should work with 2PC.

* Modifying GUC variables. I need to study the GUC code more thoroughly
before I can tell what needs to be done.

* Updates to shadow/group files, that is, CREATE USER and friends. Needs
some tricks to delay the writing of pg_pwd/pg_group.

* Large objects. AFAICS, no particular problem here, but I'd like to deal
with them later when the more important stuff are ok.

Plus a couple of minor details:

* Temporary tables. The seem to work somehow, but I haven't tested them
much. I have a feeling that nasty things might happen if you commit the
prepared transaction from another backend etc.

* initdb gives a warning about a missing file. It's harmless, but I
don't see how to detect that you're running under initdb. Also, if you
try to prapare a transaction with a global transaction identifier that's
already in use, you first get a warning and then an error.

I'm going to tackle the above problems later, but I would like to get
this applied to the cvs trunk with the current functionality first, after
discussion of course. The rest are nice to have for the sake of
completeness but probably not necessary for most users.

- Heikki


---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

               http://www.postgresql.org/docs/faqs/FAQ.html

From pgsql-general-owner+M58726=pgman=candle.pha.pa.us@postgresql.org Fri Mar 19 01:00:58 2004
Return-path: <pgsql-general-owner+M58726=pgman=candle.pha.pa.us@postgresql.org>
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id i2J80ud15620
	for <pgman@candle.pha.pa.us>; Fri, 19 Mar 2004 03:00:57 -0500 (EST)
X-Original-To: pgsql-general-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.2])
	by svr1.postgresql.org (Postfix) with ESMTP id 53189D1BB74
	for <pgsql-general-postgresql.org@localhost.postgresql.org>; Fri, 19 Mar 2004 07:31:13 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
	with ESMTP id 87439-01
	for <pgsql-general-postgresql.org@localhost.postgresql.org>;
	Fri, 19 Mar 2004 03:31:05 -0400 (AST)
Received: from builder.localdomain (unknown [61.94.124.129])
	by svr1.postgresql.org (Postfix) with SMTP id 12CC3D1B80B
	for <pgsql-general@postgresql.org>; Fri, 19 Mar 2004 03:30:59 -0400 (AST)
Received: (qmail 11710 invoked from network); 19 Mar 2004 07:30:55 -0000
Received: from unknown (HELO zara.6.isreserved.com) (192.168.0.113)
  by builder.localdomain with SMTP; 19 Mar 2004 07:30:55 -0000
Message-ID: <405AA1B0.3080600@zara.6.isreserved.com>
Date: Fri, 19 Mar 2004 14:30:56 +0700
From: David Garamond <lists@zara.6.isreserved.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031208
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Matthew Hixson <hixson@poindextrose.org>
cc: pgsql-general@postgresql.org
Subject: Re: [GENERAL] two phase commit
References: <68FB14C4-7963-11D8-B6BA-000A95D05926@poindextrose.org>
In-Reply-To: <68FB14C4-7963-11D8-B6BA-000A95D05926@poindextrose.org>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
X-Virus-Scanned: by amavisd-new at postgresql.org
X-Mailing-List: pgsql-general
Precedence: bulk
Sender: pgsql-general-owner@postgresql.org
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on 
	candle.pha.pa.us
X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham 
	version=2.61
Status: OR

Matthew Hixson wrote:
>  From what I can find with Google it looks like TPC might make it into 
> Postgrs 7.5.  I was wondering if there were any commercial products or 
> open source projects that can provide TPC on top of Postgres.  Or is 
> this something that really needs to be handled by the database itself?

With the 2PC patch submitted, you can interface with it at the SQL level 
(i.e. you prepare, commit, cancel prepare, etc. by issuing ordinary SQL 
commands). This means it will be pretty trivial to make it work with 
other components like TX manager, etc. For example, I'm guessing it will 
be easy to add 2PC support to npgsql (.NET data provider).

PS: It's really nice to see Postgres coming along feature-wise. 2PC and 
nested transaction (aside from native Windows port, of course) are the 
main things Firebird has that Potsgres hasn't. After that, we can pretty 
much leave FB in the dust... :-)

-- 
dave

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
      subscribe-nomail command to majordomo@postgresql.org so that your
      message can get through to the mailing list cleanly

From pgsql-hackers-owner+M51632@postgresql.org Tue Mar 23 10:18:06 2004
Return-path: <pgsql-hackers-owner+M51632@postgresql.org>
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id i2NHI0d16063
	for <pgman@candle.pha.pa.us>; Tue, 23 Mar 2004 12:18:02 -0500 (EST)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.2])
	by svr1.postgresql.org (Postfix) with ESMTP
	id BFD64D1EBB0; Tue, 23 Mar 2004 16:11:15 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
	with ESMTP id 35500-05; Tue, 23 Mar 2004 12:11:00 -0400 (AST)
Received: from smtp-4.hut.fi (smtp-4.hut.fi [130.233.228.94])
	by svr1.postgresql.org (Postfix) with ESMTP
	id 411C3D1E2D7; Tue, 23 Mar 2004 12:10:58 -0400 (AST)
Received: from kosh.hut.fi (kosh.hut.fi [130.233.228.10])
	by smtp-4.hut.fi (8.12.10/8.12.10) with ESMTP id i2NGAZTZ028569;
	Tue, 23 Mar 2004 18:10:45 +0200
Date: Tue, 23 Mar 2004 18:10:35 +0200 (EET)
From: Heikki Linnakangas <hlinnaka@iki.fi>
X-X-Sender: hlinnaka@kosh.hut.fi
To: pgsql-hackers@postgresql.org
cc: pgsql-patches@postgresql.org
Subject: Re: [HACKERS] Two-phase commit
In-Reply-To: <Pine.OSF.4.58.0402220324250.126984@kosh.hut.fi>
Message-ID: <Pine.OSF.4.58.0403231758350.513267@kosh.hut.fi>
References: <Pine.OSF.4.58.0402042200330.238747@kosh.hut.fi>
	<200402080138.i181cPl15259@candle.pha.pa.us> <Pine.OSF.4.58.0402220324250.126984@kosh.hut.fi>
MIME-Version: 1.0
Content-Type: MULTIPART/MIXED; BOUNDARY="182774146-1920606540-1080058235=:513267"
X-RAVMilter-Version: 8.4.3(snapshot 20030212) (smtp-4.hut.fi)
X-Virus-Scanned: by amavisd-new at postgresql.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on 
	candle.pha.pa.us
X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham 
	version=2.61
Status: ORr

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.
  Send mail to mime@docserver.cac.washington.edu for more info.

--182774146-1920606540-1080058235=:513267
Content-Type: TEXT/PLAIN; charset=US-ASCII

I have again updated my two-phase commit patches. Only minor
modifications.

I haven't received any comments and there hasn't been any discussion on
the implementation, I suppose that nobody has given it a try. :(

There is still some rough edges, but I think it's good enough as a first
cut. I personally consider it ready to be applied to cvs tip, but then
again I'm still a newbie :). Please take a look!

The patch is also available here: http://www.iki.fi/hlinnaka/pgsql/

- Heikki
--182774146-1920606540-1080058235=:513267
Content-Type: APPLICATION/x-gzip; name="twophase_20040321.diff.gz"
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.OSF.4.58.0403231810351.513267@kosh.hut.fi>
Content-Description: 
Content-Disposition: attachment; filename="twophase_20040321.diff.gz"

[encoded data, 32757 characters]
JDZD3rXwWv0AOdZM0cui6/ONpU/fU5qUEvPMFPTMEPX8f0brErRBgQEA

--182774146-1920606540-1080058235=:513267
Content-Type: text/plain
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0


---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
      joining column's datatypes do not match

--182774146-1920606540-1080058235=:513267--

From pgsql-hackers-owner+M51641@postgresql.org Tue Mar 23 11:32:25 2004
Return-path: <pgsql-hackers-owner+M51641@postgresql.org>
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id i2NIWId29116
	for <pgman@candle.pha.pa.us>; Tue, 23 Mar 2004 13:32:24 -0500 (EST)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.2])
	by svr1.postgresql.org (Postfix) with ESMTP id 28F60D1EC42
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Tue, 23 Mar 2004 18:28:57 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
	with ESMTP id 11478-09
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
	Tue, 23 Mar 2004 14:28:55 -0400 (AST)
Received: from alvh.no-ip.org (200.85.202.54.DSL.surnet.cl [200.85.202.54])
	by svr1.postgresql.org (Postfix) with ESMTP id 7D042D1EC0D
	for <pgsql-hackers@postgresql.org>; Tue, 23 Mar 2004 14:28:54 -0400 (AST)
Received: by alvh.no-ip.org (Postfix, from userid 500)
	id 8877D5759D; Tue, 23 Mar 2004 13:28:43 -0500 (EST)
Date: Tue, 23 Mar 2004 14:28:43 -0400
From: Alvaro Herrera <alvherre@dcc.uchile.cl>
To: Heikki Linnakangas <hlinnaka@iki.fi>
cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Two-phase commit
Message-ID: <20040323182843.GF3863@dcc.uchile.cl>
References: <Pine.OSF.4.58.0402042200330.238747@kosh.hut.fi> <200402080138.i181cPl15259@candle.pha.pa.us> <Pine.OSF.4.58.0402220324250.126984@kosh.hut.fi> <Pine.OSF.4.58.0403231758350.513267@kosh.hut.fi>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Pine.OSF.4.58.0403231758350.513267@kosh.hut.fi>
User-Agent: Mutt/1.4.1i
X-Virus-Scanned: by amavisd-new at postgresql.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on 
	candle.pha.pa.us
X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham 
	version=2.61
Status: ORr

On Tue, Mar 23, 2004 at 06:10:35PM +0200, Heikki Linnakangas wrote:
> I have again updated my two-phase commit patches. Only minor
> modifications.
> 
> I haven't received any comments and there hasn't been any discussion on
> the implementation, I suppose that nobody has given it a try. :(

I haven't tried it, but I see it conflicts big time with my
modifications in access/transam/xact.c for subtransactions support.

I am currently writing a proposal for nested transactions which will go
to -hackers, and I will be posting some code to -patches shortly
thereafter which should give you an idea where I am heading.  Maybe then
we can have the opinion from the devel community about both things,
whether they should be applied or not.

-- 
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"El miedo atento y previsor es la madre de la seguridad" (E. Burke)

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

               http://archives.postgresql.org

From jtv@xs4all.nl Tue Jun 29 10:18:43 2004
Return-path: <jtv@xs4all.nl>
Received: from smtp-out3.xs4all.nl (smtp-out3.xs4all.nl [194.109.24.13])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id i5TEIeC19886
	for <pgman@candle.pha.pa.us>; Tue, 29 Jun 2004 10:18:42 -0400 (EDT)
Received: from xs1.xs4all.nl (xs1.xs4all.nl [194.109.21.2])
	by smtp-out3.xs4all.nl (8.12.10/8.12.10) with ESMTP id i5TEIcdQ066827;
	Tue, 29 Jun 2004 16:18:38 +0200 (CEST)
Received: from xs1.xs4all.nl (jtv@localhost.xs4all.nl [127.0.0.1])
	by xs1.xs4all.nl (8.12.10/8.12.10) with ESMTP id i5TEIcfr057746;
	Tue, 29 Jun 2004 16:18:38 +0200 (CEST)
	(envelope-from jtv@xs4all.nl)
Received: (from jtv@localhost)
	by xs1.xs4all.nl (8.12.10/8.12.9/Submit) id i5TEIcD1057745;
	Tue, 29 Jun 2004 16:18:38 +0200 (CEST)
	(envelope-from jtv)
Date: Tue, 29 Jun 2004 16:18:38 +0200
From: "Jeroen T. Vermeulen" <jtv@xs4all.nl>
To: pgadmin@pse-consulting.de, Bruce Momjian <pgman@candle.pha.pa.us>
Subject: Two-phase commit
Message-ID: <20040629141837.GL56428@xs4all.nl>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.4.1i
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on 
	candle.pha.pa.us
X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham 
	version=2.61
Status: ORr

Hi Andreas, Bruce:

I briefly discussed this with both of you separately.  The 2-phase-commit
patch that's being worked on has the shortcoming that the "supercommit"
is assumed to come over the same connection as the regular database
session, i.e. the transaction manager (the middleware that manages the
two-phase transaction across multibple databases etc.) also has to serve
as a proxy for the regular database session.

A "real" two-phase commit system has the two phases of the transaction
going over separate connections--one between application and database
server, one between middleware and database server.  Andreas quite
sensibly asked me why; and the real answer only just popped back into
my head.

I think the real reason is the same reason why we want 2PC in the first
place.  Connections may be lost _just_ when the middleware is sending out
second-phase commits to the various resource managers.  In that case, it
is absolutely essential that the transaction manager be able to reconnect
to the resource manager (in this case, postgres) and continue where it
left off.  A regular session can't do that, and that's exactly where the
2nd phase is being implemented right now.

The way the current patch does things is more like the Oracle model.
Oracle has a "prepare commit" command (which I think even makes more
sense for this approach than making things look like nested transactions).
However I'm told that Oracle builds on the assumption of a two-tier
system, and it doesn't work very well.  If two-tier were good enough, then
I suspect you might as well use something like my robusttransaction class
which catches the special case of losing connection during commit, then
tries to reconnect and find out whether the commit went through or not.

As I mentioned to Bruce at FOSDEM last year, what I'd really need to get
that right (and the same goes for 2pc) I needed to be able to query the
transaction log in some way, to find out what happened to a transaction.
I've always assumed, and still believe, that what I did here was just a
hack to make the worst case a bit less likely.  It's not what I would call
a solution to the atomicity problem.

On the other hand, of course, at least one traditional argument for 3-tier
setups doesn't apply to us: we don't need connection pooling on the
middleware just to keep the per-connection licensing cost down.  :)


Jeroen

From pgsql-hackers-owner+M59934=pgman=candle.pha.pa.us@postgresql.org Tue Oct 12 15:27:20 2004
Return-path: <pgsql-hackers-owner+M59934=pgman=candle.pha.pa.us@postgresql.org>
Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id i9CJRJp09792
	for <pgman@candle.pha.pa.us>; Tue, 12 Oct 2004 15:27:19 -0400 (EDT)
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id E0D5B32AD90
	for <pgman@candle.pha.pa.us>; Tue, 12 Oct 2004 20:27:15 +0100 (BST)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 89028-10 for <pgman@candle.pha.pa.us>;
	Tue, 12 Oct 2004 19:27:14 +0000 (GMT)
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by svr1.postgresql.org (Postfix) with ESMTP id 524D832AC81
	for <pgman@candle.pha.pa.us>; Tue, 12 Oct 2004 20:27:15 +0100 (BST)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 7B81632B6BE
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Thu,  7 Oct 2004 12:18:35 +0100 (BST)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 22385-04
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
	Thu,  7 Oct 2004 11:17:11 +0000 (GMT)
Received: from smtp-3.hut.fi (smtp-3.hut.fi [130.233.228.93])
	by svr1.postgresql.org (Postfix) with ESMTP id 50FBE32B722
	for <pgsql-hackers@postgresql.org>; Thu,  7 Oct 2004 12:16:45 +0100 (BST)
Received: from kosh.hut.fi (kosh.hut.fi [130.233.228.10])
	by smtp-3.hut.fi (8.12.10/8.12.10) with ESMTP id i97BFicq021410;
	Thu, 7 Oct 2004 14:15:45 +0300
Date: Thu, 7 Oct 2004 14:15:44 +0300 (EEST)
From: Heikki Linnakangas <hlinnaka@iki.fi>
X-X-Sender: hlinnaka@kosh.hut.fi
To: Tom Lane <tgl@sss.pgh.pa.us>
cc: Heikki Linnakangas <hlinnaka@iki.fi>, pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Two-phase commit
In-Reply-To: <354.1097099170@sss.pgh.pa.us>
Message-ID: <Pine.OSF.4.61.0410071357420.432862@kosh.hut.fi>
References: <Pine.OSF.4.58.0402042200330.238747@kosh.hut.fi>
	<200402080138.i181cPl15259@candle.pha.pa.us> <Pine.OSF.4.58.0402220324250.126984@kosh.hut.fi>
	<Pine.OSF.4.58.0403231758350.513267@kosh.hut.fi> <354.1097099170@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
X-RAVMilter-Version: 8.4.3(snapshot 20030212) (smtp-3.hut.fi)
X-Virus-Scanned: by amavisd-new at hub.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on 
	candle.pha.pa.us
X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham 
	version=2.61
Status: OR

On Wed, 6 Oct 2004, Tom Lane wrote:

> Quite some time ago, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>> I haven't received any comments and there hasn't been any discussion on
>> the implementation, I suppose that nobody has given it a try. :(
>
> I finally got around to taking a close look at this.  There's a good bit
> undone, as you well know, but it seems like it can be the basis for a
> workable feature.  I do have a few comments to make.

Great!

> At the API level, I like the PREPARE/COMMIT/ROLLBACK statements, but I
> think you have missed a bet in that it needs to be possible to issue
> "COMMIT PREPARED gid" for the same gid several times without error.
> Consider a scenario where the transaction monitor crashes during the
> commit phase.  When it recovers, it will be aware that it had committed
> to commit, but it won't know which nodes were successfully committed.
> So it will need to resend the COMMIT commands.  It would be bad for the
> nodes to simply say "yes boss" if they are told to COMMIT a gid they
> have no record of.  So I think the gid's have to stick around after
> COMMIT PREPARED or ROLLBACK PREPARED, and there needs to be a fourth
> command (RELEASE PREPARED?) to actually remove the state data when the
> transaction monitor is satisfied that everything's done.  RELEASE of
> an unknown gid is okay to be a no-op.

Hmm. I don't see a problem with the "yes boss" approach. Some kind of a 
warning is appropriate, of course, but I don't see a reason for an 
additional step. After all, you would still fall back to the "yes boss" 
approach on the RELEASE PREPARED command.

The transaction monitor knows if the 1st phase succeeded or not, so if the 
COMMIT PREPARED doesn't find the transaction anymore, the monitor knows 
that it's previous commit/rollback succeeded.

> Implementation-wise, I really dislike storing the info in a shared hash
> table, because I don't see any reasonable bound on the size of the hash
> table (your existing code uses 100 which is about as arbitrary as it
> gets).  Plus the actual content of each entry is not fixed-size either.
> This is not very workable given our fixed-size shared memory mechanism.

I fully agree, I'm very dissatisfied with that part.

> The idea that occurs to me instead is to not use WAL or shared memory at
> all for keeping the prepared-transaction state info.  Instead, suppose
> that we store the status information in a file named after the GID,
> "$PGDATA/pg_twophase/gid".  We could write the file with a CRC similarly
> to what's done for pg_control.  Once such a file is written and fsync'd,
> it's equally as reliable as a WAL record would be, so it seems safe
> enough to me to report the PREPARE as done.  COMMIT, ROLLBACK, and the
> pg_prepared_xacts system view would look into the pg_twophase directory
> to find out all about active prepared transactions; RELEASE PREPARED
> would simply delete the appropriate file.  (Note: commit or rollback
> would need to take the transaction XID from the GID file and then look
> in pg_clog to find out if the transaction were already committed.  These
> operations do not change the pg_twophase file, but they do write a
> normal transaction-commit or -abort WAL record and update pg_clog.)

That sounds like a clever idea! I thought about using a single file 
myself, but the multi-file approach is much simpler.

> I think this would offer better performance as well as being more
> scalable, because the implementation you have looks like it would have
> some contention for the shared GID hashtable.

I guess the performance would depend a lot on how good/bad the filesystem 
is at creating and deleting a lot of small files.

> I would be inclined to require GIDs to be numbers (probably int8's)
> instead of strings, so that we don't have any problems with funny
> characters in the file names.  That's negotiable though, as we could
> certainly uuencode the strings or something to avoid that trap.

I'm afraid we have to support arbitrary strings. I think at least the Java 
Transaction API requires that, I'm not sure though if that could be 
worked around in the JDBC driver.

> You were concerned about how to mark prepared transactions in pg_clog,
> given that Alvaro had already commandeered state '11' for
> subtransactions.  Since only a toplevel transaction can be prepared,
> it might work to allow state '11' with a zero pg_subtrans parent link
> to mean a prepared transaction.  This would imply factoring prepared
> XIDs into GlobalXmin (so that pg_subtrans entries don't get recycled
> too soon) but we probably have to do that anyway.  AFAICS, prepared
> but uncommitted XIDs have to be considered still InProgress, so if
> they are less than GlobalXmin we'd lose.

Yes, they must be considered InProgress. The snapshot code needs to be 
modified to handle an arbitrary number of in progress transactions.


I've been thinking if it would be useful to have the COMMIT
PREPARED/ROLLBACK PREPARED commands under transaction control themselves. 
You could for example do "BEGIN; COMMIT PREPARED mygid; COMMIT PREPARED 
mygid2; COMMIT;" to atomically commit two already-prepared transactions, 
and even chain the 2PC transactions like "BEGIN; COMMIT PREPARED mygid; 
PREPARE TRANSACTION mygid2". It seems feasible to implement, just postpone 
the actual 2nd phase commit to the end of the commit of the enclosing 
transaction.

- Heikki

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

From pgsql-hackers-owner+M59924=pgman=candle.pha.pa.us@postgresql.org Tue Oct 12 15:25:43 2004
Return-path: <pgsql-hackers-owner+M59924=pgman=candle.pha.pa.us@postgresql.org>
Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id i9CJPfp08866
	for <pgman@candle.pha.pa.us>; Tue, 12 Oct 2004 15:25:42 -0400 (EDT)
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id C0C3D32BB7C
	for <pgman@candle.pha.pa.us>; Tue, 12 Oct 2004 20:25:37 +0100 (BST)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 86781-07 for <pgman@candle.pha.pa.us>;
	Tue, 12 Oct 2004 19:25:36 +0000 (GMT)
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by svr1.postgresql.org (Postfix) with ESMTP id 2F57A32BB46
	for <pgman@candle.pha.pa.us>; Tue, 12 Oct 2004 20:25:37 +0100 (BST)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id B4BAD32A071
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Sat,  9 Oct 2004 12:48:23 +0100 (BST)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 38695-06
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
	Sat,  9 Oct 2004 11:48:09 +0000 (GMT)
Received: from smtp-3.hut.fi (smtp-3.hut.fi [130.233.228.93])
	by svr1.postgresql.org (Postfix) with ESMTP id A887832A4FE
	for <pgsql-hackers@postgresql.org>; Sat,  9 Oct 2004 12:48:10 +0100 (BST)
Received: from kosh.hut.fi (kosh.hut.fi [130.233.228.10])
	by smtp-3.hut.fi (8.12.10/8.12.10) with ESMTP id i99Bm9UD031611
	for <pgsql-hackers@postgresql.org>; Sat, 9 Oct 2004 14:48:09 +0300
Date: Sat, 9 Oct 2004 14:48:09 +0300 (EEST)
From: Heikki Linnakangas <hlinnaka@iki.fi>
X-X-Sender: hlinnaka@kosh.hut.fi
To: pgsql-hackers@postgresql.org
Subject: [HACKERS] Two-phase commit patch updated
In-Reply-To: <354.1097099170@sss.pgh.pa.us>
Message-ID: <Pine.OSF.4.61.0410091429320.339489@kosh.hut.fi>
References: <Pine.OSF.4.58.0402042200330.238747@kosh.hut.fi>
	<200402080138.i181cPl15259@candle.pha.pa.us> <Pine.OSF.4.58.0402220324250.126984@kosh.hut.fi>
	<Pine.OSF.4.58.0403231758350.513267@kosh.hut.fi> <354.1097099170@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
X-RAVMilter-Version: 8.4.3(snapshot 20030212) (smtp-3.hut.fi)
X-Virus-Scanned: by amavisd-new at hub.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on 
	candle.pha.pa.us
X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham 
	version=2.61
Status: OR

I brought the 2PC patch up to date:

http://www.hut.fi/~hlinnaka/pgsql/

There's no new functionality, I just fixed all the bit rot so that it 
applies to the current CVS tip.

- Heikki

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

From pgsql-hackers-owner+M60193=pgman=candle.pha.pa.us@postgresql.org Wed Oct 20 11:43:40 2004
Return-path: <pgsql-hackers-owner+M60193=pgman=candle.pha.pa.us@postgresql.org>
Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id i9KFhdf10906
	for <pgman@candle.pha.pa.us>; Wed, 20 Oct 2004 11:43:40 -0400 (EDT)
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 6358632A158
	for <pgman@candle.pha.pa.us>; Wed, 20 Oct 2004 16:43:37 +0100 (BST)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 60338-02 for <pgman@candle.pha.pa.us>;
	Wed, 20 Oct 2004 15:43:36 +0000 (GMT)
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by svr1.postgresql.org (Postfix) with ESMTP id 01CF832A12A
	for <pgman@candle.pha.pa.us>; Wed, 20 Oct 2004 16:43:37 +0100 (BST)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 0A2B932B107
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Wed, 20 Oct 2004 16:41:15 +0100 (BST)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 74851-10
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
	Wed, 20 Oct 2004 15:41:08 +0000 (GMT)
Received: from mailsrvr2.bull.com (mailsrvr2.bull.com [192.90.162.8])
	by svr1.postgresql.org (Postfix) with ESMTP id 58BB532AF80
	for <pgsql-hackers@postgresql.org>; Wed, 20 Oct 2004 16:41:07 +0100 (BST)
Received: from us-phx1.az05.bull.com (us-phx1.az05.bull.com [141.112.40.1])
	by mailsrvr2.bull.com (8.13.1/8.13.1) with ESMTP id i9KFeskm013495
	for <pgsql-hackers@postgresql.org>; Wed, 20 Oct 2004 08:40:54 -0700
To: pgsql-hackers@postgresql.org
Subject: [HACKERS] 2PC support
MIME-Version: 1.0
X-Mailer: Lotus Notes Release 6.5.1 January 21, 2004
Message-ID: <OFC4281B9F.ABCA4E46-ON07256F33.0054BD37-07256F33.00562470@us-phx1.az05.bull.com>
From: Michael.Giroux@objectweb.org
Date: Wed, 20 Oct 2004 08:40:53 -0700
X-MIMETrack: Serialize by Router on US-PHX1/US/BULL(Release 6.5.1|January 21, 2004) at
	10/20/2004 08:40:54 AM,
	Serialize complete at 10/20/2004 08:40:54 AM
Content-Type: text/plain; charset="US-ASCII"
X-BullServices-MailScanner-Information: Please contact the ISP for more information
X-BullServices-MailScanner: Not scanned: please contact your Internet E-Mail Service Provider for details
X-MailScanner-From: michael.giroux@objectweb.org
X-Virus-Scanned: by amavisd-new at hub.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Virus-Scanned: by amavisd-new at hub.org
Status: OR

Hi,
I've read a few of the recent posts about 2-phase-commit support.  I 
noticed some discussion related to persisting the state of in-doubt 
transactions, and thought you might be able to take advantage of work that 
has been going on in the area of transaction logs. 

I'm working on a project (howl.objectweb.org) to implement a journal 
designed specifically to support transaction managers 2PC protocol.  The 
project was concieved during a meeting between ObjectWeb and Apache 
Geronimo developers last year at ApacheCon 2003.  The journal is currently 
used by the Apache Geronimo TM, Codehaus ActiveMQ, and ObjectWeb JOTM 
projects.

Although the main objective is to support TM 2PC protocol, it could be 
used as a database before-image journal as well.  The code is BSD 
licensed.  More at http://howl.objectweb.org.

Also, with respect to the requirements for the 2PC protocol, there is an 
excellent discussion by Mike Spille.  Mike wrote a series of articles 
titled "XA Exposed".  You can review the articles at 
http://www.theserverside.com.  Search for "xa exposed".

If any of you will be at ApacheCon 2004 next month, I'm giving a paper on 
the HOWL journal.  I'll be happy to talk with you there.

Michael Giroux,  objectweb.org

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
      subscribe-nomail command to majordomo@postgresql.org so that your
      message can get through to the mailing list cleanly

From pgsql-hackers-owner+M63326=pgman=candle.pha.pa.us@postgresql.org Wed Jan 19 13:02:04 2005
Return-path: <pgsql-hackers-owner+M63326=pgman=candle.pha.pa.us@postgresql.org>
Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j0JJ23w15071
	for <pgman@candle.pha.pa.us>; Wed, 19 Jan 2005 14:02:03 -0500 (EST)
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 86EA13A438A
	for <pgman@candle.pha.pa.us>; Wed, 19 Jan 2005 19:02:00 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 39979-07 for <pgman@candle.pha.pa.us>;
	Wed, 19 Jan 2005 19:01:58 +0000 (GMT)
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by svr1.postgresql.org (Postfix) with ESMTP id DC7E53A4447
	for <pgman@candle.pha.pa.us>; Wed, 19 Jan 2005 19:01:45 +0000 (GMT)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id E30A83A43DE
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Wed, 19 Jan 2005 18:59:59 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 39733-06
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
	Wed, 19 Jan 2005 18:59:49 +0000 (GMT)
Received: from smtp-3.hut.fi (smtp-3.hut.fi [130.233.228.93])
	by svr1.postgresql.org (Postfix) with ESMTP id 83C883A42C9
	for <pgsql-hackers@postgresql.org>; Wed, 19 Jan 2005 18:59:48 +0000 (GMT)
Received: from localhost (putosiko.hut.fi [130.233.228.114])
	by smtp-3.hut.fi (8.12.10/8.12.10) with ESMTP id j0JIxkeL013491
	for <pgsql-hackers@postgresql.org>; Wed, 19 Jan 2005 20:59:46 +0200
Received: from smtp-3.hut.fi ([130.233.228.93])
	by localhost (putosiko.hut.fi [130.233.228.114]) (amavisd-new, port 10024)
	with LMTP id 17639-01-10 for <pgsql-hackers@postgresql.org>;
	Wed, 19 Jan 2005 20:59:45 +0200 (EET)
Received: from kosh.hut.fi (kosh.hut.fi [130.233.228.10])
	by smtp-3.hut.fi (8.12.10/8.12.10) with ESMTP id j0JItvEJ011614
	for <pgsql-hackers@postgresql.org>; Wed, 19 Jan 2005 20:55:57 +0200
Date: Wed, 19 Jan 2005 20:55:57 +0200 (EET)
From: Heikki Linnakangas <hlinnaka@iki.fi>
X-X-Sender: hlinnaka@kosh.hut.fi
To: pgsql-hackers@postgresql.org
Subject: [HACKERS] Two-phase commit for 8.1
Message-ID: <Pine.OSF.4.61.0501192055310.62540@kosh.hut.fi>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
X-TKK-Virus-Scanned: by amavisd-new-2.1.2-hutcc at putosiko.hut.fi
X-Virus-Scanned: by amavisd-new at hub.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on 
	candle.pha.pa.us
X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham 
	version=2.61
Status: OR

Hi,

Now that we got 8.0 out of the door, I'm submitting my two-phase commit patch 
again for discussion.

http://www.hut.fi/~hlinnaka/pgsql/

Do we want it in 8.1, if we want a short development cycle? It needs a new 
pg_twophase subdirectory, and it introduces a new system view, so I guess it 
requires an initdb (or pg_upgrade).

Any comments on the implementation or the new commands?

I would appreciate help testing the JDBC driver with an application server that 
does XA recovery properly.

- Heikki

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

From pgsql-hackers-owner+M63328=pgman=candle.pha.pa.us@postgresql.org Wed Jan 19 15:00:49 2005
Return-path: <pgsql-hackers-owner+M63328=pgman=candle.pha.pa.us@postgresql.org>
Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j0JL0lw26408
	for <pgman@candle.pha.pa.us>; Wed, 19 Jan 2005 16:00:47 -0500 (EST)
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id AA4F63A44F7
	for <pgman@candle.pha.pa.us>; Wed, 19 Jan 2005 21:00:43 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 69508-07 for <pgman@candle.pha.pa.us>;
	Wed, 19 Jan 2005 21:00:41 +0000 (GMT)
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by svr1.postgresql.org (Postfix) with ESMTP id 954C03A4493
	for <pgman@candle.pha.pa.us>; Wed, 19 Jan 2005 21:00:42 +0000 (GMT)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id AEE2A3A4447
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Wed, 19 Jan 2005 20:59:06 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 69310-05
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
	Wed, 19 Jan 2005 20:58:55 +0000 (GMT)
Received: from hub.org (hub.org [200.46.204.220])
	by svr1.postgresql.org (Postfix) with ESMTP id BD0E03A43DA
	for <pgsql-hackers@postgresql.org>; Wed, 19 Jan 2005 20:58:56 +0000 (GMT)
Received: from ganymede.hub.org (blk-224-186-245.eastlink.ca [24.224.186.245])
	by hub.org (Postfix) with ESMTP id 864D2129256;
	Wed, 19 Jan 2005 16:58:54 -0400 (AST)
Received: by ganymede.hub.org (Postfix, from userid 1000)
	id B19BA5E34A; Wed, 19 Jan 2005 16:58:55 -0400 (AST)
Received: from localhost (localhost [127.0.0.1])
	by ganymede.hub.org (Postfix) with ESMTP id AD3675DCBC;
	Wed, 19 Jan 2005 16:58:55 -0400 (AST)
Date: Wed, 19 Jan 2005 16:58:55 -0400 (AST)
From: "Marc G. Fournier" <scrappy@postgresql.org>
X-X-Sender: scrappy@ganymede.hub.org
To: Heikki Linnakangas <hlinnaka@iki.fi>
cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Two-phase commit for 8.1
In-Reply-To: <Pine.OSF.4.61.0501192055310.62540@kosh.hut.fi>
Message-ID: <20050119165747.U47533@ganymede.hub.org>
References: <Pine.OSF.4.61.0501192055310.62540@kosh.hut.fi>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
X-Virus-Scanned: by amavisd-new at hub.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on 
	candle.pha.pa.us
X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham 
	version=2.61
Status: OR


If the patch is ready to be committed early in the cycle, I'd say most 
definitely ... just depends on how late in the cycle its ready ...

I *believe* that 8.1, we're looking at a 2mo cycle before beta, so figure 
beta for ~April 1st (no april fools jokes, eh?) ...

On Wed, 19 Jan 2005, Heikki Linnakangas wrote:

> Hi,
>
> Now that we got 8.0 out of the door, I'm submitting my two-phase commit patch 
> again for discussion.
>
> http://www.hut.fi/~hlinnaka/pgsql/
>
> Do we want it in 8.1, if we want a short development cycle? It needs a new 
> pg_twophase subdirectory, and it introduces a new system view, so I guess it 
> requires an initdb (or pg_upgrade).
>
> Any comments on the implementation or the new commands?
>
> I would appreciate help testing the JDBC driver with an application server 
> that does XA recovery properly.
>
> - Heikki
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
>   (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
>

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

               http://www.postgresql.org/docs/faqs/FAQ.html

From pgsql-hackers-owner+M63331=pgman=candle.pha.pa.us@postgresql.org Wed Jan 19 18:45:25 2005
Return-path: <pgsql-hackers-owner+M63331=pgman=candle.pha.pa.us@postgresql.org>
Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j0K0jOw15340
	for <pgman@candle.pha.pa.us>; Wed, 19 Jan 2005 19:45:24 -0500 (EST)
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 40B203A45A5
	for <pgman@candle.pha.pa.us>; Thu, 20 Jan 2005 00:45:20 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 91025-08 for <pgman@candle.pha.pa.us>;
	Thu, 20 Jan 2005 00:45:16 +0000 (GMT)
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by svr1.postgresql.org (Postfix) with ESMTP id 46A8F3A459D
	for <pgman@candle.pha.pa.us>; Thu, 20 Jan 2005 00:45:19 +0000 (GMT)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 042FE3A4340;
	Thu, 20 Jan 2005 00:42:14 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 91212-03; Thu, 20 Jan 2005 00:42:05 +0000 (GMT)
Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130])
	by svr1.postgresql.org (Postfix) with ESMTP id 493723A42C9;
	Thu, 20 Jan 2005 00:42:08 +0000 (GMT)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
	by sss.pgh.pa.us (8.13.1/8.13.1) with ESMTP id j0K0g4GY001116;
	Wed, 19 Jan 2005 19:42:04 -0500 (EST)
To: "Marc G. Fournier" <scrappy@postgresql.org>
cc: Heikki Linnakangas <hlinnaka@iki.fi>, pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Two-phase commit for 8.1 
In-Reply-To: <20050119165747.U47533@ganymede.hub.org> 
References: <Pine.OSF.4.61.0501192055310.62540@kosh.hut.fi> <20050119165747.U47533@ganymede.hub.org>
Comments: In-reply-to "Marc G. Fournier" <scrappy@postgresql.org>
	message dated "Wed, 19 Jan 2005 16:58:55 -0400"
Date: Wed, 19 Jan 2005 19:42:03 -0500
Message-ID: <1115.1106181723@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
X-Virus-Scanned: by amavisd-new at hub.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Virus-Scanned: by amavisd-new at hub.org
Status: OR

"Marc G. Fournier" <scrappy@postgresql.org> writes:
> If the patch is ready to be committed early in the cycle, I'd say most 
> definitely ... just depends on how late in the cycle its ready ...

My recollection is that it's quite far from being complete.  I had hoped
to spend some time during the 8.1 cycle helping Heikki finish it up,
but if we stick to the 2-month-dev-cycle idea I'm afraid there's no way
it'll be done in time.  I thought that "some time" would probably amount
to a solid man-month or so, and there's no way I can spend half my time
on just one feature for this cycle.

If Heikki wants this in for 8.1, the right thing to do is vote against
the short-dev-cycle idea.  But we need a plausible answer about what to
do about ARC to make that credible...

			regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

               http://archives.postgresql.org

From pgsql-hackers-owner+M63432=pgman=candle.pha.pa.us@postgresql.org Sun Jan 23 05:41:58 2005
Return-path: <pgsql-hackers-owner+M63432=pgman=candle.pha.pa.us@postgresql.org>
Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j0NBfvw06363
	for <pgman@candle.pha.pa.us>; Sun, 23 Jan 2005 06:41:58 -0500 (EST)
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id D0CA53A3BE7
	for <pgman@candle.pha.pa.us>; Sun, 23 Jan 2005 11:41:55 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 51743-08 for <pgman@candle.pha.pa.us>;
	Sun, 23 Jan 2005 11:41:52 +0000 (GMT)
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by svr1.postgresql.org (Postfix) with ESMTP id E67533A51E4
	for <pgman@candle.pha.pa.us>; Sun, 23 Jan 2005 11:41:10 +0000 (GMT)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id CB87F3A525A
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Sun, 23 Jan 2005 11:39:49 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 51345-07
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
	Sun, 23 Jan 2005 11:39:32 +0000 (GMT)
Received: from smtp-3.hut.fi (smtp-3.hut.fi [130.233.228.93])
	by svr1.postgresql.org (Postfix) with ESMTP id 5F3223A1AA3
	for <pgsql-hackers@postgresql.org>; Sun, 23 Jan 2005 11:38:50 +0000 (GMT)
Received: from localhost (katosiko.hut.fi [130.233.228.115])
	by smtp-3.hut.fi (8.12.10/8.12.10) with ESMTP id j0NBcmeL011515;
	Sun, 23 Jan 2005 13:38:48 +0200
Received: from smtp-3.hut.fi ([130.233.228.93])
	by localhost (katosiko.hut.fi [130.233.228.115]) (amavisd-new, port 10024)
	with LMTP id 04384-02-2; Sun, 23 Jan 2005 13:38:47 +0200 (EET)
Received: from kosh.hut.fi (kosh.hut.fi [130.233.228.10])
	by smtp-3.hut.fi (8.12.10/8.12.10) with ESMTP id j0NBbUEJ011361;
	Sun, 23 Jan 2005 13:37:30 +0200
Date: Sun, 23 Jan 2005 13:37:30 +0200 (EET)
From: Heikki Linnakangas <hlinnaka@iki.fi>
X-X-Sender: hlinnaka@kosh.hut.fi
To: =?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= <postgres@cybertec.at>
cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Two-phase commit for 8.1
In-Reply-To: <41F3699B.2060608@cybertec.at>
Message-ID: <Pine.OSF.4.61.0501231147320.64318@kosh.hut.fi>
References: <Pine.OSF.4.61.0501192055310.62540@kosh.hut.fi>
	<20050119165747.U47533@ganymede.hub.org> <1115.1106181723@sss.pgh.pa.us>
	<Pine.OSF.4.61.0501212033520.352045@kosh.hut.fi> <41F3699B.2060608@cybertec.at>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8BIT
X-TKK-Virus-Scanned: by amavisd-new-2.1.2-hutcc at katosiko.hut.fi
X-Virus-Scanned: by amavisd-new at hub.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on 
	candle.pha.pa.us
X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham 
	version=2.61
Status: OR

On Sun, 23 Jan 2005, Hans-Jürgen Schönig wrote:

> Heikki,
>
> What is still missing to complete the 2PC patch?.

Here's my TODO on things that need to be done:

     * large objects
     * guc variables
     * notify/listen

Large objects and notify/listen should be quite straightforward. GUC 
variables need some thinking, but shouldn't be much work.

As the patch gets more attention, I'm sure more issues will come up.

- Heikki

---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
      joining column's datatypes do not match

From pgsql-hackers-owner+M63434=pgman=candle.pha.pa.us@postgresql.org Sun Jan 23 09:15:24 2005
Return-path: <pgsql-hackers-owner+M63434=pgman=candle.pha.pa.us@postgresql.org>
Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j0NFFMw25305
	for <pgman@candle.pha.pa.us>; Sun, 23 Jan 2005 10:15:23 -0500 (EST)
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 3A5DA3A3BE7
	for <pgman@candle.pha.pa.us>; Sun, 23 Jan 2005 15:15:16 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 99539-08 for <pgman@candle.pha.pa.us>;
	Sun, 23 Jan 2005 15:15:11 +0000 (GMT)
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by svr1.postgresql.org (Postfix) with ESMTP id 4A7C43A1A03
	for <pgman@candle.pha.pa.us>; Sun, 23 Jan 2005 15:15:15 +0000 (GMT)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 08EA83A5242
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Sun, 23 Jan 2005 15:13:52 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 00254-02
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
	Sun, 23 Jan 2005 15:13:46 +0000 (GMT)
Received: from sunsite.dcc.uchile.cl (sunsite.dcc.uchile.cl [192.80.24.2])
	by svr1.postgresql.org (Postfix) with ESMTP id AF7C83A520D
	for <pgsql-hackers@postgresql.org>; Sun, 23 Jan 2005 15:13:48 +0000 (GMT)
Received: from anakena.dcc.uchile.cl ([192.80.24.3])
	by sunsite.dcc.uchile.cl (8.12.11/8.12.11) with ESMTP id j0NFDbu4014311;
	Sun, 23 Jan 2005 12:13:42 -0300 (CLST)
Received: by anakena.dcc.uchile.cl (Postfix, from userid 4151)
	id 666A5527DA; Sun, 23 Jan 2005 12:13:37 -0300 (CLST)
Date: Sun, 23 Jan 2005 12:13:37 -0300
From: Alvaro Herrera <alvherre@dcc.uchile.cl>
To: Heikki Linnakangas <hlinnaka@iki.fi>
cc: =?iso-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= <postgres@cybertec.at>,
   pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Two-phase commit for 8.1
Message-ID: <20050123151337.GD4406@dcc.uchile.cl>
References: <Pine.OSF.4.61.0501192055310.62540@kosh.hut.fi> <20050119165747.U47533@ganymede.hub.org> <1115.1106181723@sss.pgh.pa.us> <Pine.OSF.4.61.0501212033520.352045@kosh.hut.fi> <41F3699B.2060608@cybertec.at> <Pine.OSF.4.61.0501231147320.64318@kosh.hut.fi>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Pine.OSF.4.61.0501231147320.64318@kosh.hut.fi>
User-Agent: Mutt/1.5.6i
X-Virus-Scanned: by amavisd-new at hub.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on 
	candle.pha.pa.us
X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham 
	version=2.61
Status: OR

On Sun, Jan 23, 2005 at 01:37:30PM +0200, Heikki Linnakangas wrote:

> As the patch gets more attention, I'm sure more issues will come up.

I see the changes to the lock manager are huge.  Can you explain what's
the idea behind those?  Do you release the locks and then reacquire
them, or do you reassign them to a pseudo process?  Are there
possibilities of deadlock somewhere?

-- 
Alvaro Herrera (<alvherre[@]dcc.uchile.cl>)
Thou shalt study thy libraries and strive not to reinvent them without
cause, that thy code may be short and readable and thy days pleasant
and productive. (7th Commandment for C Programmers)

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
      subscribe-nomail command to majordomo@postgresql.org so that your
      message can get through to the mailing list cleanly

From pgsql-hackers-owner+M63438=pgman=candle.pha.pa.us@postgresql.org Sun Jan 23 11:36:40 2005
Return-path: <pgsql-hackers-owner+M63438=pgman=candle.pha.pa.us@postgresql.org>
Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j0NHaaw11289
	for <pgman@candle.pha.pa.us>; Sun, 23 Jan 2005 12:36:37 -0500 (EST)
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 7A1633A4A77
	for <pgman@candle.pha.pa.us>; Sun, 23 Jan 2005 17:36:32 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 13791-10 for <pgman@candle.pha.pa.us>;
	Sun, 23 Jan 2005 17:36:31 +0000 (GMT)
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by svr1.postgresql.org (Postfix) with ESMTP id 852863A49EC
	for <pgman@candle.pha.pa.us>; Sun, 23 Jan 2005 17:36:31 +0000 (GMT)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 252743A4B01
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Sun, 23 Jan 2005 17:35:26 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 13880-08
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
	Sun, 23 Jan 2005 17:35:17 +0000 (GMT)
Received: from smtp-1.hut.fi (smtp-1.hut.fi [130.233.228.91])
	by svr1.postgresql.org (Postfix) with ESMTP id 5B7633A19A5
	for <pgsql-hackers@postgresql.org>; Sun, 23 Jan 2005 17:35:12 +0000 (GMT)
Received: from localhost (katosiko.hut.fi [130.233.228.115])
	by smtp-1.hut.fi (8.12.10/8.12.10) with ESMTP id j0NHYwms014946;
	Sun, 23 Jan 2005 19:34:58 +0200
Received: from smtp-1.hut.fi ([130.233.228.91])
	by localhost (katosiko.hut.fi [130.233.228.115]) (amavisd-new, port 10024)
	with LMTP id 04068-01; Sun, 23 Jan 2005 19:34:57 +0200 (EET)
Received: from kosh.hut.fi (kosh.hut.fi [130.233.228.10])
	by smtp-1.hut.fi (8.12.10/8.12.10) with ESMTP id j0NHWt0g014723;
	Sun, 23 Jan 2005 19:32:55 +0200
Date: Sun, 23 Jan 2005 19:32:55 +0200 (EET)
From: Heikki Linnakangas <hlinnaka@iki.fi>
X-X-Sender: hlinnaka@kosh.hut.fi
To: Alvaro Herrera <alvherre@dcc.uchile.cl>
cc: =?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= <postgres@cybertec.at>,
   pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Two-phase commit for 8.1
In-Reply-To: <20050123151337.GD4406@dcc.uchile.cl>
Message-ID: <Pine.OSF.4.61.0501231916520.414393@kosh.hut.fi>
References: <Pine.OSF.4.61.0501192055310.62540@kosh.hut.fi>
	<20050119165747.U47533@ganymede.hub.org> <1115.1106181723@sss.pgh.pa.us>
	<Pine.OSF.4.61.0501212033520.352045@kosh.hut.fi> <41F3699B.2060608@cybertec.at>
	<Pine.OSF.4.61.0501231147320.64318@kosh.hut.fi> <20050123151337.GD4406@dcc.uchile.cl>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
X-TKK-Virus-Scanned: by amavisd-new-2.1.2-hutcc at katosiko.hut.fi
X-Virus-Scanned: by amavisd-new at hub.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on 
	candle.pha.pa.us
X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham 
	version=2.61
Status: OR

On Sun, 23 Jan 2005, Alvaro Herrera wrote:

> On Sun, Jan 23, 2005 at 01:37:30PM +0200, Heikki Linnakangas wrote:
>
>> As the patch gets more attention, I'm sure more issues will come up.

> I see the changes to the lock manager are huge.  Can you explain what's
> the idea behind those?  Do you release the locks and then reacquire
> them, or do you reassign them to a pseudo process?

I reassign them to a pseudo process (persistedLocksProc).

Much of the changes in lock.c are just about moving code around. 
Some copy-paste code has been put in functions. LockAcquire has been
changed to take PGPROC as an argument, so that locks can be acquired on 
behalf of the pseudo process.

Then there's completely new code for persisting locks on PREPARE 
TRANSACTION and reacquiring them on recovery.

If it helps, I could try to split it into two patches, one with code 
rearrangements that don't change current behaviour, and then the actual 
2PC stuff on top of that.

> Are there possibilities of deadlock somewhere?

Not that I know of.

- Heikki

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

               http://www.postgresql.org/docs/faq

From pgsql-hackers-owner+M63788=pgman=candle.pha.pa.us@postgresql.org Mon Jan 31 09:35:11 2005
Return-path: <pgsql-hackers-owner+M63788=pgman=candle.pha.pa.us@postgresql.org>
Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j0VFZAw06420
	for <pgman@candle.pha.pa.us>; Mon, 31 Jan 2005 10:35:10 -0500 (EST)
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 46B838B9C28
	for <pgman@candle.pha.pa.us>; Mon, 31 Jan 2005 15:35:07 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 86064-08 for <pgman@candle.pha.pa.us>;
	Mon, 31 Jan 2005 15:35:03 +0000 (GMT)
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by svr1.postgresql.org (Postfix) with ESMTP id 5D6918B9E6A
	for <pgman@candle.pha.pa.us>; Mon, 31 Jan 2005 15:23:48 +0000 (GMT)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id E92DF8B9CC6
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 31 Jan 2005 15:21:14 +0000 (GMT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 83005-03
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
	Mon, 31 Jan 2005 15:21:04 +0000 (GMT)
Received: from sunsite.dcc.uchile.cl (sunsite.dcc.uchile.cl [192.80.24.2])
	by svr1.postgresql.org (Postfix) with ESMTP id A315B8B9D3E
	for <pgsql-hackers@postgresql.org>; Mon, 31 Jan 2005 15:01:43 +0000 (GMT)
Received: from anakena.dcc.uchile.cl ([192.80.24.3])
	by sunsite.dcc.uchile.cl (8.12.11/8.12.11) with ESMTP id j0VF1KCf026107;
	Mon, 31 Jan 2005 12:01:24 -0300 (CLST)
Received: by anakena.dcc.uchile.cl (Postfix, from userid 4151)
	id F287A52515; Mon, 31 Jan 2005 12:01:19 -0300 (CLST)
Date: Mon, 31 Jan 2005 12:01:19 -0300
From: Alvaro Herrera <alvherre@dcc.uchile.cl>
To: Heikki Linnakangas <hlinnaka@iki.fi>
cc: =?iso-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= <postgres@cybertec.at>,
   pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Two-phase commit for 8.1
Message-ID: <20050131150119.GI22141@dcc.uchile.cl>
References: <Pine.OSF.4.61.0501192055310.62540@kosh.hut.fi> <20050119165747.U47533@ganymede.hub.org> <1115.1106181723@sss.pgh.pa.us> <Pine.OSF.4.61.0501212033520.352045@kosh.hut.fi> <41F3699B.2060608@cybertec.at> <Pine.OSF.4.61.0501231147320.64318@kosh.hut.fi> <20050123151337.GD4406@dcc.uchile.cl> <Pine.OSF.4.61.0501231916520.414393@kosh.hut.fi>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <Pine.OSF.4.61.0501231916520.414393@kosh.hut.fi>
User-Agent: Mutt/1.5.6i
X-Virus-Scanned: by amavisd-new at hub.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on 
	candle.pha.pa.us
X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham 
	version=2.61
Status: OR

On Sun, Jan 23, 2005 at 07:32:55PM +0200, Heikki Linnakangas wrote:

> If it helps, I could try to split it into two patches, one with code 
> rearrangements that don't change current behaviour, and then the actual 
> 2PC stuff on top of that.

I think that'd be a good idea, because such a patch could be merged
right now, and the actual 2PC stuff would be smaller and easier to
review.

You'd only need a committer to actually commit the initial patch ...

-- 
Alvaro Herrera (<alvherre[@]dcc.uchile.cl>)
"No hay cielo posible sin hundir nuestras raíces
 en la profundidad de la tierra"                        (Malucha Pinto)

---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
      joining column's datatypes do not match

From pgsql-hackers-owner+M66050@postgresql.org Wed Apr  6 10:04:39 2005
Return-path: <pgsql-hackers-owner+M66050@postgresql.org>
Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71])
	by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j36E4cO20150
	for <pgman@candle.pha.pa.us>; Wed, 6 Apr 2005 10:04:38 -0400 (EDT)
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 248C2537FE
	for <pgman@candle.pha.pa.us>; Wed,  6 Apr 2005 15:04:32 +0100 (BST)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 64844-06 for <pgman@candle.pha.pa.us>;
	Wed,  6 Apr 2005 14:04:32 +0000 (GMT)
Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
	by svr1.postgresql.org (Postfix) with ESMTP id D2EB5537F8
	for <pgman@candle.pha.pa.us>; Wed,  6 Apr 2005 15:04:31 +0100 (BST)
X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id AA7FE539FA
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Wed,  6 Apr 2005 15:01:33 +0100 (BST)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 62842-09
	for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
	Wed,  6 Apr 2005 14:01:26 +0000 (GMT)
Received: from smtp-1.hut.fi (smtp-1.hut.fi [130.233.228.91])
	by svr1.postgresql.org (Postfix) with ESMTP id F02FD538C8
	for <pgsql-hackers@postgresql.org>; Wed,  6 Apr 2005 15:01:22 +0100 (BST)
Received: from localhost (katosiko.hut.fi [130.233.228.115])
	by smtp-1.hut.fi (8.12.10/8.12.10) with ESMTP id j36E11Yl028611;
	Wed, 6 Apr 2005 17:01:01 +0300
Received: from smtp-1.hut.fi ([130.233.228.91])
	by localhost (katosiko.hut.fi [130.233.228.115]) (amavisd-new, port 10024)
	with LMTP id 22125-05; Wed,  6 Apr 2005 17:01:00 +0300 (EEST)
Received: from kosh.hut.fi (kosh.hut.fi [130.233.228.10])
	by smtp-1.hut.fi (8.12.10/8.12.10) with ESMTP id j36DxlPJ028376;
	Wed, 6 Apr 2005 16:59:48 +0300
Date: Wed, 6 Apr 2005 16:59:47 +0300 (EEST)
From: Heikki Linnakangas <hlinnaka@iki.fi>
X-X-Sender: hlinnaka@kosh.hut.fi
To: Alvaro Herrera <alvherre@dcc.uchile.cl>
cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Two phase commit
In-Reply-To: <20050405230731.GA26960@surnet.cl>
Message-ID: <Pine.OSF.4.61.0504061652300.502110@kosh.hut.fi>
References: <20050405230731.GA26960@surnet.cl>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
X-TKK-Virus-Scanned: by amavisd-new-2.1.2-hutcc at katosiko.hut.fi
X-Virus-Scanned: by amavisd-new at hub.org
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
X-Virus-Scanned: by amavisd-new at hub.org
Status: OR

On Tue, 5 Apr 2005, Alvaro Herrera wrote:

> What happenned to your two phase commit patch?  Are you still working on
> it?  Have you got something done from the last time we heard from you?

I've kept it up-to-date by doing "cvs update" every now and then and 
fixing possible conflicts.

It would be nice if you hackers could take a serious look at it and tell 
what needs to be done to get it finally committed.

I've been busy with other things and haven't had the time to push it.

I try to update my "project page" every time I update the patch:

http://www.hut.fi/~hlinnaka/pgsql/

There isn't any big issues left as far as I know.

- Heikki

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

