Commit ee4ba01d authored by Amit Kapila's avatar Amit Kapila

Fix the bugs in selecting the transaction for streaming.

There were two problems:
a. We were always selecting the next available txn instead of selecting it
when it is larger than the previous transaction.
b. We were selecting the transactions which haven't made any changes to
the database (base snapshot is not set). Later it was hitting an Assert
because we don't decode such transactions and the changes in txn remain as
it is. It is better not to choose such transactions for streaming in the
first place.

Reported-by: Haiying Tang
Author: Dilip Kumar
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/OS0PR01MB61133B94E63177040F7ECDA1FB429@OS0PR01MB6113.jpnprd01.prod.outlook.com
parent 3c80e96d
......@@ -3362,19 +3362,22 @@ ReorderBufferLargestTXN(ReorderBuffer *rb)
* This can be seen as an optimized version of ReorderBufferLargestTXN, which
* should give us the same transaction (because we don't update memory account
* for subtransaction with streaming, so it's always 0). But we can simply
* iterate over the limited number of toplevel transactions.
* iterate over the limited number of toplevel transactions that have a base
* snapshot. There is no use of selecting a transaction that doesn't have base
* snapshot because we don't decode such transactions.
*
* Note that, we skip transactions that contains incomplete changes. There
* is a scope of optimization here such that we can select the largest transaction
* which has complete changes. But that will make the code and design quite complex
* and that might not be worth the benefit. If we plan to stream the transactions
* that contains incomplete changes then we need to find a way to partially
* stream/truncate the transaction changes in-memory and build a mechanism to
* partially truncate the spilled files. Additionally, whenever we partially
* stream the transaction we need to maintain the last streamed lsn and next time
* we need to restore from that segment and the offset in WAL. As we stream the
* changes from the top transaction and restore them subtransaction wise, we need
* to even remember the subxact from where we streamed the last change.
* is a scope of optimization here such that we can select the largest
* transaction which has incomplete changes. But that will make the code and
* design quite complex and that might not be worth the benefit. If we plan to
* stream the transactions that contains incomplete changes then we need to
* find a way to partially stream/truncate the transaction changes in-memory
* and build a mechanism to partially truncate the spilled files.
* Additionally, whenever we partially stream the transaction we need to
* maintain the last streamed lsn and next time we need to restore from that
* segment and the offset in WAL. As we stream the changes from the top
* transaction and restore them subtransaction wise, we need to even remember
* the subxact from where we streamed the last change.
*/
static ReorderBufferTXN *
ReorderBufferLargestTopTXN(ReorderBuffer *rb)
......@@ -3383,14 +3386,19 @@ ReorderBufferLargestTopTXN(ReorderBuffer *rb)
Size largest_size = 0;
ReorderBufferTXN *largest = NULL;
/* Find the largest top-level transaction. */
dlist_foreach(iter, &rb->toplevel_by_lsn)
/* Find the largest top-level transaction having a base snapshot. */
dlist_foreach(iter, &rb->txns_by_base_snapshot_lsn)
{
ReorderBufferTXN *txn;
txn = dlist_container(ReorderBufferTXN, node, iter.cur);
txn = dlist_container(ReorderBufferTXN, base_snapshot_node, iter.cur);
/* must not be a subtxn */
Assert(!rbtxn_is_known_subxact(txn));
/* base_snapshot must be set */
Assert(txn->base_snapshot != NULL);
if ((largest != NULL || txn->total_size > largest_size) &&
if ((largest == NULL || txn->total_size > largest_size) &&
(txn->total_size > 0) && !(rbtxn_has_incomplete_tuple(txn)))
{
largest = txn;
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment