ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Павлухин Иван <vololo...@gmail.com>
Subject Re: Historical rebalance
Date Mon, 26 Nov 2018 08:08:24 GMT
Igor,

Could you please clarify some points?

> 1) preserve list of active txs, sorted by the time of their first update (using WAL ptr
of first WAL record in tx)

Is this list maintained per transaction or per checkpoint (or per
something else)? Why can't we track only oldest active transaction
instead of whole active list?

> 4) find a checkpoint where the earliest tx exists in persisted txs and use saved WAL
ptr as a start point or apply current approach in case the active tx list (sent on previous
step) is empty

What is the base storage state on a demanding node to which we are
applying WAL records to? I mean the state before applying WAL records.
Do we apply all records simply one by one or filter out some of them?
пт, 23 нояб. 2018 г. в 11:22, Seliverstov Igor <gvvinblade@gmail.com>:
>
> Hi Igniters,
>
> Currently I’m working on possible approaches how to implement historical rebalance
(delta rebalance using WAL iterator) over MVCC caches.
>
> The main difficulty is that MVCC writes changes on tx active phase while partition update
version, aka update counter, is being applied on tx finish. This means we cannot start iteration
over WAL right from the pointer where the update counter updated, but should include updates,
which the transaction that updated the counter did.
>
> These updates may be much earlier than the point where the update counter was updated,
so we have to be able to identify the point where the first update happened.
>
> The proposed approach includes:
>
> 1) preserve list of active txs, sorted by the time of their first update (using WAL ptr
of first WAL record in tx)
>
> 2) persist this list on each checkpoint (together with TxLog for example)
>
> 4) send whole active tx list (transactions which were in active state at the time the
node was crushed, empty list in case of graceful node stop) as a part of partition demand
message.
>
> 4) find a checkpoint where the earliest tx exists in persisted txs and use saved WAL
ptr as a start point or apply current approach in case the active tx list (sent on previous
step) is empty
>
> 5) start iteration.
>
> Your thoughts?
>
> Regards,
> Igor



-- 
Best regards,
Ivan Pavlukhin

Mime
View raw message