ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Scherbakov (Jira)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-11704) Write tombstones during rebalance to get rid of deferred delete buffer
Date Tue, 15 Oct 2019 15:22:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-11704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16952020#comment-16952020
] 

Alexei Scherbakov commented on IGNITE-11704:
--------------------------------------------

[~jokser]

4. Typo in the comment. My understanding is the code will be called when datastreamer initiates
first update for an entry, is it true ?
6. 
* Looks like it's not necessary to preload 256k keys for historical rebalance, you need only
one update in each partition. 
* Test looks similar but my idea is to delay each batch and remove all containing keys in
the batch, then release batch. Such scenario should bring all partition keys to tombstones
and looks interesting.

In other aspects looks good.


> Write tombstones during rebalance to get rid of deferred delete buffer
> ----------------------------------------------------------------------
>
>                 Key: IGNITE-11704
>                 URL: https://issues.apache.org/jira/browse/IGNITE-11704
>             Project: Ignite
>          Issue Type: Improvement
>            Reporter: Alexey Goncharuk
>            Assignee: Pavel Kovalenko
>            Priority: Major
>              Labels: rebalance
>             Fix For: 2.8
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currently Ignite relies on deferred delete buffer in order to handle write-remove conflicts
during rebalance. Given the limit size of the buffer, this approach is fundamentally flawed,
especially in case when persistence is enabled.
> I suggest to extend the logic of data storage to be able to store key tombstones - to
keep version for deleted entries. The tombstones will be stored when rebalance is in progress
and should be cleaned up when rebalance is completed.
> Later this approach may be used to implement fast partition rebalance based on merkle
trees (in this case, tombstones should be written on an incomplete baseline).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message