ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pavel Pereslegin (Jira)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-11584) Implement batch insertion of new cache entries in FreeList to improve rebalancing
Date Mon, 11 Nov 2019 11:53:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-11584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16971468#comment-16971468

Pavel Pereslegin commented on IGNITE-11584:

Additional notes on risks and assumptions:
 # These changes were checked for memory leaks in {{testSimpleRebalancing}} and {{testComplexRebalancing}}
tests, using the following [method|https://github.com/apache/ignite/commit/472d5aa17de31a0eca780a61f9a655ec20a77e25#diff-50ffa3c92f2815b7e31d1dcc22ec6cc1R2329].
 # If WAL is enabled during rebalancing, it is possible that after a node failure, the WAL
recovery procedure may add unused rows into the page memory. Expecting partition eviction
in such a case.
 # Rebalancing of "staled" keys can run slower, due to rollbacks. Rollback occurred when the
node already has the key with a newer version - in this case, preloader will remove a "pre-created"
data row from the page memory.

> Implement batch insertion of new cache entries in FreeList to improve rebalancing
> ---------------------------------------------------------------------------------
>                 Key: IGNITE-11584
>                 URL: https://issues.apache.org/jira/browse/IGNITE-11584
>             Project: Ignite
>          Issue Type: Sub-task
>    Affects Versions: 2.7
>            Reporter: Pavel Pereslegin
>            Assignee: Pavel Pereslegin
>            Priority: Major
>              Labels: iep-32
>             Fix For: 2.8
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
> Main goals:
>  * Implement batch insert operation into FreeList - insert several data rows at once
>  * Use batch insertion in the preloader
> Implementation notes:
>  # Preloader cannot lock multiple cache entries at once, because this may lead to a deadlock
with concurrent batch updates. Therefore, it pre-creates batch of data rows in the page memory,
and then sequentially initializes the cache entries one by one.
>  # Batch writing of data rows into data pages uses the free list as usual because other
approaches increase memory fragmentation (for example, using only "reuse" or "most free" buckets).
>  # Eviction tracker assumes that only data pages with "heads" of fragmented data row
are tracked, so all other fragments of large data row should be written on separate data pages
(without other data rows which may cause page tracking).

This message was sent by Atlassian Jira

View raw message