ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joel Lang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (IGNITE-7812) Slow rebalancing in case of enabled persistence
Date Sat, 07 Apr 2018 06:03:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-7812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16429252#comment-16429252
] 

Joel Lang edited comment on IGNITE-7812 at 4/7/18 6:02 AM:
-----------------------------------------------------------

I've found that this happens after page eviction begins for the cache. It seems to be related
to file reads in PageMemoryImpl.acquirePage(), primarily as a result of the H2 query index
being updated when new rows are received during rebalancing. The segment write lock is held
during these read operations which dramatically slows down other operations. It was a crippling
slowdown in my test case (50x or more) which was done on a development VM which uses a HDD,
not a SSD. We would be looking at 1-2 days of rebalance time for a cache with 6 million entries
and 2 indexes in that case. While we do plan to use a SSD for production, this is still an
excessive slowdown.

Fortunately, it seems as if this has already been fixed for 2.5, the disk read happens after
the segment write lock is released now: [PageMemoryImpl.java#L750|https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/pagemem/PageMemoryImpl.java#L750]


was (Author: langj):
I've found that this happens after page eviction begins for the cache. It seems to be related
to file reads in PageMemoryImpl.acquirePage(), primarily as a result of the H2 query index
being updated when new rows are received during rebalancing. The segment write lock is held
during these read operations which dramatically slows down other operations. It was a crippling
slowdown in my test case (50x or more) which was done on a development VM which uses a HDD,
not a SSD. We would be looking at 1-2 days of rebalance time for a cache with 6 million entries
and 2 indexes in t hat case.

Fortunately, it seems as if this has already been fixed for 2.5, the disk read happens after
the segment write lock is released now: [PageMemoryImpl.java#L750|https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/pagemem/PageMemoryImpl.java#L750]

> Slow rebalancing in case of enabled persistence
> -----------------------------------------------
>
>                 Key: IGNITE-7812
>                 URL: https://issues.apache.org/jira/browse/IGNITE-7812
>             Project: Ignite
>          Issue Type: Task
>            Reporter: Alexey Goncharuk
>            Assignee: Pavel Kovalenko
>            Priority: Major
>
> A user reported that rebalancing take significantly larger amounts of time when persistence
is enabled even in LOG_ONLY mode.
> Need to investigate how the performance of rebalancing may be increased.
> Also, it would be great to estimate the benefit of file transfer for rebalancing.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message