ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stanilovsky Evgeny (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-4552) Optimize GridDhtLocalPartition.rmvQueue
Date Thu, 26 Jan 2017 09:55:24 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-4552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15839491#comment-15839491
] 

Stanilovsky Evgeny commented on IGNITE-4552:
--------------------------------------------

all done, take a look into PR plz.

https://github.com/apache/ignite/pull/1465

i understand that use tuples instead of wrappers is bad style for java, if it so i`m ready
for refactoring.
i write my own simple tests for this case, did i need to add it, there are very simple code
that creates 10 caches and sequentially add and remove item in the loop?
tests generate more items than removed queue is configured for, thus i tests not only timeout
eviction, but queue remainingCapacity overhead too.

my local results are :
original code:                            common working time: 143580 (ms), heap size after:
407Mb
refactoring linkeq queue code: common working time: 143822 (ms), heap size after: 74Mb (like
on grid startup moment).

wait for criticism.


> Optimize GridDhtLocalPartition.rmvQueue
> ---------------------------------------
>
>                 Key: IGNITE-4552
>                 URL: https://issues.apache.org/jira/browse/IGNITE-4552
>             Project: Ignite
>          Issue Type: Bug
>          Components: cache
>    Affects Versions: 1.6
>            Reporter: Alexei Scherbakov
>            Assignee: Stanilovsky Evgeny
>             Fix For: 2.0
>
>         Attachments: Screenshot_20170124_155355.png
>
>
> Current implementation stores deferred entry removals in rmvQueue for consistency guaranties.
> This can lead to significant heap over-usage(I observed several Gbs) in case of many
caches with removals, because currently queue is cleared lazily after reaching max capacity(200_000
by default).
> This can be mitigated by using lower IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE, but can
lead to consistency issues in case of frequent cache updates.
> Possible optimizations:
> * Use single fixed size queue per all caches to overcome limitations of IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE
workaround.
> * Do queue cleaning in background
> * Move queue to an off-heap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message