ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Semen Boikov (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (IGNITE-4552) Optimize GridDhtLocalPartition.rmvQueue
Date Wed, 01 Feb 2017 08:59:51 GMT

     [ https://issues.apache.org/jira/browse/IGNITE-4552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Semen Boikov reassigned IGNITE-4552:

    Assignee: Stanilovsky Evgeny  (was: Semen Boikov)

Reviewed changes, my comments:
- it does not compile with jdk7, please switch to jdk7
- 'while (rmvQueue.peekFirst().expireTime() < U.currentTimeMillis())' - you need check
result of 'peekFirst()' for null
- do not schedule cleanup task per-partition, it should be single task per node
- error in 'onDeferredDelete', now if size > limit you remove all itmes from queue, actually
you need pollFirst while size > limit
- queue.peekFirst.expireTime > 'current time' is necessary in 'onDeferredDelete' since
there can be too many expired entries and cleanup task will not be able to do cleanup fast
- please follow coding conventions (remove unused imports, add javadoc for all methods, fields,
'Override' must be on the same line with method)


> Optimize GridDhtLocalPartition.rmvQueue
> ---------------------------------------
>                 Key: IGNITE-4552
>                 URL: https://issues.apache.org/jira/browse/IGNITE-4552
>             Project: Ignite
>          Issue Type: Bug
>          Components: cache
>    Affects Versions: 1.6
>            Reporter: Alexei Scherbakov
>            Assignee: Stanilovsky Evgeny
>             Fix For: 2.0
>         Attachments: benchmark-put-remove-simultaneously.properties, Plot_ThroughputLatencyProbe_01_31_origin.png,
Plot_ThroughputLatencyProbe_01_4552_mine.png, Plot_ThroughputLatencyProbe_01_mine_3node.png,
Plot_ThroughputLatencyProbe_01_mine.png, Plot_ThroughputLatencyProbe_01_origin_3node.png,
Plot_ThroughputLatencyProbe_01_origin.png, Screenshot_20170124_155355.png
> Current implementation stores deferred entry removals in rmvQueue for consistency guaranties.
> This can lead to significant heap over-usage(I observed several Gbs) in case of many
caches with removals, because currently queue is cleared lazily after reaching max capacity(200_000
by default).
> This can be mitigated by using lower IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE, but can
lead to consistency issues in case of frequent cache updates.
> Possible optimizations:
> * Use single fixed size queue per all caches to overcome limitations of IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE
> * Do queue cleaning in background
> * Move queue to an off-heap.

This message was sent by Atlassian JIRA

View raw message