ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Mashenkov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-8892) Iterating over large dataset via ScanQuery can fails with OOME.
Date Thu, 28 Jun 2018 13:29:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-8892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16526303#comment-16526303
] 

Andrew Mashenkov commented on IGNITE-8892:
------------------------------------------

I've found the root cause.

We have a collection that never cleared:  GridCacheQueryFutureAdapter.allCol and it is reachable
from GC root while GridCacheDistributedQueryFuture is reachable.

> Iterating over large dataset via ScanQuery can fails with OOME.
> ---------------------------------------------------------------
>
>                 Key: IGNITE-8892
>                 URL: https://issues.apache.org/jira/browse/IGNITE-8892
>             Project: Ignite
>          Issue Type: Bug
>          Components: cache
>            Reporter: Andrew Mashenkov
>            Priority: Critical
>              Labels: OutOfMemoryError
>             Fix For: 2.7
>
>         Attachments: ScanQueryOOM.java
>
>
> Seems, iterating over query iterator (ScanQuery at least, but may be other affected as
well) on client node cause memory leakage.
> The use case is quite simple.
>  Start server and client. Put much data into cache, then iterate over all entries via
ScanQuery.
>  Looks like JVM crashed due to OOM as GridCacheDistributedQueryManager.futs map contains
to many GridCacheDistributedQueryFuture futures.
> I've put 15kk entries into cache and client failed with OOM after iterating over 10kk
entry.
>  In heapdump I observer 2*10^9 GridCacheDistributedQueryFuture futures. 
> We have to check
>  # if these futures removed from map correctly.
>  # we don't create unnecessary futures.
> PFA repro.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message