ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Mashenkov <andrey.mashen...@gmail.com>
Subject Potential OOM while iterating over query cursor. Review needed.
Date Fri, 29 Jun 2018 08:37:37 GMT
Hi Igniters,

There is an issue IGNITE-8892 [1] related to OOM during distributed query
execution.
This issue is not limited with ScanQuery usage and looks like affected all
query types.

The use case is quite simple. 1 server and 1 client.
Client starts scan query with default flags and iterate over cursor.
If whole query result is not fit to memory - JVM will crashed with OOM,
but it is not expected as client takes entries one by one and throw out
them immediately.
Reproducer is attached to the ticket.

Same query works fine if query starts on server. Seems, we have no
DistributedQueryFuture in that case and all works fine.

I've found GridCacheDistributedQueryFuture collects all entries and try to
return the collection via onDone().
This behaviour turn on with a flag 'keepAll' which is true by default.
Iterating over cache via cache.iterator() has no OOM issues as we set
keepAll flag to false.

Why we have keepAll=true by default as seems noone expects future.get()
will return any data and all queries works through queue in paging mode?
Will it be ok to get rid of 'allCol' and keepAll flag at all?

I've made a PR and TC looks fine.
Could someone review it, please?

[1] https://issues.apache.org/jira/browse/IGNITE-8892


-- 
Best regards,
Andrey V. Mashenkov

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message