ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Denis Mekhanikov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-9196) SQL: Memory leak in MapNodeResults
Date Mon, 13 Aug 2018 13:58:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-9196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16578311#comment-16578311
] 

Denis Mekhanikov commented on IGNITE-9196:
------------------------------------------

[~vozerov] we can do that. We could look at contents of the following map: {{MapNodeResults#res}}
 To get a handle of it, you need to follow a pretty long chain of links: {{GridQueryProcessor->idx->mapQryExec->qryRess->res}}.
Each one would be accessed via reflection, since all the fields on the way are private and
don't have accessors.

It would be too implementation-dependent. I don't really like the idea of digging through
an incapsulated code via reflection.

Testing a memory leak by comparing sizes of heap dumps seems less ugly to me.

There is already a test, that checks for memory leaks this way: [MemoryLeaksOnRestartNodeTest|https://github.com/apache/ignite/blob/9a4a145be514e650258715a7e682d427d5812d16/modules/core/src/test/java/org/apache/ignite/internal/MemoryLeaksOnRestartNodeTest.java].
This test is not added to any test suite. I guess, it's because it takes too much time to
finish. How about creating a test suite, that is run only once in a while? We could add such
long-running tests there and run them e.g. before each release.

> SQL: Memory leak in MapNodeResults
> ----------------------------------
>
>                 Key: IGNITE-9196
>                 URL: https://issues.apache.org/jira/browse/IGNITE-9196
>             Project: Ignite
>          Issue Type: Bug
>          Components: sql
>    Affects Versions: 2.6
>            Reporter: Denis Mekhanikov
>            Assignee: Denis Mekhanikov
>            Priority: Blocker
>              Labels: sql-stability
>             Fix For: 2.7
>
>
> When size of a SQL query result set is a multiple of {{Query#pageSize}}, then {{MapQueryResult}}
is never closed and removed from {{MapNodeResults#res}} collection.
> The following code leads to OOME when run with 1Gb heap:
> {code:java}
> public class MemLeakRepro {
>     public static void main(String[] args) {
>         Ignition.start(getConfiguration("server"));
>         try (Ignite client = Ignition.start(getConfiguration("client").setClientMode(true)))
{
>             IgniteCache<Integer, Person> cache = startPeopleCache(client);
>             int pages = 10;
>             int pageSize = 1024;
>             for (int i = 0; i < pages * pageSize; i++) {
>                 Person p = new Person("Person #" + i, 25);
>                 cache.put(i, p);
>             }
>             for (int i = 0; i < 1_000_000; i++) {
>                 if (i % 1000 == 0)
>                     System.out.println("Select iteration #" + i);
>                 Query<List<?>> qry = new SqlFieldsQuery("select * from people");
>                 qry.setPageSize(pageSize);
>                 QueryCursor<List<?>> cursor = cache.query(qry);
>                 cursor.getAll();
>                 cursor.close();
>             }
>         }
>     }
>     private static IgniteConfiguration getConfiguration(String instanceName) {
>         IgniteConfiguration igniteCfg = new IgniteConfiguration();
>         igniteCfg.setIgniteInstanceName(instanceName);
>         TcpDiscoverySpi discoSpi = new TcpDiscoverySpi();
>         discoSpi.setIpFinder(new TcpDiscoveryVmIpFinder(true));
>         return igniteCfg;
>     }
>     private static IgniteCache<Integer, Person> startPeopleCache(Ignite node) {
>         CacheConfiguration<Integer, Person> cacheCfg = new CacheConfiguration<>("cache");
>         QueryEntity qe = new QueryEntity(Integer.class, Person.class);
>         qe.setTableName("people");
>         cacheCfg.setQueryEntities(Collections.singleton(qe));
>         cacheCfg.setSqlSchema("PUBLIC");
>         return node.getOrCreateCache(cacheCfg);
>     }
>     public static class Person {
>         @QuerySqlField
>         private String name;
>         @QuerySqlField
>         private int age;
>         public Person(String name, int age) {
>             this.name = name;
>             this.age = age;
>         }
>     }
> }
> {code}
>  
> At the same time it works perfectly fine, when there are, for example, {{pages * pageSize
- 1}} records in cache instead.
> The reason for it is that {{MapQueryResult#fetchNextPage(...)}} method doesn't return
true, when the result set size is a multiple of the page size.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message