ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhengqingzheng <zhengqingzh...@huawei.com>
Subject re: Ignite cache data size problem.
Date Wed, 27 Apr 2016 03:06:15 GMT
Hi Vladimir,
Sorry to reply so late.  The loadCache process took 8 hours to load all the data (this time
,there is no exception occurred, but memory consumption up to 56gb, 80% of all the heap that
I have defined, which include 10 nodes, each node allocate 7gb heap).
I give you all the log files which were located at work/log/ folder. Please see the attachment.


发件人: Vladimir Ozerov [mailto:vozerov@gridgain.com]
发送时间: 2016年4月25日 20:21
收件人: user@ignite.apache.org
主题: Re: Ignite cache data size problem.

Hi Kevin,

I performed several experiments. Essentially, I put 1M entries of the class you provided with
fields initialized as follows:


for (int i = 0; i < 1_000_000; i++) {
    UniqueField field = new UniqueField();

    field.setDate(new Date());
    field.setGuid(UUID.randomUUID().toString());
    field.setMsg(String.valueOf(i));
    field.setNum(BigDecimal.valueOf(ThreadLocalRandom.current().nextDouble()));
    field.setOId(String.valueOf(i));
    field.setOrgId(String.valueOf(i));

    cache.put(i, field);
}

My results are:
1) Onheap, no indexes - about 400Mb is required to store 1M objects, or ~20Gb for 47M objects.
2) Onheap, with indexes - about 650Mb, or ~30Gb for 47M objects.
3) Offheap, with indexes - about 400Mb offheap memory is required, or ~20Gb for all your objects.

Could you please provide more information on the error you receive? Also, you could try load
entries in a batches of a well-defined size (say, 1M), and see what happens to the system.
I expect you should see similar numbers.

Vladimir.

On Fri, Apr 22, 2016 at 3:26 PM, kevin.zheng <zhengqingzheng@huawei.com<mailto:zhengqingzheng@huawei.com>>
wrote:
BTW, I created 4 + 3 nodes on two servers.
each node I called command like this ./ignite.sh  -J-Xmx8g -J-Xms8g

kind regards,
Kevin



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-cache-data-size-problem-tp4449p4454.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Mime
View raw message