Mailing-List: contact user-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@ignite.apache.org
Date: Tue, 16 Feb 2016 21:02:34 -0800 (PST)
From: vkulichenko <valentin.kulichenko@gmail.com>
To: user@ignite.apache.org
Message-ID: <1455685354714-3050.post@n6.nabble.com>
In-Reply-To: <1455677540081-3049.post@n6.nabble.com>
References: <1455677540081-3049.post@n6.nabble.com>
Subject: Re: Huge heap size for ignite cache when storing 20 million objects
 of small size
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hi vidhu,

First of all, can you please subscribe to the mailing list so that community
receives email notifications? Please follow this simple instruction:
http://apache-ignite-users.70518.x6.nabble.com/mailing_list/MailingListOptions.jtp?forum=1


vidhu wrote
> I am facing a problem where my heap reaches to about 2.8 gb when I am
> storing 20 million objects. Size  of each object is approx. 462 bytes.
> Structure of my object is a simple pojo having 101 fields each of int
> type.
> 
> I tried using following 3 marshalling techniques:
> 
> 1.) Default marshaller of Ignite. (Size of heap ~ 2.8 gb, looking with
> Visual VM find that most of objects on heap are byte array.)
> 
> 2.) Optimized marshaller. (Again size was ~ 2.8 gb, looking with Visual VM
> find that this time size of objects on heap of byte array is half + there
> are lot of objects of class that I put in cache. )
> 
> 3.) Tried implementing Binarylizable in my class that I am storing in
> cache. I used rawWriter. This time size reduced considerably but I was not
> able to read values out of cache, (getting null on read). 
> 
> I am implementing this in java version 1.8.

20,000,000 * 462 bytes is already around 8.6GB of memory, so 2.8 is not
enough even without Ignite's overhead. Did you try to allocate more heap
memory and/or start more nodes? With your sizings I would start with two
nodes, 8GB of memory per node.

As for options you tried, I would recommend to use default binary marshaller
because it's much more compact. Also you should take a look at offheap
memory [1]. It can be useful if you're going to have a lot of data.

[1] https://apacheignite.readme.io/docs/off-heap-memory

-Val


--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Huge-heap-size-for-ignite-cache-when-storing-20-million-objects-of-small-size-tp3049p3050.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.