ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yair Ogen <yairo...@gmail.com>
Subject inflated cache size
Date Sun, 29 Oct 2017 07:23:15 GMT

I'm trying out the ignite-spark support. I have a dataframe that was
created from reading a csv file sized around 800MB.

It seems that When I store the rdd from this dataframe in ignite using
saveValues api in IgniteContext it takes around 2GB of RAM.

Naturally once we add more dataframes, joins and computations we get OOM
errors even though we have more than enough RAM.

Any ideas why the inflated memory?

Attached is my config.


View raw message