ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jörn Franke <jornfra...@gmail.com>
Subject Re: Help needed
Date Fri, 10 Feb 2017 05:30:09 GMT
Not sure I got the picture of your setup, but the ignite cache should be started indecently
of the application and not within the application.

Aside from that, can you please elaborate more on the problem you like to solve - maybe with
pseudocode? I am not sure if the approach you have selected is the right one.

> On 10 Feb 2017, at 04:23, Ranjit Sahu <ranjit.sahu@gmail.com> wrote:
> Hi Guys,
> We are trying to use ignite key-value store inside the Spark cluster. What we are to
do is :
> 1) Load the data to a data frame in spark
> 2) While doing transformation on the data frame, on each executor node we are starting
the ignite cache and load the data. 
> The issue what we are seeing is , when i start  with 40 executor nodes, i can see only
24+ nodes in ignite topology. There is nothing in the log with exceptions and we are not seeing
the full data available in the cache too.
> We were not able to use SharedRDD of ignite as we are kind of in a Nested RDD  situation
and too avoid that we are trying to work with Ignite cache.
> Any clue what's going wrong here.
> Thanks,
> Ranjit

View raw message