ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ranjit Sahu <ranjit.s...@gmail.com>
Subject Help needed
Date Fri, 10 Feb 2017 03:23:05 GMT
Hi Guys,

We are trying to use ignite key-value store inside the Spark cluster. What
we are to do is :

1) Load the data to a data frame in spark
2) While doing transformation on the data frame, on each executor node we
are starting the ignite cache and load the data.


The issue what we are seeing is , when i start  with 40 executor nodes, i
can see only 24+ nodes in ignite topology. There is nothing in the log with
exceptions and we are not seeing the full data available in the cache too.

We were not able to use SharedRDD of ignite as we are kind of in a Nested
RDD  situation and too avoid that we are trying to work with Ignite cache.

Any clue what's going wrong here.

Thanks,
Ranjit

Mime
View raw message