ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ranjit Sahu <ranjit.s...@gmail.com>
Subject Re: Help needed
Date Wed, 15 Feb 2017 11:24:19 GMT
Yep, i will change that , so that we can load to basically all nodes. It
will end up again as my initial function. Start and load on each executor.
One more thing i noticed is , when i connect as Client and load it is 150+%
slow. Any pointer here ?

We have a shared service in java which is used for entity extarction and
resolution . We are working on making it spark friendly.
The resolution function basically needs an accesor-> DAO and connects to
external resource for functioning.
the idea is to use ignite in place of it. The flow is something like this.
call on one rdd of input data the resolve function. Inside resolve function
use ignite to query and resolve to correct entity.


The blocker for us now is, when i start the executor nodes , suppose 20,
they(all) are not joining each other with ignite. I can see many sub groups
sometimes also.
What should we look at. I did not configure any Static ip for discovery
here as resources are managed by yarn for spark.

I have tried the multicast group address which we use in another project
for oracle coherence. That is also not helping.


Thanks a ton  and i appreciate your help and quick response for my emails.

Thanks,
Ranjit



On Tue, Feb 14, 2017 at 9:54 PM, vkulichenko <valentin.kulichenko@gmail.com>
wrote:

> Well, a node will be started only on executors where getOrStart is called.
> So
> if number of nodes is less then number of executors, it means that you
> didn't visit all executors when mapping the original RDD.
>
> What is the business use case behind that? I'm not sure you chose a right
> approach.
>
> -Val
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Help-needed-tp10540p10636.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Mime
View raw message