ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paolo Di Tommaso <paolo.ditomm...@gmail.com>
Subject Ignite - Spark integration
Date Sun, 12 Jun 2016 13:24:37 GMT
Hi,

I'm giving a try to the Spark integration provided by Ignite by using the
embedded deployment mode described here
<https://apacheignite-fs.readme.io/docs/installation-deployment>.

 I've setup a local cluster made up a master and a worker node.

This is my basic Ignite-Spark application:

public class JavaLaunchIgnite {

    static public void main(String... args) {
        // -- spark context
        SparkConf sparkConf = new SparkConf().setAppName("Spark-Ignite");
        JavaSparkContext sc = new JavaSparkContext(sparkConf);

        // -- ignite configuration
        IgniteOutClosure cfg = new IgniteOutClosure() {
            @Override public Object apply() {
                return new IgniteConfiguration();
            }};
        // -- ignite context
        JavaIgniteContext<Integer,Integer> ic = new
JavaIgniteContext<Integer,Integer>(sc, cfg);
        final Ignite ignite = ic.ignite();
        ic.ignite().compute().broadcast(new IgniteRunnable() {
            @Override public void run() {
                System.out.println(">>> Hello Node: " +
ignite.cluster().localNode().id());
            }});

        ic.close(true);
        System.out.println(">>> DONE");
    }
}

However when I submit it it simply hangs. By using the Spark web console, I
can see that the application is correctly deployed and running but it never
stops.

In the Spark worker node I can find any log produced by Ignite (which is
supposed to deploy an Ignite worker). See here
<http://pastebin.com/KdEA0KUq>.

Instead I can see the Ignite output in the log of the spark-submit log. See
here <http://pastebin.com/Ff6fxYBF>.


Does anybody have any clue why this app just hangs?


Cheers,
Paolo

Mime
View raw message