mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dick Davies <d...@hellooperator.net>
Subject Re: spark on mesos.
Date Fri, 27 Feb 2015 21:02:57 GMT
What do you mean by spark daemons?

the spark shell (or any other spark application) acts as a Mesos framework,
so until that's running spark isn't 'on' Mesos.

On 27 February 2015 at 16:23, Dan Dong <dongdan39@gmail.com> wrote:
> Hi, All,
>   When I configured and started spark daemons, why I could not see it in
> "Active Frameworks" on Mesos UI(for Hadoop, it will show up immediately when
> hadoop daemons started)?. I can see the spark framework on mesos UI only
> when I run the spark-shell command interactively. Is it normal?
>
> export
> MESOS_NATIVE_LIBRARY=/home/ubuntu/mesos-0.21.0/build/lib/libmesos-0.21.0.so
> export
> PROTOBUF_JAR=/home/ubuntu/hadoop-2.5.0-cdh5.2.0/protobuf-java-2.5.0.jar
> export MESOS_JAR=/home/ubuntu/hadoop-2.5.0-cdh5.2.0//mesos-0.21.0.jar
> export
> SPARK_EXECUTOR_URI=hdfs://clus-1:9000/user/ubuntu/spark-1.1.1-bin-hadoop2.4_mesos.tar.gz
> export MASTER=mesos://clus-1:5050/mesos
>
>
> Cheers,
> Dan
>

Mime
View raw message