mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Iulian DragoČ™ <iulian.dra...@typesafe.com>
Subject Re: spark on mesos with docker issue
Date Thu, 22 Oct 2015 20:28:08 GMT
On Thu, Oct 22, 2015 at 9:10 PM, Rad Gruchalski <radek@gruchalski.com>
wrote:

> Stavros,
>
> Spark does not support this. I am currently in the process of submitting
> patches for it however it first has to pass through the legal team at the
> company I work for.
>

What exactly is missing in Spark?


> Kind regards,
> Radek Gruchalski
> radek@gruchalski.com <radek@gruchalski.com>
> de.linkedin.com/in/radgruchalski/
>
>
> *Confidentiality:*This communication is intended for the above-named
> person and may be confidential and/or legally privileged.
> If it has come to you in error you must take no action based on it, nor
> must you copy or show it to anyone; please delete/destroy and inform the
> sender immediately.
>
> On Thursday, 22 October 2015 at 21:08, Stavros Kontopoulos wrote:
>
> Bridge... with the latest mesos library vesion 0.25...
>
> On Thu, Oct 22, 2015 at 9:07 PM, Elizabeth Lingg <elizabeth@mesosphere.io>
> wrote:
>
> Are you using Bridge or Host Networking?
>
> -Elizabeth
>
>
>
> On Thu, Oct 22, 2015 at 12:02 PM, Stavros Kontopoulos <
> st.kontopoulos@gmail.com> wrote:
>
> Hi,
>
> Im using spark on mesos on docker. I have linked my slaves to the master
> and a
> spark repl works fine inside the master container.
>
> If i try to crate the same spark repl form the host i get stuck at the
> point when the framework tries to register to the mesos master (here the
> framework is the spark repl itself).
> I can ping the container from my host and vice versa. So networking its
> not the problem.
> What i noticed form the logs is that mesos does not resolve the correct ip:
>
> Framework failover timeout, removing framework
> b3605c33-f573-4d40-806f-b9b0abee2e32-0012 (Spark shell) at
> scheduler-b6dbe4f8-4760-4cbe-bb5e-a991e35aba1b@127.0.1.1:40186
>
> docker0 interface is on 172.17.x.x and my host is one such ip so i didnt
> expect there to see
> 127.0.1.1. I have tried several things like spark.driver.host,
> SPARK_LOCAL_IP to be set correctly but with no result...
> I suspect this is a mesos problem on docker...
>
> Thnx,
>
> S.
>
>
>
>
>


-- 

--
Iulian Dragos

------
Reactive Apps on the JVM
www.typesafe.com

Mime
View raw message