beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amit Sela <amitsel...@gmail.com>
Subject Re: Beam Spark/Flink runner with DC/OS
Date Sun, 22 Jan 2017 09:32:46 GMT
I'lll join JB's comment on the Spark runner saying that submitting Beam
pipelines using the Spark runner can be done using Spark's spark-submit
script, find out more in the Spark runner documentation
<https://beam.apache.org/documentation/runners/spark/>.

Amit.

On Sun, Jan 22, 2017 at 8:03 AM Jean-Baptiste Onofré <jb@nanthrax.net>
wrote:

> Hi,
>
> Not directly DCOS (I think Stephen did some test on it), but I have a
> platform running Spark and Flink with Beam on Mesos + Marathon.
>
> It basically doesn't have anything special as running piplines uses
> spark-submit (as on in Spark "natively").
>
> Regards
> JB
>
> On 01/22/2017 12:56 AM, Chaoran Yu wrote:
> > Hello all,
> >
> >   Has anyone had experience using Beam on DC/OS? I want to run Beam code
> >
> > executed with Spark runner on DC/OS. As a next step, I would like to run
> the
> >
> > Flink runner as well. There doesn't seem to exist any information
> > about running
> >
> > Beam on DC/OS I can find on the web. So some pointers are greatly
> > appreciated.
> >
> > Thank you,
> >
> > Chaoran Yu
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Mime
View raw message