hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Kim <bbuil...@gmail.com>
Subject Re: Steps to Run Spark Scala job from Oozie on EC2 Hadoop clsuter
Date Mon, 07 Mar 2016 15:52:02 GMT
To comment…

At my company, we have not gotten it to work in any other mode than local. If we try any of
the yarn modes, it fails with a “file does not exist” error when trying to locate the
executable jar. I mentioned this to the Hue users group, which we used for this, and they
replied that the Spark Action is very basic implementation and that they will be writing their
own for production use.

That’s all I know...

> On Mar 7, 2016, at 1:18 AM, Deepak Sharma <deepakmca05@gmail.com> wrote:
> 
> There is Spark action defined for oozie workflows.
> Though I am not sure if it supports only Java SPARK jobs or Scala jobs as well.
> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html <https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html>
> Thanks
> Deepak
> 
> On Mon, Mar 7, 2016 at 2:44 PM, Divya Gehlot <divya.htconex@gmail.com <mailto:divya.htconex@gmail.com>>
wrote:
> Hi,
> 
> Could somebody help me by providing the steps /redirect me  to blog/documentation on
how to run Spark job written in scala through Oozie.
> 
> Would really appreciate the help.
> 
> 
> 
> Thanks,
> Divya 
> 
> 
> 
> -- 
> Thanks
> Deepak
> www.bigdatabig.com <http://www.bigdatabig.com/>
> www.keosha.net <http://www.keosha.net/>

Mime
View raw message