hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neelesh Salian <nsal...@cloudera.com>
Subject Re: Steps to Run Spark Scala job from Oozie on EC2 Hadoop clsuter
Date Mon, 07 Mar 2016 15:58:09 GMT
Hi Divya,

This link should have the details that you need to begin using the Spark
Action on Oozie:
https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html

Thanks.

On Mon, Mar 7, 2016 at 7:52 AM, Benjamin Kim <bbuild11@gmail.com> wrote:

> To comment…
>
> At my company, we have not gotten it to work in any other mode than local.
> If we try any of the yarn modes, it fails with a “file does not exist”
> error when trying to locate the executable jar. I mentioned this to the Hue
> users group, which we used for this, and they replied that the Spark Action
> is very basic implementation and that they will be writing their own for
> production use.
>
> That’s all I know...
>
> On Mar 7, 2016, at 1:18 AM, Deepak Sharma <deepakmca05@gmail.com> wrote:
>
> There is Spark action defined for oozie workflows.
> Though I am not sure if it supports only Java SPARK jobs or Scala jobs as
> well.
> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html
> Thanks
> Deepak
>
> On Mon, Mar 7, 2016 at 2:44 PM, Divya Gehlot <divya.htconex@gmail.com>
> wrote:
>
>> Hi,
>>
>> Could somebody help me by providing the steps /redirect me  to
>> blog/documentation on how to run Spark job written in scala through Oozie.
>>
>> Would really appreciate the help.
>>
>>
>>
>> Thanks,
>> Divya
>>
>
>
>
> --
> Thanks
> Deepak
> www.bigdatabig.com
> www.keosha.net
>
>
>


-- 
Neelesh Srinivas Salian
Customer Operations Engineer

Mime
View raw message