crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Danushka Menikkumbura <danushka.menikkumb...@gmail.com>
Subject Re: Spark Examples
Date Mon, 12 Jan 2015 23:13:19 GMT
Aha!. This is perfect.

Thanks J!

Danushka

On Mon, Jan 12, 2015 at 5:57 PM, Josh Wills <jwills@cloudera.com> wrote:

> Hey Danushka,
>
> The spark branch on my crunch-demo app has a Spark example as well:
>
> https://github.com/jwills/crunch-demo/tree/spark
>
> I/you would need to update the POM to work against version 0.11.0-hadoop2
> and the latest Spark release, either Spark 1.1 or 1.2 should work fine.
>
> J
>
> On Mon, Jan 12, 2015 at 2:27 PM, Danushka Menikkumbura <
> danushka.menikkumbura@gmail.com> wrote:
>
> > Thanks Jinal!
> >
> > On Mon, Jan 12, 2015 at 4:43 PM, Jinal Shah <jinalshah2007@gmail.com>
> > wrote:
> >
> > > Hi Danushka,
> > >
> > > The user guide might be a good resource to start with
> > > https://crunch.apache.org/user-guide.html#sparkpipeline
> > >
> > > You can use the Crunch WordCount example:
> > >
> > >
> >
> https://github.com/apache/crunch/blob/master/crunch-archetype/src/main/resources/archetype-resources/src/main/java/WordCount.java
> > >
> > > These are some blogs you can use to learn
> > > http://crunch-tutor.blogspot.com/2014/12/crunch-dofn-example.html
> > > http://engineering.cerner.com/blog/scaling-people-with-apache-crunch/
> > >
> > >
> > > Thanks
> > > Jinal
> > >
> > > On Mon, Jan 12, 2015 at 1:19 PM, Danushka Menikkumbura <
> > > danushka.menikkumbura@gmail.com> wrote:
> > >
> > > > Hi devs,
> > > >
> > > > I am trying to understand the behavior of Crunch with Spark. In order
> > to
> > > do
> > > > that I need to run an example so that I can step through the code.
> > Could
> > > > someone point me to any resources that you think would be helpful
> > > please?.
> > > >
> > > > Thanks,
> > > > Danushka
> > > >
> > >
> >
>
>
>
> --
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message