hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Randles <trand...@lanl.gov>
Subject Re: Spark vs Tez
Date Tue, 21 Oct 2014 14:04:55 GMT
Yeah, compared to something as performant as java...
</sarcasm>

On 10/20/2014 10:16 PM, Adaryl "Bob" Wakefield, MBA wrote:
> Using an interpreted scripting language with something that is billing
> itself as being fast doesn’t sound like the best idea...
> B.
> *From:* Russell Jurney <mailto:russell.jurney@gmail.com>
> *Sent:* Saturday, October 18, 2014 7:38 AM
> *To:* user@hadoop.apache.org <mailto:user@hadoop.apache.org>
> *Subject:* Re: Spark vs Tez
> Check out PySpark. No Scala required.
>
> On Friday, October 17, 2014, Adaryl "Bob" Wakefield, MBA
> <adaryl.wakefield@hotmail.com <mailto:adaryl.wakefield@hotmail.com>> wrote:
>
>     “The only problem with Spark adoption is the steep learning curve of
>     Scala , and understanding the API properly.”
>     This is why I’m looking for reasons to avoid Spark. In my mind, it’s
>     one more thing to have to master and doesn’t really have anything to
>     offer that can’t be done with other tools that are already inside my
>     skillset. I spoke with some software engineers recently and
>     basically the discussion boiled down to if you need to master Java
>     or Scala go with Java. Three months into Java I don’t want to stop
>     that and start learning Scala.
>     B.
>     *From:* kartik saxena
>     <javascript:_e(%7B%7D,'cvml','kartik.sxn@gmail.com');>
>     *Sent:* Friday, October 17, 2014 1:12 PM
>     *To:* javascript:_e(%7B%7D,'cvml','user@hadoop.apache.org');
>     *Subject:* Re: Spark vs Tez
>     I did a performance benchmark during my summer internship . I am
>     currently a grad student. Can't reveal much about the specific
>     project but Spark is still faster than around 4-5th iteration of Tez
>     of the same query/dataset. By Iteration I mean utilizing the
>     "hot-container" property of Apache Tez  . See latest release of Tez
>     and some hortonworks tutorials on their website.
>     The only problem with Spark adoption is the steep learning curve of
>     Scala , and understanding the API properly.
>     Thanks
>     On Fri, Oct 17, 2014 at 11:06 AM, Adaryl "Bob" Wakefield, MBA
>     <javascript:_e(%7B%7D,'cvml','adaryl.wakefield@hotmail.com');> wrote:
>
>         Does anybody have any performance figures on how Spark stacks up
>         against Tez? If you don’t have figures, does anybody have an
>         opinion? Spark seems so popular but I’m not really seeing why.
>         B.
>
>
>
> --
> Russell Jurney twitter.com/rjurney
> <http://twitter.com/rjurney>russell.jurney@gmail.com
> <mailto:russell.jurney@gmail.com> datasyndrome.com
> <http://datasyndrome.com/>

Mime
View raw message