spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gerard Maas <gerard.m...@gmail.com>
Subject Re: spark code style
Date Fri, 21 Nov 2014 09:46:21 GMT
I suppose that here function(x) = function3(function2(function1(x)))

In that case, the difference will be modularity and readability of your
program.
If function{1,2,3} are logically different steps and potentially reusable
somewhere else, I'd keep them separate.

A sequence of map transformations will be pipelined by Spark with little
overhead.

-kr, Gerard.

On Fri, Nov 21, 2014 at 10:20 AM, Kevin Jung <itsjb.jung@samsung.com> wrote:

> Hi all.
> Here are two code snippets.
> And they will produce the same result.
>
> 1.
> rdd.map( function )
>
> 2.
> rdd.map( function1 ).map( function2 ).map( function3 )
>
> What are the pros and cons of these two methods?
>
> Regards
> Kevin
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/spark-code-style-tp19463.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message