spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mayur Rustagi <mayur.rust...@gmail.com>
Subject Re: What's the difference between map and transform in spark streaming?
Date Thu, 27 Feb 2014 21:46:24 GMT
DStream -> RDD ->partitions ->  rows

map works on each rows
mapPartitions: works on each partition
transform: each rdd

Mayur Rustagi
Ph: +919632149971
h <https://twitter.com/mayur_rustagi>ttp://www.sigmoidanalytics.com
https://twitter.com/mayur_rustagi



On Thu, Feb 27, 2014 at 1:42 PM, Aureliano Buendia <buendia360@gmail.com>wrote:

> Hi,
>
> Is transform supposed to be a generalized form of map? They seem to be
> doing the same job.
>

Mime
View raw message