spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From cjwang ...@cjwang.us>
Subject Re: Finding previous and next element in a sorted RDD
Date Fri, 22 Aug 2014 01:13:39 GMT
One way is to do zipWithIndex on the RDD.  Then use the index as a key.  Add
or subtract 1 for previous or next element.  Then use cogroup or join to
bind them together.

val idx = input.zipWithIndex
val previous = idx.map(x => (x._2+1, x._1))
val current = idx.map(x => (x._2, x._1))
val next = idx.map(x => (x._2-1, x._1))

val joined = current leftOuterJoin previous leftOuterJoin next

Code looks clean to me, but I feel uneasy about the performance of join.  



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Finding-previous-and-next-element-in-a-sorted-RDD-tp12621p12623.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message