spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From cjwang>
Subject Re: Finding previous and next element in a sorted RDD
Date Fri, 22 Aug 2014 01:13:39 GMT
One way is to do zipWithIndex on the RDD.  Then use the index as a key.  Add
or subtract 1 for previous or next element.  Then use cogroup or join to
bind them together.

val idx = input.zipWithIndex
val previous = => (x._2+1, x._1))
val current = => (x._2, x._1))
val next = => (x._2-1, x._1))

val joined = current leftOuterJoin previous leftOuterJoin next

Code looks clean to me, but I feel uneasy about the performance of join.  

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message