spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ConcreteVitamin <concretevita...@gmail.com>
Subject Re: SparkR: filter() function?
Date Thu, 30 Jan 2014 02:49:49 GMT
I opened an issue
(https://github.com/amplab-extras/SparkR-pkg/issues/14) to track this.

Zongheng

On Wed, Jan 29, 2014 at 6:11 PM, Shivaram Venkataraman
<shivaram@eecs.berkeley.edu> wrote:
> You could do a mapPartitions and Filter within each partition -- Something like
>
> numbers <- parallelize(sc, 1:20, 4L)
> evenNumbers <- lapplyPartition(numbers, function(part) { Filter(
> function(x) { x%%2 == 0} , part) })
> collect(evenNumbers)
>
> It should be simple to add this to the SparkR API as well -- Let me
> know if you want to send a PR !
>
> Thanks
> Shivaram
>
> On Wed, Jan 29, 2014 at 5:45 PM, Justin Lent <justinlent@gmail.com> wrote:
>> any idea when the Scala filter() function equivalent will be availalbe
>> in the SparkR implementation? Or is there a simple way to implement it
>> in R with the existing functions?
>>
>> Thanks!
>> -Justin

Mime
View raw message