flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Márton Balassi <balassi.mar...@gmail.com>
Subject Re: Counting the number of elements in a dataset
Date Sat, 22 Nov 2014 19:17:12 GMT
Hey,

There was a thread recently on the dev list that might be interesting to
you [1].
I do not know the exact state of the code though.

[1]
http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/Hi-Aggregation-support-td2311.html#a2547

Cheers,

Marton

On Sat, Nov 22, 2014 at 8:09 PM, Sebastian Schelter <ssc@apache.org> wrote:

> Hi,
>
> Is there a simple way to count the number of elements of a dataset? At the
> moment, I have to use the following code, which is pretty verbose and
> unefficient.
>
>     val numVertices =
>       (srcVertices union targetVertices).distinct.reduceGroup { iter =>
>         var count = 1L
>         while (iter.hasNext) {
>           count += 1
>           iter.next
>         }
>         count
>       }
>
> Best,
> Sebastian
>

Mime
View raw message