flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ufuk Celebi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-758) Add count method to DataSet and implement CountOperator
Date Mon, 07 Jul 2014 08:07:34 GMT

    [ https://issues.apache.org/jira/browse/FLINK-758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14053421#comment-14053421
] 

Ufuk Celebi commented on FLINK-758:
-----------------------------------

Yes.

But regarding the group reduce (grouped/ungrouped): I disagree that you can code the initial
value on your own in all cases as your UDF will not be called if the previous operator did
not produce anything, e.g. after a filter.

> Add count method to DataSet and implement CountOperator
> -------------------------------------------------------
>
>                 Key: FLINK-758
>                 URL: https://issues.apache.org/jira/browse/FLINK-758
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: GitHub Import
>              Labels: github-import
>             Fix For: pre-apache
>
>         Attachments: pull-request-758-7518001488867571817.patch
>
>
> At the request of @twalthr. This is the count operator I've implemented some time ago
to get the to know the new Java API. It introduces `DataSet.count()`, which is executed as
a map (to ones) and reduce (sum up the ones). I initially didn't do the PR, because of the
following problem: empty DataSets don't work as the first map won't have any input to operate
on.
> If more people think that we should include this operator we can think about a possible
solution to the problem.
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/stratosphere/stratosphere/pull/758
> Created by: [uce|https://github.com/uce]
> Labels: enhancement, java api, 
> Milestone: Release 0.6 (unplanned)
> Created at: Tue May 06 10:42:33 CEST 2014
> State: open



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message