beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aviem Zur (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (BEAM-775) Remove Aggregators from the Java SDK
Date Wed, 22 Mar 2017 03:47:41 GMT

    [ https://issues.apache.org/jira/browse/BEAM-775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15935726#comment-15935726
] 

Aviem Zur edited comment on BEAM-775 at 3/22/17 3:46 AM:
---------------------------------------------------------

As [~aljoscha] mentioned we are adding this assertion to Flink runner as well, and it has
surfaced bugs which were silenced by the fact that the tests passed even though no {{PAssert}}
assertions actually happened -  https://github.com/apache/beam/pull/2263
All runners should add such an assertion as well, as they might also have silent bugs.
There is a ticket to change to doing this assertion in the {{TestPipeline}} side of things
so that runner writers won't have to know to do this: https://issues.apache.org/jira/browse/BEAM-1763
In any case, once all runners support metrics this can be done using metrics. 


was (Author: aviemzur):
As [~aljoscha] mentioned we are adding this assertion to Flink runner as well, and it has
surfaced bugs which were silenced by the fact that the tests passed even though no {{PAssert}}
assertions actually happened -  https://github.com/apache/beam/pull/2263
All runners should add such an assertion as well, as they might also have silent bugs.
There is a ticket to change to doing this assertion in the {{TestPipeline}} side of things
so that runner writers won't have to know to do this: https://issues.apache.org/jira/browse/BEAM-1763
In any case, once all runners support metrics this can be done using metrics. Until then the
only way to do this is with aggregators.

> Remove Aggregators from the Java SDK
> ------------------------------------
>
>                 Key: BEAM-775
>                 URL: https://issues.apache.org/jira/browse/BEAM-775
>             Project: Beam
>          Issue Type: Sub-task
>          Components: sdk-java-core
>            Reporter: Ben Chambers
>            Assignee: Pablo Estrada
>              Labels: backward-incompatible
>             Fix For: First stable release
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message