beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kenneth Knowles (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-2120) TestDataflowRunner prints all messages from the job, repeatedly
Date Fri, 28 Apr 2017 23:55:04 GMT

    [ https://issues.apache.org/jira/browse/BEAM-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15989598#comment-15989598
] 

Kenneth Knowles commented on BEAM-2120:
---------------------------------------

Yes, that is the code I read. It seems that DataflowPipelineJob's behavior is poor. I will
fix it there.

TestDataflowRunner's polling is not the same as DataflowPipelineJob's loop. It has different
termination criteria. They could be unified with some higher-order function hackery, but it
is not worth the readability hit for this case.

> TestDataflowRunner prints all messages from the job, repeatedly
> ---------------------------------------------------------------
>
>                 Key: BEAM-2120
>                 URL: https://issues.apache.org/jira/browse/BEAM-2120
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-dataflow
>            Reporter: Eugene Kirpichov
>            Assignee: Kenneth Knowles
>
> This is due to https://github.com/apache/beam/commit/ed8bd62652126d8d0cf054cee5cc79dda88e3415
> Every call to job.waitUntilFinish() prints all messages because the pagination by lastTimestamp
happens inside job.waitUntilFinish() rather than outside.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message