beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kenneth Knowles (JIRA)" <>
Subject [jira] [Commented] (BEAM-3159) DoFnTester should be deprecated in favor of TestPipeline
Date Thu, 25 Jan 2018 18:42:00 GMT


Kenneth Knowles commented on BEAM-3159:

I think TestPipeline + DirectRunner has all the capabilities to do this, doesn't it?

> DoFnTester should be deprecated in favor of TestPipeline
> --------------------------------------------------------
>                 Key: BEAM-3159
>                 URL:
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-core
>            Reporter: Ben Chambers
>            Assignee: Kenneth Knowles
>            Priority: Minor
> Reasons:
> 1. The logical unit within a Beam pipeline is a transform. Either a small transform like
a ParDo or a larger composite transform. Unit tests should focus on these units, rather than
probing specific behaviors of the user-defined functions.
> 2. The way that a runner interacts with a user-defined function is not necessarily obvious.
DoFnTester allows testing non-sensical cases that wouldn't arise in practice, since it allows
low-level interactions with the actual UDFs.
> Instead, we should encourage the use of TestPipeline with the direct runner. This allows
testing a single transform (such as a ParDo running a UDF) in context. It also makes it easier
to test things like side-inputs and multiple outputs, since you use the same techniques in
the test as you would in a real pipeline, rather than requiring a whole new API.

This message was sent by Atlassian JIRA

View raw message