beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Cwik (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (BEAM-2238) TestPipeline / PipelineOptionsFactory crashes on to unregistered options
Date Tue, 09 May 2017 22:09:04 GMT

     [ https://issues.apache.org/jira/browse/BEAM-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Luke Cwik reassigned BEAM-2238:
-------------------------------

    Assignee: Luke Cwik

> TestPipeline / PipelineOptionsFactory crashes on to unregistered options
> ------------------------------------------------------------------------
>
>                 Key: BEAM-2238
>                 URL: https://issues.apache.org/jira/browse/BEAM-2238
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-core
>            Reporter: Kenneth Knowles
>            Assignee: Luke Cwik
>
> When {{TestSparkPipelineOptions}} is not registered, this failure occurs on all tests
using {{TestPipeline}}:
> {code}
> java.lang.IllegalArgumentException: Class interface org.apache.beam.sdk.testing.TestPipelineOptions
missing a property named 'forceStreaming'.
> 	at org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1573)
> 	at org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:104)
> 	at org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:289)
> 	at org.apache.beam.sdk.testing.TestPipeline.testingPipelineOptions(TestPipeline.java:403)
> 	at org.apache.beam.sdk.testing.TestPipeline.create(TestPipeline.java:264)
> 	at org.apache.beam.runners.spark.translation.streaming.StreamingSourceMetricsTest.<init>(StreamingSourceMetricsTest.java:49)
> {code}
> However, it is (I believe) our recommended best practice to not register such options
as they are not intended for users. They should just be "cast" via {{as}} by the runner that
wants to use them. Further, {{TestSparkPipelineOptions}} cannot be registered due to pulling
in a hamcrest dependency (see BEAM-2237).
> The user doesn't specify this option, and the runner doesn't mutate it. This should probably
not crash.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message