beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kenneth Knowles (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-2238) TestPipeline / PipelineOptionsFactory crashes on to unregistered options
Date Tue, 09 May 2017 21:49:04 GMT
Kenneth Knowles created BEAM-2238:
-------------------------------------

             Summary: TestPipeline / PipelineOptionsFactory crashes on to unregistered options
                 Key: BEAM-2238
                 URL: https://issues.apache.org/jira/browse/BEAM-2238
             Project: Beam
          Issue Type: Bug
          Components: sdk-java-core
            Reporter: Kenneth Knowles


When {{TestSparkPipelineOptions}} is not registered, this failure occurs on all tests using
{{TestPipeline}}:

{code}
java.lang.IllegalArgumentException: Class interface org.apache.beam.sdk.testing.TestPipelineOptions
missing a property named 'forceStreaming'.
	at org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1573)
	at org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:104)
	at org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:289)
	at org.apache.beam.sdk.testing.TestPipeline.testingPipelineOptions(TestPipeline.java:403)
	at org.apache.beam.sdk.testing.TestPipeline.create(TestPipeline.java:264)
	at org.apache.beam.runners.spark.translation.streaming.StreamingSourceMetricsTest.<init>(StreamingSourceMetricsTest.java:49)
{code}

However, it is (I believe) our recommended best practice to not register such options as they
are not intended for users. They should just be "cast" via {{as}} by the runner that wants
to use them. Further, {{TestSparkPipelineOptions}} cannot be registered due to pulling in
a hamcrest dependency (see BEAM-2237).

The user doesn't specify this option, and the runner doesn't mutate it. This should probably
not crash.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message