beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kenneth Knowles (JIRA)" <>
Subject [jira] [Created] (BEAM-2238) TestPipeline / PipelineOptionsFactory crashes on to unregistered options
Date Tue, 09 May 2017 21:49:04 GMT
Kenneth Knowles created BEAM-2238:

             Summary: TestPipeline / PipelineOptionsFactory crashes on to unregistered options
                 Key: BEAM-2238
             Project: Beam
          Issue Type: Bug
          Components: sdk-java-core
            Reporter: Kenneth Knowles

When {{TestSparkPipelineOptions}} is not registered, this failure occurs on all tests using

java.lang.IllegalArgumentException: Class interface org.apache.beam.sdk.testing.TestPipelineOptions
missing a property named 'forceStreaming'.
	at org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(
	at org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(
	at org.apache.beam.sdk.options.PipelineOptionsFactory$
	at org.apache.beam.sdk.testing.TestPipeline.testingPipelineOptions(
	at org.apache.beam.sdk.testing.TestPipeline.create(
	at org.apache.beam.runners.spark.translation.streaming.StreamingSourceMetricsTest.<init>(

However, it is (I believe) our recommended best practice to not register such options as they
are not intended for users. They should just be "cast" via {{as}} by the runner that wants
to use them. Further, {{TestSparkPipelineOptions}} cannot be registered due to pulling in
a hamcrest dependency (see BEAM-2237).

The user doesn't specify this option, and the runner doesn't mutate it. This should probably
not crash.

This message was sent by Atlassian JIRA

View raw message