beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xu Mingmin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-2833) TestPipeline fail to get Coder
Date Fri, 01 Sep 2017 21:27:00 GMT

    [ https://issues.apache.org/jira/browse/BEAM-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151171#comment-16151171
] 

Xu Mingmin commented on BEAM-2833:
----------------------------------

Kenn:
Robert's bisect also indicated that the issue lies somewhere within the toProto/fromProto
translations. The lines you mention are just the flag flip to enable those, which represent
a lot of functionality. To get a root cause, I'd check in a debugger and/or write validation
routine for the raw proto.

Orthogonal to finding and addressing the root cause, since there are also some performance
issues that may or may not be worth solving for 2.2.0 I have suggested putting this proto
round-trip (and other related bits, until we achieve acceptable performance) behind an PipelineOption
as an experiment.

> TestPipeline fail to get Coder
> ------------------------------
>
>                 Key: BEAM-2833
>                 URL: https://issues.apache.org/jira/browse/BEAM-2833
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-core, testing
>            Reporter: Xu Mingmin
>             Fix For: 2.2.0
>
>
> See this error when preparing merge branch DSL_SQL to master, it seems caused by https://github.com/apache/beam/pull/3334,
and only happen with {{TestPipelin}} but work fine with {{org.apache.beam.sdk.Pipeline}}.

> More error message can be found in https://builds.apache.org/job/beam_PreCommit_Java_MavenInstall/14092/#showFailuresLink.

> {code}
> Error Message
> java.lang.IllegalArgumentException
> Stacktrace
> org.apache.beam.runners.direct.repackaged.runners.core.construction.java.repackaged.com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.IllegalArgumentException
> 	at org.apache.beam.runners.direct.repackaged.runners.core.construction.java.repackaged.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2214)
> 	at org.apache.beam.runners.direct.repackaged.runners.core.construction.java.repackaged.com.google.common.cache.LocalCache.get(LocalCache.java:4053)
> 	at org.apache.beam.runners.direct.repackaged.runners.core.construction.java.repackaged.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4057)
> 	at org.apache.beam.runners.direct.repackaged.runners.core.construction.java.repackaged.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4986)
> 	at org.apache.beam.runners.direct.repackaged.runners.core.construction.RehydratedComponents.getCoder(RehydratedComponents.java:154)
> 	at org.apache.beam.runners.direct.repackaged.runners.core.construction.TestStreamTranslation.fromProto(TestStreamTranslation.java:68)
> 	at org.apache.beam.runners.direct.repackaged.runners.core.construction.TestStreamTranslation.getTestStream(TestStreamTranslation.java:101)
> 	at org.apache.beam.runners.direct.TestStreamEvaluatorFactory$DirectTestStreamFactory.getReplacementTransform(TestStreamEvaluatorFactory.java:179)
> 	at org.apache.beam.sdk.Pipeline.applyReplacement(Pipeline.java:540)
> 	at org.apache.beam.sdk.Pipeline.replace(Pipeline.java:280)
> 	at org.apache.beam.sdk.Pipeline.replaceAll(Pipeline.java:201)
> 	at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:169)
> 	at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:62)
> 	at org.apache.beam.sdk.Pipeline.run(Pipeline.java:303)
> 	at org.apache.beam.sdk.testing.TestPipeline.run(TestPipeline.java:344)
> 	at org.apache.beam.sdk.testing.TestPipeline.run(TestPipeline.java:331)
> 	at org.apache.beam.sdk.extensions.sql.BeamSqlDslAggregationTest.runSessionWindow(BeamSqlDslAggregationTest.java:367)
> 	at org.apache.beam.sdk.extensions.sql.BeamSqlDslAggregationTest.testSessionWindowWithBounded(BeamSqlDslAggregationTest.java:338)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 	at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:239)
> 	at org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:320)
> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> 	at org.junit.runners.Suite.runChild(Suite.java:128)
> 	at org.junit.runners.Suite.runChild(Suite.java:27)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> 	at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
> 	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
> 	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
> 	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
> 	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
> 	at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:157)
> 	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
> 	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
> 	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)
> Caused by: java.lang.IllegalArgumentException
> 	at org.apache.beam.runners.direct.repackaged.sdk.common.runner.v1.RunnerApi$Components.getCodersOrThrow(RunnerApi.java:1695)
> 	at org.apache.beam.runners.direct.repackaged.runners.core.construction.RehydratedComponents$2.load(RehydratedComponents.java:75)
> 	at org.apache.beam.runners.direct.repackaged.runners.core.construction.RehydratedComponents$2.load(RehydratedComponents.java:71)
> 	at org.apache.beam.runners.direct.repackaged.runners.core.construction.java.repackaged.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3628)
> 	at org.apache.beam.runners.direct.repackaged.runners.core.construction.java.repackaged.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2336)
> 	at org.apache.beam.runners.direct.repackaged.runners.core.construction.java.repackaged.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2295)
> 	at org.apache.beam.runners.direct.repackaged.runners.core.construction.java.repackaged.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2208)
> 	... 56 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message