spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wenchen Fan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-19982) JavaDatasetSuite.testJavaBeanEncoder sometimes fails with "Unable to generate an encoder for inner class"
Date Fri, 17 Mar 2017 01:29:41 GMT

    [ https://issues.apache.org/jira/browse/SPARK-19982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15929296#comment-15929296
] 

Wenchen Fan commented on SPARK-19982:
-------------------------------------

yea makes sense, the test harness should hold the {{this}} reference. I have no idea why this
can go wrong, maybe we should just move the test class to top-level so it's not an inner class
anymore.

> JavaDatasetSuite.testJavaBeanEncoder sometimes fails with "Unable to generate an encoder
for inner class"
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-19982
>                 URL: https://issues.apache.org/jira/browse/SPARK-19982
>             Project: Spark
>          Issue Type: Bug
>          Components: Tests
>    Affects Versions: 2.1.0
>            Reporter: Jose Soltren
>              Labels: flaky-test
>
> JavaDatasetSuite.testJavaBeanEncoder fails sporadically with the error below:
> Unable to generate an encoder for inner class `test.org.apache.spark.sql.JavaDatasetSuite$SimpleJavaBean`
without access to the scope that this class was defined in. Try moving this class out of its
parent class.
> From https://spark-tests.appspot.com/test-logs/35475788
> [~vanzin] looked into this back in October and reported:
> I ran this test in a loop (both alone and with the rest of the spark-sql tests) and never
got a failure. I even used the same JDK as Jenkins (1.7.0_51).
> Also looked at the code and nothing seems wrong. The errors is when an entry with the
parent class name is missing from the map kept in OuterScopes.scala, but the test populates
that map in its first line. So it doesn't look like a race nor some issue with weak references
(the map uses weak values).
>   public void testJavaBeanEncoder() {
>     OuterScopes.addOuterScope(this);



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message