beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amit Sela (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1039) Spark context is never actually re-used in tests
Date Tue, 22 Nov 2016 20:04:58 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15687734#comment-15687734
] 

Amit Sela commented on BEAM-1039:
---------------------------------

[~staslev] to the best of my knowledge {{Boolean#getBoolean(String)}} actually relates specifically
to System properties as described in the Java Docs: https://docs.oracle.com/javase/7/docs/api/java/lang/Boolean.html#getBoolean(java.lang.String).
Am I missing something here ?

> Spark context is never actually re-used in tests
> ------------------------------------------------
>
>                 Key: BEAM-1039
>                 URL: https://issues.apache.org/jira/browse/BEAM-1039
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-spark
>            Reporter: Stas Levin
>            Assignee: Stas Levin
>
> I think that due to a slight typo {{Boolean.getBoolean(TEST_REUSE_SPARK_CONTEXT)}} got
mixed up with
> {{Boolean.getBoolean(System.getProperty(TEST_REUSE_SPARK_CONTEXT))}} in the following
2 places, which may have caused {{SparkContext}} to behave not quite as expected.
> *  https://github.com/apache/incubator-beam/blob/master/runners/spark/src/main/java/org/apache/beam/runners/spark/translation/SparkContextFactory.java#L52
> *  https://github.com/apache/incubator-beam/blob/master/runners/spark/src/main/java/org/apache/beam/runners/spark/translation/SparkContextFactory.java#L70
> I'll take care of it as part of something I'm working on, so a PR should follow shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message