spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Juliusz Sompolski (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-22938) Assert that SQLConf.get is accessed only on the driver.
Date Tue, 02 Jan 2018 14:26:01 GMT
Juliusz Sompolski created SPARK-22938:
-----------------------------------------

             Summary: Assert that SQLConf.get is accessed only on the driver.
                 Key: SPARK-22938
                 URL: https://issues.apache.org/jira/browse/SPARK-22938
             Project: Spark
          Issue Type: Test
          Components: SQL
    Affects Versions: 2.2.1
            Reporter: Juliusz Sompolski


Assert if code tries to access SQLConf.get on executor.
This can lead to hard to detect bugs, where the executor will read fallbackConf, falling back
to default config values, ignoring potentially changed non-default configs.
If a config is to be passed to executor code, it needs to be read on the driver, and passed
explicitly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message