spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "anxu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-17336) Repeated calls sbin/spark-config.sh file Causes ${PYTHONPATH} Value duplicate
Date Wed, 31 Aug 2016 13:12:20 GMT
anxu created SPARK-17336:
----------------------------

             Summary: Repeated calls sbin/spark-config.sh file Causes ${PYTHONPATH} Value
duplicate
                 Key: SPARK-17336
                 URL: https://issues.apache.org/jira/browse/SPARK-17336
             Project: Spark
          Issue Type: Bug
          Components: PySpark
    Affects Versions: 2.0.0
            Reporter: anxu


On Spark start up by command: sbin/start-all.sh, the sbin/spark-config.sh Repeated calls.
In sbin/spark-config.sh code.
{code:title=sbin/spark-config.sh|borderStyle=solid}
# Add the PySpark classes to the PYTHONPATH:
export PYTHONPATH="${SPARK_HOME}/python:${PYTHONPATH}"
export PYTHONPATH="${SPARK_HOME}/python/lib/py4j-0.10.3-src.zip:${PYTHONPATH}"
{code}
{color:red}PYTHONPATH{color} has duplicate Value.
example:
{code:borderStyle=solid}
axu4iMac:spark-2.0.0-hadoop2.4 axu$ sbin/start-all.sh  | grep PYTHONPATH
axu.print [Log] [6,16,31] [sbin/spark-config.sh] 定义PYTHONPATH
axu.print [sbin/spark-config.sh] [Define Global] PYTHONPATH(1): [/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python:]
axu.print [Log] [7,17,32] [sbin/spark-config.sh] 再次定义PYTHONPATH
axu.print [sbin/spark-config.sh] [Define Global] PYTHONPATH(2): [/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python/lib/py4j-0.10.1-src.zip:/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python:]
axu.print [Log] [6,16,31] [sbin/spark-config.sh] 定义PYTHONPATH
axu.print [sbin/spark-config.sh] [Define Global] PYTHONPATH(1): [/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python:/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python/lib/py4j-0.10.1-src.zip:/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python:]
axu.print [Log] [7,17,32] [sbin/spark-config.sh] 再次定义PYTHONPATH
axu.print [sbin/spark-config.sh] [Define Global] PYTHONPATH(2): [/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python/lib/py4j-0.10.1-src.zip:/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python:/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python/lib/py4j-0.10.1-src.zip:/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python:]
axu.print [Log] [6,16,31] [sbin/spark-config.sh] 定义PYTHONPATH
axu.print [sbin/spark-config.sh] [Define Global] PYTHONPATH(1): [/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python:/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python/lib/py4j-0.10.1-src.zip:/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python:/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python/lib/py4j-0.10.1-src.zip:/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python:]
axu.print [Log] [7,17,32] [sbin/spark-config.sh] 再次定义PYTHONPATH
axu.print [sbin/spark-config.sh] [Define Global] PYTHONPATH(2): [/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python/lib/py4j-0.10.1-src.zip:/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python:/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python/lib/py4j-0.10.1-src.zip:/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python:/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python/lib/py4j-0.10.1-src.zip:/Users/axu/code/axuProject/spark-2.0.0-hadoop2.4/python:]
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message