spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Rosen (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-4890) Upgrade Boto to 2.34.0; automatically download Boto from PyPi instead of packaging it
Date Thu, 18 Dec 2014 21:42:13 GMT
Josh Rosen created SPARK-4890:
---------------------------------

             Summary: Upgrade Boto to 2.34.0; automatically download Boto from PyPi instead
of packaging it
                 Key: SPARK-4890
                 URL: https://issues.apache.org/jira/browse/SPARK-4890
             Project: Spark
          Issue Type: Improvement
            Reporter: Josh Rosen
            Assignee: Josh Rosen


We should upgrade to a newer version of Boto (2.34.0), since this is blocking several features.
 It looks like newer versions of Boto don't work properly when they're loaded from a zipfile
since they try to read a JSON file from a path relative to the Boto library sources.

Therefore, I think we should change {{spark-ec2}} to automatically download Boto from PyPi
if it's not present in {{SPARK_EC2_DIR/lib}}, similar to what we do in the {{sbt/sbt}} scripts.
 This shouldn't ben an issue for users since they already need to have an internet connection
to launch an EC2 cluster.  By performing the downloading in {{spark_ec2.py}} instead of the
Bash script, this should also work for Windows users.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message