spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "thauvin damien (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-15965) No FileSystem for scheme: s3n or s3a spark-2.0.0 and spark-1.6.1
Date Wed, 15 Jun 2016 15:30:10 GMT
thauvin damien created SPARK-15965:
--------------------------------------

             Summary: No FileSystem for scheme: s3n or s3a  spark-2.0.0 and spark-1.6.1
                 Key: SPARK-15965
                 URL: https://issues.apache.org/jira/browse/SPARK-15965
             Project: Spark
          Issue Type: Bug
          Components: Build
    Affects Versions: 1.6.1
         Environment: Debian GNU/Linux 8
java version "1.7.0_79"
            Reporter: thauvin damien


The spark programming-guide explain that Spark can create distributed datasets on Amazon S3
. 
But since the pre-buid "Hadoop 2.6" the S3 access doesn't work with s3n or s3a.
Any version of spark : spark-1.3.1 ; spark-1.6.1 even spark-2.0.0 with hadoop.7.2 .
I understand this is an Hadoop Issue (SPARK-7442)  but can you make some documentation to
explain what jar we need to add and where ? ( for standalone installation) .
"hadoop-aws-x.x.x.jar and aws-java-sdk-x.x.x.jar is enough ? 
What env variable we need to set and what file we need to modifiy .
Is it "$CLASSPATH "or a variable in "spark-defaults.conf" with variable "spark.driver.extraClassPath"
and "spark.executor.extraClassPath"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message