hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phillips, Caleb" <Caleb.Phill...@nrel.gov>
Subject Re: fs.s3a.endpoint not working
Date Mon, 11 Jan 2016 15:39:29 GMT
Hi All,

Just wanted to send this out again since there was no response
(admittedly, originally sent in the midst of the US holiday season) and it
seems to be an issue that continues to come up (see e.g., the email from
Han Ju on Jan 5).

If anyone has successfully connected Hadoop to a non-AWS S3-compatable
object store, it’d be very helpful to hear how you made it work. The
fs.s3a.endpoint configuration directive appears non-functional at our site
(with Hadoop 2.6.3).

--
Caleb Phillips, Ph.D.
Data Scientist | Computational Science Center

National Renewable Energy Laboratory (NREL)
15013 Denver West Parkway | Golden, CO 80401
303-275-4297 | caleb.phillips@nrel.gov






On 12/22/15, 1:39 PM, "Phillips, Caleb" <Caleb.Phillips@nrel.gov> wrote:

>Hi All,
>
>New to this list. Looking for a bit of help:
>
>I'm having trouble connecting Hadoop to a S3-compatable (non AWS) object
>store.
>
>This issue was discussed, but left unresolved, in this thread:
>
>https://mail-archives.apache.org/mod_mbox/spark-user/201507.mbox/%3CCA+0W_
>Au5Es_fLUgZMGwkkgA3JyA1ASi3u+isJCuYmfnTvNkGuQ@mail.gmail.com%3E
>
>And here, on Cloudera's forums (the second post is mine):
>
>https://community.cloudera.com/t5/Data-Ingestion-Integration/fs-s3a-endpoi
>nt-ignored-in-hdfs-site-xml/m-p/33694#M1180
>
>I'm running Hadoop 2.6.3 with Java 1.8 (65) on a Linux host. Using
>Hadoop, I'm able to connect to S3 on AWS, and e.g., list/put/get files.
>
>However, when I point the fs.s3a.endpoint configuration directive at my
>non-AWS S3-Compatable object storage, it appears to still point at (and
>authenticate against) AWS.
>
>I've checked and double-checked my credentials and configuration using
>both Python's boto library and the s3cmd tool, both of which connect to
>this non-AWS data store just fine.
>
>Any help would be much appreciated. Thanks!
>
>--
>Caleb Phillips, Ph.D.
>Data Scientist | Computational Science Center
>
>National Renewable Energy Laboratory (NREL)
>15013 Denver West Parkway | Golden, CO 80401
>303-275-4297 | caleb.phillips@nrel.gov
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
>For additional commands, e-mail: user-help@hadoop.apache.org
>

Mime
View raw message