hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anu Engineer <aengin...@hortonworks.com>
Subject Re: Custom s3 endpoint for s3n/s3a
Date Tue, 05 Jan 2016 17:49:03 GMT
I have never tried it myself, so I have no idea if it works. But this JIRA (https://issues.apache.org/jira/browse/HADOOP-11261)
seems to indicate that you might want to setup “fs.s3a.endpoint”.


From: Han JU <ju.han.felix@gmail.com<mailto:ju.han.felix@gmail.com>>
Date: Tuesday, January 5, 2016 at 9:35 AM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Custom s3 endpoint for s3n/s3a


For test purpose we need to configure a custom s3 endpoint for s3n/s3a. More precisely we've
need to test that parquet writes correctly the content to s3.
We've setup a s3rver, so the endpoint should be `http://s3rver:8000`. I've tried different
method but no luck so far.

Things I've tried:
  - use url like `s3n://s3rver:8000/mybucket/file`
  - set jets3t.properties (also set them in Configuration)
  - tried a `AmazonS3Client` with `setEndpoint("s3rver:8000")`

Any suggestions?

JU Han

Software Engineer @ Teads.tv

+33 0619608888
View raw message