hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raimon Bosch <raimon.bo...@gmail.com>
Subject cannot use distcp in some s3 buckets
Date Thu, 13 Oct 2011 20:40:43 GMT
Hi,

I've been having some problems with one of our s3 buckets. I have asked on
amazon support with no luck yet
https://forums.aws.amazon.com/thread.jspa?threadID=78001.

I'm getting this exception only with our oldest s3 bucket with this command:
"hadoop distcp s3://<MY_BUCKET_NAME>/logfile-20110815.gz
/tmp/logfile-20110815.gz"

java.lang.IllegalArgumentException: Invalid hostname in URI
s3://<MY_BUCKET_NAME>/logfile-20110815.gz /tmp/logfile-20110815.gz
at org.apache.hadoop.fs.s3.S3Credentials.initialize(S3Credentials.java:41)
at
org.apache.hadoop.fs.s3.Jets3tFileSystemStore.initialize(Jets3tFileSystemStore.java:82)

As you can see, hadoop is rejecting my url before starting to do the
authorization steps. Someone has been in a similar issue? I have already
tested the same operation in newer s3 buckets and the command is working
correctly.

Thanks in advance,
Raimon Bosch.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message