hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From slitz <slitzferr...@gmail.com>
Subject Re: Namenode Exceptions with S3
Date Wed, 09 Jul 2008 14:10:41 GMT
I'm having the exact same problem, any tip?

slitz

On Wed, Jul 2, 2008 at 12:34 AM, Lincoln Ritter <lincoln@lincolnritter.com>
wrote:

> Hello,
>
> I am trying to use S3 with Hadoop 0.17.0 on EC2.  Using this style of
> configuration:
>
> <property>
>  <name>fs.default.name</name>
>  <value>s3://$HDFS_BUCKET</value>
> </property>
>
> <property>
>  <name>fs.s3.awsAccessKeyId</name>
>  <value>$AWS_ACCESS_KEY_ID</value>
> </property>
>
> <property>
>  <name>fs.s3.awsSecretAccessKey</name>
>  <value>$AWS_SECRET_ACCESS_KEY</value>
> </property>
>
> on startup of the cluster with the bucket having no non-alphabetic
> characters, I get:
>
> 2008-07-01 16:10:49,171 ERROR org.apache.hadoop.dfs.NameNode:
> java.lang.RuntimeException: Not a host:port pair: XXXXX
>        at
> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:121)
>        at org.apache.hadoop.dfs.NameNode.initialize(NameNode.java:121)
>        at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:178)
>        at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:164)
>        at org.apache.hadoop.dfs.NameNode.createNameNode(NameNode.java:848)
>        at org.apache.hadoop.dfs.NameNode.main(NameNode.java:857)
>
> If I use this style of configuration:
>
> <property>
>  <name>fs.default.name</name>
>  <value>s3://$AWS_ACCESS_KEY:$AWS_SECRET_ACCESS_KEY@$HDFS_BUCKET</value>
> </property>
>
> I get (where the all-caps portions are the actual values...):
>
> 2008-07-01 19:05:17,540 ERROR org.apache.hadoop.dfs.NameNode:
> java.lang.NumberFormatException: For input string:
> "AWS_SECRET_ACCESS_KEY@HDFS_BUCKET"
>        at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
>        at java.lang.Integer.parseInt(Integer.java:447)
>        at java.lang.Integer.parseInt(Integer.java:497)
>        at
> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:128)
>        at org.apache.hadoop.dfs.NameNode.initialize(NameNode.java:121)
>        at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:178)
>        at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:164)
>        at org.apache.hadoop.dfs.NameNode.createNameNode(NameNode.java:848)
>        at org.apache.hadoop.dfs.NameNode.main(NameNode.java:857)
>
> These exceptions are taken from the namenode log.  The datanode logs
> show the same exceptions.
>
> Other than the above configuration changes, the configuration is
> identical to that generate by the hadoop image creation script found
> in the 0.17.0 distribution.
>
> Can anybody point me in the right direction here?
>
> -lincoln
>
> --
> lincolnritter.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message