Return-Path: 
 <user-return-1711-apmail-hadoop-hdfs-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id E9B80D760
	for <apmail-hadoop-hdfs-user-archive@minotaur.apache.org>;
 Sat, 29 Sep 2012 15:37:40 +0000 (UTC)
Received: (qmail 43797 invoked by uid 500); 29 Sep 2012 15:37:36 -0000
Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org
Received: (qmail 43479 invoked by uid 500); 29 Sep 2012 15:37:36 -0000
Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:user@hadoop.apache.org>
List-Id: <user.hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Delivered-To: mailing list user@hadoop.apache.org
Received: (qmail 43470 invoked by uid 99); 29 Sep 2012 15:37:36 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 29 Sep 2012 15:37:36 +0000
X-ASF-Spam-Status: No, hits=-0.0 required=5.0
	tests=FSL_RCVD_USER,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL
X-Spam-Check-By: apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Received: from [74.125.82.176] (HELO mail-we0-f176.google.com) (74.125.82.176)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 29 Sep 2012 15:37:28 +0000
Received: by weyz53 with SMTP id z53so2399294wey.35
        for <user@hadoop.apache.org>; Sat, 29 Sep 2012 08:37:07 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20120113;
        h=message-id:date:from:user-agent:mime-version:to:subject
         :content-type:content-transfer-encoding:x-gm-message-state;
        bh=Dl2bBwnTGlPKOQ+u3Wv61VV8nPBzXAylETXkUA9RtO4=;
        b=d0DkEyiY0hKSCife0+YMZMWT1qUUwO9wowIvAPedavjeVLT6stvpGlJHUEczZkyX2I
         03y0MOjU4xUTDwnNtfZAdYlI48LNTbJUVVhrFZn9crJgspMCWKboro4aTv6fQr1YxFIH
         bu7QAVvnXvN0Rz20qr1YcbvT4OPizULSkVrvx2YlP3WhAZHnPCbkZzjYUUVWXVY013q8
         hbpyHByRRGNwxFytKY6v0U4HWxXYlKDBsZyKv38qcxml+bM9BTmeE5P5CaDdvLcpGTpN
         rsE/lBfPoe8L8JT+fo8CczUp3MaKW2LjbjCHHtGfVb3ltUMPMAC4FMOPijJew8JfVmmJ
         Vmfw==
Received: by 10.216.119.6 with SMTP id m6mr4785143weh.215.1348933027561;
        Sat, 29 Sep 2012 08:37:07 -0700 (PDT)
Received: from [192.168.1.27] (17.165.216.87.static.jazztel.es.
 [87.216.165.17])
        by mx.google.com with ESMTPS id cu1sm5648263wib.6.2012.09.29.08.37.06
        (version=TLSv1/SSLv3 cipher=OTHER);
        Sat, 29 Sep 2012 08:37:07 -0700 (PDT)
Message-ID: <5067159D.5040604@planetalia.com>
Date: Sat, 29 Sep 2012 17:37:01 +0200
From: Alexander Hristov <alexh@planetalia.com>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
 rv:15.0) Gecko/20120907 Thunderbird/15.0.1
MIME-Version: 1.0
To: user@hadoop.apache.org
Subject: Hadoop 0.23.3 and Amazon S3
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-Gm-Message-State: 
 ALoCoQlWVy0PqCzQzpuYAYIPdUfG6Rmbj8cPi/Gz32eqGHpWhSqjVYrIITYrMpv1p/EIoDYSqjgJ

Hi Again

I have problems trying to make Hadoop use S3 or S3N as filesystem.

This is what I have in core-site.xml:

<configuration>
     <property>
       <name>fs.default.name</name>
       <value>s3n://bucketname</value>
     </property>

     <property>
       <name>fs.s3.awsAccessKeyId</name>
       <value> something <value>
     </property>

     <property>
       <name>fs.s3.awsSecretAccessKey</name>
       <value> something </value>
     </property>

     <property>
       <name>fs.s3n.awsAccessKeyId</name>
       <value> something </value>
     </property>

     <property>
       <name>fs.s3n.awsSecretAccessKey</name>
       <value> something </value>
     </property>

      <property>
         <name>hadoop.tmp.dir</name>
         <value>/tmp/hadoop</value>
      </property>
</configuration>

The secret key does not contain any slashes.

When I use s3n://buckename, I get this:

[hadoop@ahristov hadoop]$ hadoop fs -put LICENSE.txt /
put: org.jets3t.service.S3ServiceException: S3 HEAD request failed for 
'/LICENSE.txt' - ResponseCode=403, ResponseMessage=Forbidden

And when I use s3://bucketname, I get this:

[hadoop@ahristov hadoop]$ hadoop fs -put LICENSE.txt /
put: `/': No such file or directory

I couldn't find any logs generated anywhere.

On the other hand, if I use a quick and dirty Java snippet to achieve 
the same, like:

         Configuration conf = new Configuration();
conf.addResource(TestS3.class.getResourceAsStream("/res/core-s3.xml"));
         FileSystem fileSystem = FileSystem.get(conf);
         InputStream in = TestS3.class.getResourceAsStream("/res/test.txt");
         FSDataOutputStream out = fileSystem.create(new Path("/book.txt"));
         byte[] buffer = new byte[10240];
         while (true) {
             int read= in.read(buffer);
             if (read== -1) break;
             out.write(buffer,0,read);
         }
         out.close();
         in.close();

It works both with s3:// and s3n://

Regards

Alexander