Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-dev@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of momina.azam@gmail.com
 designates 209.85.220.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <55F9C7616E80514AA11400BF38C0657B16D27A3A@CH1PRD0310MB356.namprd03.prod.outlook.com>
References: 
 <CAHs+bThusjAQv1vHEgS3-ELW4DzpsUP8KT4iM2wtsyM6iMVOEw@mail.gmail.com>
	<CAHs+bTh+p+FO+rbo+O4MsKom8coMdzBb8Bs_7Yi5qBwFHmTTFA@mail.gmail.com>
	<CAHs+bTjMwZX=oq3-an3qMrXg9U63cfXWuFL4fXwMKTGErHBzQA@mail.gmail.com>
	<CAHs+bTjap8j6sAzScEU12VSEjj+7VbpAtBooGS4t1QgmO0L6Ng@mail.gmail.com>
	<55F9C7616E80514AA11400BF38C0657B16D27A3A@CH1PRD0310MB356.namprd03.prod.outlook.com>
Date: Fri, 6 Jul 2012 11:49:28 +0500
Message-ID: 
 <CAHs+bThJ8ivZoTSfHD2SmaUD_UZrtDu39gGH1bCTubuKPix3zQ@mail.gmail.com>
Subject: Re: HDFS to S3 copy issues
From: Momina Khan <momina.azam@gmail.com>
To: common-dev@hadoop.apache.org
Content-Type: multipart/alternative; boundary=bcaec5015d5d66c67404c423aa76

--bcaec5015d5d66c67404c423aa76
Content-Type: text/plain; charset=ISO-8859-1

hi Ivan,

i have tried with both ports 9000 and 9001 i get the same error dump ...

best
momina

On Fri, Jul 6, 2012 at 11:01 AM, Ivan Mitic <ivanmi@microsoft.com> wrote:

> Hi Momina,
>
> Could it be that you misspelled the port in your source path, you mind
> trying with: hdfs://10.240.113.162:9000/data/
>
> Ivan
>
> -----Original Message-----
> From: Momina Khan [mailto:momina.azam@gmail.com]
> Sent: Thursday, July 05, 2012 10:30 PM
> To: common-dev@hadoop.apache.org
> Subject: HDFS to S3 copy issues
>
> hi ... hope someone is able to help me out with this ... have tried an
> exhaustive search of google and AWS forum but there is little help in this
> regard and all that i found didnt work for me!
>
> i want to copy data from HDFS to my S3 bucket ... to test whether my HDFS
> url is correct i tried the fs -cat command which works just fine ... spits
> contents of the file ubuntu@domU-12-31-39-04-6E-58:/state/partition1/hadoop-1.0.1$
> *bin/hadoop fs -cat hdfs://10.240.113.162:9000/data/hello.txt*
>
> but when i try to distance copy the file from hdfs (same location as
> above) to my s3 bucket it says connection to server refused! have looked up
> Google exhaustively but cannot get an answer. they say that the port may be
> blocked but have checked that 9000-9001 are not blocked .... could it be an
> autghentication issue? just saying ... out of ideas.
>
> Find the call trace attached below:
>
> ubuntu@domU-12-31-39-04-6E-58:/state/partition1/hadoop-1.0.1$ *bin/hadoop
> distcp hdfs://10.240.113.162:9001/data/ s3://ID:**SECRET@momina
> *
>
> 12/07/05 12:48:37 INFO tools.DistCp: srcPaths=[hdfs://
> 10.240.113.162:9001/data]
> 12/07/05 12:48:37 INFO tools.DistCp: destPath=s3://ID:SECRET@momina
>
> 12/07/05 12:48:38 INFO ipc.Client: Retrying connect to server:
> domU-12-31-39-04-6E-58.compute-1.internal/10.240.113.162:9001. Already
> tried 0 time(s).
> 12/07/05 12:48:39 INFO ipc.Client: Retrying connect to server:
> domU-12-31-39-04-6E-58.compute-1.internal/10.240.113.162:9001. Already
> tried 1 time(s).
> 12/07/05 12:48:40 INFO ipc.Client: Retrying connect to server:
> domU-12-31-39-04-6E-58.compute-1.internal/10.240.113.162:9001. Already
> tried 2 time(s).
> 12/07/05 12:48:41 INFO ipc.Client: Retrying connect to server:
> domU-12-31-39-04-6E-58.compute-1.internal/10.240.113.162:9001. Already
> tried 3 time(s).
> 12/07/05 12:48:42 INFO ipc.Client: Retrying connect to server:
> domU-12-31-39-04-6E-58.compute-1.internal/10.240.113.162:9001. Already
> tried 4 time(s).
> 12/07/05 12:48:43 INFO ipc.Client: Retrying connect to server:
> domU-12-31-39-04-6E-58.compute-1.internal/10.240.113.162:9001. Already
> tried 5 time(s).
> 12/07/05 12:48:44 INFO ipc.Client: Retrying connect to server:
> domU-12-31-39-04-6E-58.compute-1.internal/10.240.113.162:9001. Already
> tried 6 time(s).
> 12/07/05 12:48:45 INFO ipc.Client: Retrying connect to server:
> domU-12-31-39-04-6E-58.compute-1.internal/10.240.113.162:9001. Already
> tried 7 time(s).
> 12/07/05 12:48:46 INFO ipc.Client: Retrying connect to server:
> domU-12-31-39-04-6E-58.compute-1.internal/10.240.113.162:9001. Already
> tried 8 time(s).
> 12/07/05 12:48:47 INFO ipc.Client: Retrying connect to server:
> domU-12-31-39-04-6E-58.compute-1.internal/10.240.113.162:9001. Already
> tried 9 time(s).
> With failures, global counters are inaccurate; consider running with -i
> Copy failed: java.net.ConnectException: Call to
> domU-12-31-39-04-6E-58.compute-1.internal/10.240.113.162:9001 failed on
> connection exception: java.net.ConnectException: Connection refused
>     at org.apache.hadoop.ipc.Client.wrapException(Client.java:1095)
>     at org.apache.hadoop.ipc.Client.call(Client.java:1071)
>     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>     at $Proxy1.getProtocolVersion(Unknown Source)
>     at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
>     at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
>     at
> org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:119)
>     at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:238)
>     at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:203)
>     at
>
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)
>     at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
>     at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
>     at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404)
>     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
>     at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)
>     at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:635)
>     at org.apache.hadoop.tools.DistCp.copy(DistCp.java:656)
>     at org.apache.hadoop.tools.DistCp.run(DistCp.java:881)
>     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>     at org.apache.hadoop.tools.DistCp.main(DistCp.java:908)
> Caused by: java.net.ConnectException: Connection refused
>     at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>     at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
>     at
>
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
>     at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
>     at
> org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:434)
>     at
> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:560)
>     at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:184)
>     at org.apache.hadoop.ipc.Client.getConnection(Client.java:1202)
>     at org.apache.hadoop.ipc.Client.call(Client.java:1046)
>     ... 19 more
>
> thank u!
> momina
>
>
>

--bcaec5015d5d66c67404c423aa76--