Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of harsh@cloudera.com designates
 209.85.223.181 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <0b1d01cedb46$bb1d8230$31588690$@mlab.cs.msu.su>
References: <0b1d01cedb46$bb1d8230$31588690$@mlab.cs.msu.su>
From: Harsh J <harsh@cloudera.com>
Date: Thu, 7 Nov 2013 05:18:42 +0530
Message-ID: 
 <CAOcnVr1D3-xDkmm8m=yqUVj9uLjOGDmAFk1D2mLZ0Uukr8xaqw@mail.gmail.com>
Subject: Re: access to hadoop cluster to post tasks remotely
To: "<user@hadoop.apache.org>" <user@hadoop.apache.org>,
 gerasimov@mlab.cs.msu.su
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Data in HDFS is read and written via the individual DN's 50010 ports,
which you would also need to open up to avoid these errors. Data isn't
written/read through the NameNode.

On Thu, Nov 7, 2013 at 4:50 AM, Sergey Gerasimov
<gerasimov@mlab.cs.msu.su> wrote:
> Hello,
>
>
>
> I have problems with posting jar to my cluster remotely from client machi=
ne
> located somewhere in the Web. I use original hadoop-1.2.1.
>
>
>
> I installed hadoop  on client machine (same version as in the cluster),
> configured fs.default.name and mapred.job.tracker.
>
> Access to DFS works fine remotely. I can successfully play with =93hadoop=
 fs=94
> commands.
>
>
>
> But when I send some job, for example:
>
> hadoop jar hadoop-examples -1.2.1.jar sleep 1
>
>
>
> I see output like:
>
> 13/11/07 02:44:42 INFO hdfs.DFSClient: Exception in createBlockOutputStre=
am
> xx.xx.xx.xx:50010 java.net.ConnectException: Connection timed out
>
> 13/11/07 02:44:42 INFO hdfs.DFSClient: Abandoning
> blk_1089181243677159149_31717
>
> 13/11/07 02:44:42 INFO hdfs.DFSClient: Excluding datanode xx.xx.xx.xx:500=
10
>
> 13/11/07 02:45:45 INFO hdfs.DFSClient: Exception in createBlockOutputStre=
am
> xx.xx.xx.xx:50010 java.net.ConnectException: Connection timed out
>
> 13/11/07 02:45:45 INFO hdfs.DFSClient: Abandoning
> blk_6550586867464091073_31717
>
> 13/11/07 02:45:45 INFO hdfs.DFSClient: Excluding datanode xx.xx.xx.xx:500=
10
>
> 13/11/07 02:46:48 INFO hdfs.DFSClient: Exception in createBlockOutputStre=
am
> xx.xx.xx.xx:50010 java.net.ConnectException: Connection timed out
>
> 13/11/07 02:46:48 INFO hdfs.DFSClient: Abandoning
> blk_5814098597599107248_31717
>
> 13/11/07 02:46:48 INFO hdfs.DFSClient: Excluding datanode xx.xx.xx.xx:500=
10
>
> 13/11/07 02:47:51 INFO hdfs.DFSClient: Exception in createBlockOutputStre=
am
> xx.xx.xx.xx:50010 java.net.ConnectException: Connection timed out
>
> 13/11/07 02:47:51 INFO hdfs.DFSClient: Abandoning
> blk_6368219524592897749_31717
>
>
>
> The same jar sent from inside the cluster runs fine.
>
>
>
> The network where cluster lives  is protected by firewall with only NameN=
ode
> and JobTracker ports opened externally.
>
> iptables on all nodes are off.
>
>
>
> I have no ideas about reasons of these messages in the log. To the moment=
 I
> were sure that entry point to hadoop cluster  contains just NameNode and
> JobTracker ports.
>
> Both are open.
>
>
>
> Please help!
>
>
>
>
>
>


--=20
Harsh J