Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DE4B3C679 for ; Fri, 10 Aug 2012 17:11:30 +0000 (UTC) Received: (qmail 3013 invoked by uid 500); 10 Aug 2012 17:11:25 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 2879 invoked by uid 500); 10 Aug 2012 17:11:25 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 2872 invoked by uid 99); 10 Aug 2012 17:11:25 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Aug 2012 17:11:25 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of jian.fang.subscribe@gmail.com designates 209.85.160.48 as permitted sender) Received: from [209.85.160.48] (HELO mail-pb0-f48.google.com) (209.85.160.48) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Aug 2012 17:11:20 +0000 Received: by pbbrp16 with SMTP id rp16so2425242pbb.35 for ; Fri, 10 Aug 2012 10:11:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=mTsdhsSMcMuU/IC+lTE0Q6j14GLTmqufWNSSitevLTc=; b=I6Kj40AEa37m9CqLyq+mU9k/e+kmvaqcMlASumd1xpS4sckpU2DXeYDLc5g8dvGhqm giJm/q0sMFka1m+v2E2LlHMdJ+xgiNmfussOIAvsgSzZTxd1c7sWvVWQqCWfy+xCgJKl 2ncOijZouXPqwrpNVg7rA75cxE6P1AXupZJkW1C0ucDom8uy1Q84D1jF7RCpipi+SI0f qNM41EZGLoP15ff0PG53pb7p9kFLOv2GJi+xJgQ7SgjBA5s8wnQs7CaiCNwgqSXOuz0S z0mrKdQK0atJcl4dq67cLJNG1cei6bs1ZCZo5BbnMT1iOvpizz7i9R0l1KwMpIFBCCRW +MlQ== MIME-Version: 1.0 Received: by 10.68.228.231 with SMTP id sl7mr408673pbc.45.1344618660055; Fri, 10 Aug 2012 10:11:00 -0700 (PDT) Received: by 10.68.44.134 with HTTP; Fri, 10 Aug 2012 10:10:59 -0700 (PDT) In-Reply-To: References: Date: Fri, 10 Aug 2012 13:10:59 -0400 Message-ID: Subject: Re: hftp in Hadoop 0.20.2 From: Jian Fang To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=047d7b2e0a0b974d3604c6ec6d37 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b2e0a0b974d3604c6ec6d37 Content-Type: text/plain; charset=ISO-8859-1 Here is the property in hdfs-site.xml dfs.http.address pnjhadoopnn01:50070 Thanks, Jian On Fri, Aug 10, 2012 at 11:46 AM, Harsh J wrote: > Yes the test was to figure out if there really was a listener on > 50070. Can you check the hdfs-site.xml on the NN machine for what its > dfs.http.address may really be using for its port? > > On Fri, Aug 10, 2012 at 7:48 PM, Jian Fang > wrote: > > Hi Harsh, > > > > Seems the -p requires the root privilege, which I don't have. I run > > "netstat -a | grep 50070", but did not get back anything. As I said, > telnet > > did not work either. > > > > [hadoop@pnjhadoopnn01 ~]$ telnet pnjhadoopnn01 50070 > > Trying xx.xx.xx.xx... > > telnet: connect to address xx.xx.xx.xx: Connection refused > > telnet: Unable to connect to remote host: Connection refused > > > > [hadoop@pnjhadoopnn01 ~]$ telnet localhost 50070 > > Trying 127.0.0.1... > > telnet: connect to address 127.0.0.1: Connection refused > > telnet: Unable to connect to remote host: Connection refused > > > > Thanks, > > > > Jian > > > > On Fri, Aug 10, 2012 at 1:50 AM, Harsh J wrote: > >> > >> Jian, > >> > >> From your NN, can you get us the output "netstat -anp | grep 50070"? > >> > >> On Fri, Aug 10, 2012 at 9:29 AM, Jian Fang > >> wrote: > >> > Thanks Harsh. But there is no firewall there, the two clusters are on > >> > the > >> > same networks. I cannot telnet to the port even on the same machine. > >> > > >> > > >> > On Thu, Aug 9, 2012 at 6:00 PM, Harsh J wrote: > >> >> > >> >> Hi Jian, > >> >> > >> >> HFTP is always-on by default. Can you check and make sure that the > >> >> firewall isn't the cause of the connection refused on port 50070 on > >> >> the NN and ports 50075 on the DNs here? > >> >> > >> >> On Fri, Aug 10, 2012 at 1:47 AM, Jian Fang > >> >> wrote: > >> >> > Hi, > >> >> > > >> >> > We have a hadoop cluster of version 0.20.2 in production. Now we > have > >> >> > another new Hadoop cluster using cloudera's CDH3U4. We like to run > >> >> > distcp to > >> >> > copy files between the two clusters. Since the hadoop versions are > >> >> > different, we have to use hftp protocol to copy files based on the > >> >> > hadoop > >> >> > document here: > >> >> > http://hadoop.apache.org/common/docs/r0.20.2/distcp.html#cpver. > >> >> > > >> >> > The problem is that I cannot access files via hftp from the current > >> >> > production 0.20.2 cluster even though I can see the following > setting > >> >> > from > >> >> > job tracker UI. > >> >> > > >> >> > dfs.http.address pnjhadoopnn01:50070 > >> >> > > >> >> > I tried to telnet this port, but got "connection refused" error. > >> >> > Seems > >> >> > the > >> >> > hftp service is not actually running. Could someone tell me how to > >> >> > enable > >> >> > the hftp service in the 0.20.2 hadoop cluster so that I can run > >> >> > distcp? > >> >> > > >> >> > Thanks in advance, > >> >> > > >> >> > John > >> >> > >> >> > >> >> > >> >> -- > >> >> Harsh J > >> > > >> > > >> > >> > >> > >> -- > >> Harsh J > > > > > > > > -- > Harsh J > --047d7b2e0a0b974d3604c6ec6d37 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Here is the property in hdfs-site.xml

=A0=A0 <property>
=A0= =A0=A0=A0=A0 <name>dfs.http.address</name>
=A0=A0=A0=A0=A0 &= lt;value>pnjhadoopnn01:50070</value>
=A0=A0 </property>
Thanks,

Jian

On Fri, Aug 10, 2012 at 11:46 AM, Harsh J <harsh@cloudera.com> wrote:
Yes the test was to figure out if there really was a listener on
50070. Can you check the hdfs-site.xml on the NN machine for what its
dfs.http.address may really be using for its port?

On Fri, Aug 10, 2012 at 7:48 PM, Jian Fang
<jian.fang.subscribe@gm= ail.com> wrote:
> Hi Harsh,
>
> Seems the -p requires the root privilege, which I don't have. I ru= n
> "netstat -a | grep 50070", but did not get back anything. As= I said, telnet
> did not work either.
>
> [hadoop@pnjhadoopnn01 ~]$ telnet =A0pnjhadoopnn01 50070
> Trying xx.xx.xx.xx...
> telnet: connect to address xx.xx.xx.xx: Connection refused
> telnet: Unable to connect to remote host: Connection refused
>
> [hadoop@pnjhadoopnn01 ~]$ telnet localhost 50070
> Trying 127.0.0.1...
> telnet: connect to address 127.0.0.1: Connection refused
> telnet: Unable to connect to remote host: Connection refused
>
> Thanks,
>
> Jian
>
> On Fri, Aug 10, 2012 at 1:50 AM, Harsh J <harsh@cloudera.com> wrote:
>>
>> Jian,
>>
>> From your NN, can you get us the output "netstat -anp | grep = 50070"?
>>
>> On Fri, Aug 10, 2012 at 9:29 AM, Jian Fang
>> <jian.fang.sub= scribe@gmail.com> wrote:
>> > Thanks Harsh. But there is no firewall there, the two cluster= s are on
>> > the
>> > same networks. I cannot telnet to the port even on the same m= achine.
>> >
>> >
>> > On Thu, Aug 9, 2012 at 6:00 PM, Harsh J <harsh@cloudera.com> wrote:
>> >>
>> >> Hi Jian,
>> >>
>> >> HFTP is always-on by default. Can you check and make sure= that the
>> >> firewall isn't the cause of the connection refused on= port 50070 on
>> >> the NN and ports 50075 on the DNs here?
>> >>
>> >> On Fri, Aug 10, 2012 at 1:47 AM, Jian Fang
>> >> <jian= .fang.subscribe@gmail.com> wrote:
>> >> > Hi,
>> >> >
>> >> > We have a hadoop cluster of version 0.20.2 in produc= tion. Now we have
>> >> > another new Hadoop cluster using cloudera's CDH3= U4. We like to run
>> >> > distcp to
>> >> > copy files between the two clusters. Since the hadoo= p versions are
>> >> > different, we have to use hftp protocol to copy file= s based on the
>> >> > hadoop
>> >> > document here:
>> >> > http://hadoop.apache.org/common/do= cs/r0.20.2/distcp.html#cpver.
>> >> >
>> >> > The problem is that I cannot access files via hftp f= rom the current
>> >> > production 0.20.2 cluster even though I can see the = following setting
>> >> > from
>> >> > job tracker UI.
>> >> >
>> >> > dfs.http.address pnjhadoopnn01:50070
>> >> >
>> >> > I tried to telnet this port, but got "connectio= n refused" error.
>> >> > Seems
>> >> > the
>> >> > hftp service is not actually running. Could someone = tell me how to
>> >> > enable
>> >> > the hftp service in the 0.20.2 hadoop cluster so tha= t I can run
>> >> > distcp?
>> >> >
>> >> > Thanks in advance,
>> >> >
>> >> > John
>> >>
>> >>
>> >>
>> >> --
>> >> Harsh J
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>
>



--
Harsh J

--047d7b2e0a0b974d3604c6ec6d37--