Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 424DDD9F6 for ; Fri, 10 Aug 2012 14:18:43 +0000 (UTC) Received: (qmail 33452 invoked by uid 500); 10 Aug 2012 14:18:38 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 33321 invoked by uid 500); 10 Aug 2012 14:18:38 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 33314 invoked by uid 99); 10 Aug 2012 14:18:38 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Aug 2012 14:18:38 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of jian.fang.subscribe@gmail.com designates 209.85.161.176 as permitted sender) Received: from [209.85.161.176] (HELO mail-gg0-f176.google.com) (209.85.161.176) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Aug 2012 14:18:31 +0000 Received: by ggke5 with SMTP id e5so588355ggk.35 for ; Fri, 10 Aug 2012 07:18:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=ydt4a0pW9ngkO79drSv0Y3a2XyZNdb4/AZhWN2mObbI=; b=OaPLjvSPKMFSPLWkWBx8bIjjeVYZeDYf05bLSQ1S13kfmPrstF96yi/oFBH3JiJVpG JBmdgY0jsWDu/f/G78UfMrTdmZVRNSWdTJKVTH0Hob4PmpBf3kWFUDG1Iv0YgQluyAI2 Ss3tsALqJhZup9A3YeV8gssirTDVkjJowpHj/QBN/XjY5/mkicLok9kTh6VEJQHSUmFD lBR0LMylRKbygvn/gk+Y/KATTZDXApBB9kp9pJ+oMQpQeSatkEphFp1qr3W7woLzYJzo +tBCdMO1QlFji+/Oq6sIG3r5NXCzcAKQsDp13wy1inO2/7cwIkA5E5zbiHRf/rQy/b2J 6Iew== MIME-Version: 1.0 Received: by 10.66.79.195 with SMTP id l3mr6624371pax.33.1344608289975; Fri, 10 Aug 2012 07:18:09 -0700 (PDT) Received: by 10.68.44.134 with HTTP; Fri, 10 Aug 2012 07:18:09 -0700 (PDT) In-Reply-To: References: Date: Fri, 10 Aug 2012 10:18:09 -0400 Message-ID: Subject: Re: hftp in Hadoop 0.20.2 From: Jian Fang To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=f46d04287a177c6d5004c6ea03f6 --f46d04287a177c6d5004c6ea03f6 Content-Type: text/plain; charset=ISO-8859-1 Hi Harsh, Seems the -p requires the root privilege, which I don't have. I run "netstat -a | grep 50070", but did not get back anything. As I said, telnet did not work either. [hadoop@pnjhadoopnn01 ~]$ telnet pnjhadoopnn01 50070 Trying xx.xx.xx.xx... telnet: connect to address xx.xx.xx.xx: Connection refused telnet: Unable to connect to remote host: Connection refused [hadoop@pnjhadoopnn01 ~]$ telnet localhost 50070 Trying 127.0.0.1... telnet: connect to address 127.0.0.1: Connection refused telnet: Unable to connect to remote host: Connection refused Thanks, Jian On Fri, Aug 10, 2012 at 1:50 AM, Harsh J wrote: > Jian, > > From your NN, can you get us the output "netstat -anp | grep 50070"? > > On Fri, Aug 10, 2012 at 9:29 AM, Jian Fang > wrote: > > Thanks Harsh. But there is no firewall there, the two clusters are on the > > same networks. I cannot telnet to the port even on the same machine. > > > > > > On Thu, Aug 9, 2012 at 6:00 PM, Harsh J wrote: > >> > >> Hi Jian, > >> > >> HFTP is always-on by default. Can you check and make sure that the > >> firewall isn't the cause of the connection refused on port 50070 on > >> the NN and ports 50075 on the DNs here? > >> > >> On Fri, Aug 10, 2012 at 1:47 AM, Jian Fang > >> wrote: > >> > Hi, > >> > > >> > We have a hadoop cluster of version 0.20.2 in production. Now we have > >> > another new Hadoop cluster using cloudera's CDH3U4. We like to run > >> > distcp to > >> > copy files between the two clusters. Since the hadoop versions are > >> > different, we have to use hftp protocol to copy files based on the > >> > hadoop > >> > document here: > >> > http://hadoop.apache.org/common/docs/r0.20.2/distcp.html#cpver. > >> > > >> > The problem is that I cannot access files via hftp from the current > >> > production 0.20.2 cluster even though I can see the following setting > >> > from > >> > job tracker UI. > >> > > >> > dfs.http.address pnjhadoopnn01:50070 > >> > > >> > I tried to telnet this port, but got "connection refused" error. Seems > >> > the > >> > hftp service is not actually running. Could someone tell me how to > >> > enable > >> > the hftp service in the 0.20.2 hadoop cluster so that I can run > distcp? > >> > > >> > Thanks in advance, > >> > > >> > John > >> > >> > >> > >> -- > >> Harsh J > > > > > > > > -- > Harsh J > --f46d04287a177c6d5004c6ea03f6 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi Harsh,

Seems the -p requires the root privilege, which I don'= t have. I run
"netstat -a | grep 50070", but did not get back= anything. As I said, telnet did not work either.

[hadoop@pnjhadoopn= n01 ~]$ telnet=A0 pnjhadoopnn01 50070
Trying xx.xx.xx.xx...
telnet: connect to address xx.xx.xx.xx: Connection= refused
telnet: Unable to connect to remote host: Connection refused
[hadoop@pnjhadoopnn01 ~]$ telnet localhost 50070
Trying 127.0.0.1..= .
telnet: connect to address 127.0.0.1: Conn= ection refused
telnet: Unable to connect to remote host: Connection refu= sed

Thanks,

Jian
On Fri, Aug 10= , 2012 at 1:50 AM, Harsh J <harsh@cloudera.com> wrote:
Jian,

>From your NN, can you get us the output "netstat -anp | grep 50070&quo= t;?

On Fri, Aug 10, 2012 at 9:29 AM, Jian Fang
<jian.fang.subscribe@gmail.com> wrote:
> Thanks Harsh. But there is no firewall there, the two clusters are on = the
> same networks. I cannot telnet to the port even on the same machine. >
>
> On Thu, Aug 9, 2012 at 6:00 PM, Harsh J <harsh@cloudera.com> wrote:
>>
>> Hi Jian,
>>
>> HFTP is always-on by default. Can you check and make sure that the=
>> firewall isn't the cause of the connection refused on port 500= 70 on
>> the NN and ports 50075 on the DNs here?
>>
>> On Fri, Aug 10, 2012 at 1:47 AM, Jian Fang
>> <jian.fang.sub= scribe@gmail.com> wrote:
>> > Hi,
>> >
>> > We have a hadoop cluster of version 0.20.2 in production. Now= we have
>> > another new Hadoop cluster using cloudera's CDH3U4. We li= ke to run
>> > distcp to
>> > copy files between the two clusters. Since the hadoop version= s are
>> > different, we have to use hftp protocol to copy files based o= n the
>> > hadoop
>> > document here:
>> > http://hadoop.apache.org/common/docs/r0.20.= 2/distcp.html#cpver.
>> >
>> > The problem is that I cannot access files via hftp from the c= urrent
>> > production 0.20.2 cluster even though I can see the following= setting
>> > from
>> > job tracker UI.
>> >
>> > dfs.http.address pnjhadoopnn01:50070
>> >
>> > I tried to telnet this port, but got "connection refused= " error. Seems
>> > the
>> > hftp service is not actually running. Could someone tell me h= ow to
>> > enable
>> > the hftp service in the 0.20.2 hadoop cluster so that I can r= un distcp?
>> >
>> > Thanks in advance,
>> >
>> > John
>>
>>
>>
>> --
>> Harsh J
>
>



--
Harsh J

--f46d04287a177c6d5004c6ea03f6--