Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BF345D932 for ; Mon, 15 Oct 2012 23:22:47 +0000 (UTC) Received: (qmail 99418 invoked by uid 500); 15 Oct 2012 23:22:43 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 99331 invoked by uid 500); 15 Oct 2012 23:22:43 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 99315 invoked by uid 99); 15 Oct 2012 23:22:43 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 15 Oct 2012 23:22:43 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of adi@cloudera.com designates 209.85.210.176 as permitted sender) Received: from [209.85.210.176] (HELO mail-ia0-f176.google.com) (209.85.210.176) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 15 Oct 2012 23:22:38 +0000 Received: by mail-ia0-f176.google.com with SMTP id h11so4805744iae.35 for ; Mon, 15 Oct 2012 16:22:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding:x-gm-message-state; bh=ArJA6oUoKUHAlIZ7OKf0s0jECb9gFFobfhtMHWD7mnk=; b=bj08Ccb8K/NF0xXj/eexhBk1rWOu+j8asq7VJyCr7iJfoy9XzN+CmFsltn1KFN18g3 t2OZ4Bg7hVqLYrtV83/b3mSAupBPoWn8BWTtqucNFBv08mJVFVR6Gdk5rolPp/yMYdKg j6Y9tGfQSbgLNUM4PX9x0i8wxwGPzQHd4JFFPMHFcAp3W7v1ufUN0ERQBs6gquC7Bjms JYJaeQdTs4g+vQY0reAQHPGgjochGJUR8srreXIkEPdR1v6Lg6D5JckKcBnkPrrwo9M2 YAmKrk2apP8dQzk3sVOOzdjF/TsnGRjD0gilbpH6l7qAec15Jxri+IsHhYntrWZnPPeS 0n4A== MIME-Version: 1.0 Received: by 10.50.184.165 with SMTP id ev5mr8226125igc.32.1350343338249; Mon, 15 Oct 2012 16:22:18 -0700 (PDT) Received: by 10.64.124.3 with HTTP; Mon, 15 Oct 2012 16:22:18 -0700 (PDT) In-Reply-To: References: Date: Mon, 15 Oct 2012 16:22:18 -0700 Message-ID: Subject: Re: Issue when clicking on BrowseFileSystem From: Andy Isaacson To: user@hadoop.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Gm-Message-State: ALoCoQn6hZTPnRO3zR6fCGvCzie7YmJvVSjKRlaZ/KIqn/ns1bxJQcY4FOzn8PO6Sbb09ozDZgny X-Virus-Checked: Checked by ClamAV on apache.org On Mon, Oct 15, 2012 at 1:00 PM, Kartashov, Andy w= rote: > I run Hadoop in pseudo-distrib on amazon instance in the cloud. Shall I c= hange "localhost" in both core-site and mapred-site to my< my-host-name>? It depends on the desired behavior, but generally it's easiest if you have the hostnames in *.xml match the hostname of the host in question. Accessing the webUIs can be annoying depending on your browser config, however. > p.s. I can see Namenode status through Web-based interface using url: htt= p://:50070 however cannot access JobTracker (MapReduce maste= r) using : http://:50030 :( Keep in mind that opening up your EC2 Hadoop cluster to arbitrary HTTP access means that anyone in the world who guesses your IP can access the files you have stored on HDFS. Much better to use a VPN or ssh SOCKS5 port forwarding and keep the Hadoop TCP ports firewalled off using AWS security policies. There are unsavoury people who scan EC2 IP ranges looking for unsecured services like Hadoop, so don't think that "I didn't publish my IP, I am safe" will save you! -andy > > Andy Kartashov > > > -----Original Message----- > From: Andy Isaacson [mailto:adi@cloudera.com] > Sent: Monday, October 15, 2012 3:52 PM > To: user@hadoop.apache.org > Subject: Re: Issue when clicking on BrowseFileSystem > > Oh, you've *configured* "localhost" as your hostname in the hadoop *.xml = files. Yes, that'll result in the behavior you're seeing. > > I was assuming you were using a hostname that other machines can resolve.= For example running on my laptop I use "adit420" (which is what the laptop= calls itself). When running on VMs using > kvm+virt-manager, I configure my hostnames to the hostnames of the > VMs, for example vm01.local, vm02.local, et cetera. (The .local pseudo-d= omain is managed by avahi.) > > You mustn't delete the localhost line from /etc/hosts as that might cause= a lot of other stuff to fail, possibly including sshd, xorg, Gnome, etc. > > -andy > > On Mon, Oct 15, 2012 at 12:24 PM, Kartashov, Andy wrote: >> Andy, >> >> My /etc/hosts does say: 127.0.0.1 localhost.localdomain loc= alhost >> Shall I delete this entry? >> >> The only reference to localhost is in: >> >> Core-site: >> >> fs.default.name >> hdfs://localhost:8020 >> >> >> Mapred-site >> >> mapred.job.tracker >> localhost:8021 >> >> >> Andy Kartashov >> MPAC >> Architecture R&D, Co-op >> 1340 Pickering Parkway, Pickering, L1V 0C4 >> 1 Phone : (905) 837 6269 >> 6 Mobile: (416) 722 1787 >> andy.kartashov@mpac.ca >> >> -----Original Message----- >> From: Andy Isaacson [mailto:adi@cloudera.com] >> Sent: Friday, October 12, 2012 6:24 PM >> To: user@hadoop.apache.org >> Subject: Re: Issue when clicking on BrowseFileSystem >> >> On Fri, Oct 12, 2012 at 2:09 PM, Kartashov, Andy wrote: >>> It displays: >>> /browseDirectory.jsp?namenodeInfoPort=3D50070&dir=3D/&nnaddr=3Dlocalhos= t.lo >>> c >>> aldomain:8020 >> >> OK, there are two clues there that your DN and NN both think of themselv= es as being on localhost -- the http:// portion of the URL is sho= wing the DN address as localhost, and the nnaddr=3Dlocalhost is showing the= NN address as localhost. >> >> I'd check your /etc/hosts for your hostname and look at the hdfs-site.xm= l for any occurrences of localhost. >> >> -andy >> >>> >>> >>> Andy Kartashov >>> MPAC >>> Architecture R&D, Co-op >>> 1340 Pickering Parkway, Pickering, L1V 0C4 >>> 1 Phone : (905) 837 6269 >>> 6 Mobile: (416) 722 1787 >>> andy.kartashov@mpac.ca >>> >>> >>> -----Original Message----- >>> From: Andy Isaacson [mailto:adi@cloudera.com] >>> Sent: Friday, October 12, 2012 4:31 PM >>> To: user@hadoop.apache.org >>> Subject: Re: Issue when clicking on BrowseFileSystem >>> >>> On Fri, Oct 12, 2012 at 11:42 AM, Kartashov, Andy wrote: >>>> You are absolutely right. It was inded "localhost..:" in the URL. = When I changed it to my IP address the page duly loaded. Which .xml file is= responsible for this setting? >>> >>> It depends on which URL, you left out that part of the answer. :) When = you click the link on dfsHealth.jsp, does the error page have nn_browsedfsc= ontent.jsp in the URL bar, or does it have browseDirectory.jsp in the URL b= ar? >>> >>> The NN address can be specified with dfs.namenode.http-address and the = DN with dfs.datanode.http.address but you shouldn't need to configure those= by hand, the code is supposed to be smart enough to figure it out automati= cally. It works in my Hadoop 2.0.x configurations here, so I'd like to trac= k down what is different about your configuration in case we need to fix th= e code to DTRT for that kind of configuration. >>> >>> The first thing I would check is to make sure your server name is not >>> listed as 127.0.1.1 in /etc/hosts on the server. Since this (arguably >>> broken) configuration is the default for some distros, it would be nice= if Hadoop could handle it automatically, but in my experience things work = much better when I delete that line from /etc/hosts. >>> >>> -andy >>> >>>> Cheers, >>>> Andy >>>> >>>> >>>> -----Original Message----- >>>> From: Andy Isaacson [mailto:adi@cloudera.com] >>>> Sent: Friday, October 12, 2012 2:10 PM >>>> To: user@hadoop.apache.org >>>> Subject: Re: Issue when clicking on BrowseFileSystem >>>> >>>> Andy, >>>> >>>> It sounds like your namenode thinks the datanode is on localhost. The = "browse the filesystem" URL gets its IP address from the DN registration me= ssage. Is the URL that throws dns_unresolved_hostname the nn_browsedfsconte= nt.jsp URL, or the browseDirectory.jsp that it redirects to? >>>> >>>> What's your dfs.namenode.rpc-address and http-address settings? Are th= ere any odd entries in /etc/hosts like having your hostname listed on a 127= .0.0.1 line? Is this on a single-node cluster or a multi-node cluster? >>>> >>>> If none of the above seems diagnostic, what does the >>>> "NameSystem.registerDatanode: node registration from DatanodeRegistrat= ion(192.168.122.87" log messages in your namenode log look like? >>>> >>>> -andy >>>> >>>> On Fri, Oct 12, 2012 at 10:20 AM, Kartashov, Andy wrote: >>>>> hadoop version >>>>> Hadoop 2.0.0-cdh4.0.1 >>>>> Subversion >>>>> file:///data/1/jenkins/workspace/generic-package-rhel64-6-0/topdir/ >>>>> B U I >>>>> LD/hadoop-2.0.0-cdh4.0.1/src/hadoop-common-project/hadoop-common >>>>> >>>>> Andy Kartashov >>>>> MPAC >>>>> Architecture R&D, Co-op >>>>> 1340 Pickering Parkway, Pickering, L1V 0C4 >>>>> 1 Phone : (905) 837 6269 >>>>> 6 Mobile: (416) 722 1787 >>>>> andy.kartashov@mpac.ca >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: Harsh J [mailto:harsh@cloudera.com] >>>>> Sent: Friday, October 12, 2012 11:41 AM >>>>> To: user@hadoop.apache.org >>>>> Subject: Re: Issue when clicking on BrowseFileSystem >>>>> >>>>> What versions of Hadoop are you both running? I seem to have it worki= ng fine on a consistent networking environment setup. >>>>> >>>>> On Fri, Oct 12, 2012 at 6:37 PM, Kartashov, Andy wrote: >>>>>> I do too can open :50070/dfshealth.jsp page and click NameN= ode Logs link however when I click on Browse the filesystem link I get the = following: >>>>>> >>>>>> Network Error (dns_unresolved_hostname) Your requested host >>>>>> "localhost.localdomain" could not be resolved by DNS. >>>>>> For assistance, contact your network support team. >>>>>> >>>>>> Rgds, >>>>>> Andy >>>>>> >>>>>> -----Original Message----- >>>>>> From: Serge Blazhiyevskyy [mailto:Serge.Blazhiyevskyy@nice.com] >>>>>> Sent: Thursday, October 11, 2012 5:35 PM >>>>>> To: user@hadoop.apache.org; murthy nvvs >>>>>> Subject: Re: Issue when clicking on BrowseFileSystem >>>>>> >>>>>> Take a look at your conf files. Normally it is in hdfs-site.xml >>>>>> >>>>>> Look for name node name or address. Set it to loclahost or >>>>>> 127.0.0.1 >>>>>> >>>>>> That might help. >>>>>> >>>>>> Serge >>>>>> >>>>>> On 10/11/12 2:31 PM, "Andy Isaacson" wrote: >>>>>> >>>>>>>What Hadoop version did you install? Can you provide the .address >>>>>>>settings for your NN and one of your DNs? If you visit >>>>>>>/dfshealth.jsp on your NN, how many live nodes and dead nodes does i= t report? >>>>>>> >>>>>>>-andy >>>>>>> >>>>>>>On Wed, Oct 10, 2012 at 9:54 PM, murthy nvvs >>>>>>> >>>>>>>wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I had made Hadoop setup & set all the configuration files. >>>>>>>> when go to this url through browser http://localhost:50070, it >>>>>>>>opened well but when i clicked on BrowseTheFileSystem , its >>>>>>>>redirecting to >>>>>>>>0.0.0.0:50070/browseDirectory.jsp?namenodeinfoport=3D500070&dir=3D/ >>>>>>>> its showing some 404 error in this page. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Murthy >>>>>> >>>>>> NOTICE: This e-mail message and any attachments are confidential, >>>>>> subject to copyright and may be privileged. Any unauthorized use, >>>>>> copying or disclosure is prohibited. If you are not the intended >>>>>> recipient, please delete and contact the sender immediately. >>>>>> Please consider the environment before printing this e-mail. AVIS >>>>>> : le pr=E9sent courriel et toute pi=E8ce jointe qui l'accompagne son= t >>>>>> confidentiels, prot=E9g=E9s par le droit d'auteur et peuvent =EAtre >>>>>> couverts par le secret professionnel. Toute utilisation, copie ou >>>>>> divulgation non autoris=E9e est interdite. Si vous n'=EAtes pas le >>>>>> destinataire pr=E9vu de ce courriel, supprimez-le et contactez imm= =E9diatement l'exp=E9diteur. >>>>>> Veuillez penser =E0 l'environnement avant d'imprimer le pr=E9sent >>>>>> courriel >>>>> >>>>> >>>>> >>>>> -- >>>>> Harsh J >>>>> NOTICE: This e-mail message and any attachments are confidential, >>>>> subject to copyright and may be privileged. Any unauthorized use, >>>>> copying or disclosure is prohibited. If you are not the intended >>>>> recipient, please delete and contact the sender immediately. Please >>>>> consider the environment before printing this e-mail. AVIS : le >>>>> pr=E9sent courriel et toute pi=E8ce jointe qui l'accompagne sont >>>>> confidentiels, prot=E9g=E9s par le droit d'auteur et peuvent =EAtre >>>>> couverts par le secret professionnel. Toute utilisation, copie ou >>>>> divulgation non autoris=E9e est interdite. Si vous n'=EAtes pas le >>>>> destinataire pr=E9vu de ce courriel, supprimez-le et contactez imm=E9= diatement l'exp=E9diteur. >>>>> Veuillez penser =E0 l'environnement avant d'imprimer le pr=E9sent >>>>> courriel >>>> NOTICE: This e-mail message and any attachments are confidential, >>>> subject to copyright and may be privileged. Any unauthorized use, >>>> copying or disclosure is prohibited. If you are not the intended >>>> recipient, please delete and contact the sender immediately. Please >>>> consider the environment before printing this e-mail. AVIS : le >>>> pr=E9sent courriel et toute pi=E8ce jointe qui l'accompagne sont >>>> confidentiels, prot=E9g=E9s par le droit d'auteur et peuvent =EAtre >>>> couverts par le secret professionnel. Toute utilisation, copie ou >>>> divulgation non autoris=E9e est interdite. Si vous n'=EAtes pas le >>>> destinataire pr=E9vu de ce courriel, supprimez-le et contactez imm=E9d= iatement l'exp=E9diteur. >>>> Veuillez penser =E0 l'environnement avant d'imprimer le pr=E9sent >>>> courriel >>> NOTICE: This e-mail message and any attachments are confidential, >>> subject to copyright and may be privileged. Any unauthorized use, >>> copying or disclosure is prohibited. If you are not the intended >>> recipient, please delete and contact the sender immediately. Please >>> consider the environment before printing this e-mail. AVIS : le >>> pr=E9sent courriel et toute pi=E8ce jointe qui l'accompagne sont >>> confidentiels, prot=E9g=E9s par le droit d'auteur et peuvent =EAtre >>> couverts par le secret professionnel. Toute utilisation, copie ou >>> divulgation non autoris=E9e est interdite. Si vous n'=EAtes pas le >>> destinataire pr=E9vu de ce courriel, supprimez-le et contactez imm=E9di= atement l'exp=E9diteur. >>> Veuillez penser =E0 l'environnement avant d'imprimer le pr=E9sent >>> courriel >> NOTICE: This e-mail message and any attachments are confidential, >> subject to copyright and may be privileged. Any unauthorized use, >> copying or disclosure is prohibited. If you are not the intended >> recipient, please delete and contact the sender immediately. Please >> consider the environment before printing this e-mail. AVIS : le >> pr=E9sent courriel et toute pi=E8ce jointe qui l'accompagne sont >> confidentiels, prot=E9g=E9s par le droit d'auteur et peuvent =EAtre couv= erts >> par le secret professionnel. Toute utilisation, copie ou divulgation >> non autoris=E9e est interdite. Si vous n'=EAtes pas le destinataire pr= =E9vu >> de ce courriel, supprimez-le et contactez imm=E9diatement l'exp=E9diteur= . >> Veuillez penser =E0 l'environnement avant d'imprimer le pr=E9sent courri= el > NOTICE: This e-mail message and any attachments are confidential, subject= to copyright and may be privileged. Any unauthorized use, copying or discl= osure is prohibited. If you are not the intended recipient, please delete a= nd contact the sender immediately. Please consider the environment before p= rinting this e-mail. AVIS : le pr=E9sent courriel et toute pi=E8ce jointe q= ui l'accompagne sont confidentiels, prot=E9g=E9s par le droit d'auteur et p= euvent =EAtre couverts par le secret professionnel. Toute utilisation, copi= e ou divulgation non autoris=E9e est interdite. Si vous n'=EAtes pas le des= tinataire pr=E9vu de ce courriel, supprimez-le et contactez imm=E9diatement= l'exp=E9diteur. Veuillez penser =E0 l'environnement avant d'imprimer le pr= =E9sent courriel