Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3A721E26B for ; Sun, 24 Feb 2013 11:54:16 +0000 (UTC) Received: (qmail 93456 invoked by uid 500); 24 Feb 2013 11:54:11 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 93329 invoked by uid 500); 24 Feb 2013 11:54:10 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 93295 invoked by uid 99); 24 Feb 2013 11:54:09 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 24 Feb 2013 11:54:09 +0000 X-ASF-Spam-Status: No, hits=2.9 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [106.10.148.242] (HELO nm11-vm3.bullet.mail.sg3.yahoo.com) (106.10.148.242) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 24 Feb 2013 11:54:01 +0000 Received: from [106.10.166.120] by nm11.bullet.mail.sg3.yahoo.com with NNFMP; 24 Feb 2013 11:53:38 -0000 Received: from [106.10.150.27] by tm9.bullet.mail.sg3.yahoo.com with NNFMP; 24 Feb 2013 11:53:38 -0000 Received: from [127.0.0.1] by omp1028.mail.sg3.yahoo.com with NNFMP; 24 Feb 2013 11:53:38 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 615230.64801.bm@omp1028.mail.sg3.yahoo.com Received: (qmail 43761 invoked by uid 60001); 24 Feb 2013 11:53:38 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.in; s=s1024; t=1361706818; bh=p7QgNGDATMHHGLXNqtnuJXS+9AKOmp9TBVGfgUpwwxs=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=zWQ9fMo92RRyMYziAeDx5Bq6DBL7zITBqd2KfOjowoXYhovCQjPGeyd2k+vBP9RK/tN2ddpz49UDgilZW4AhX9uXF5oBDgo+p2ep3N5wp9Bgmguo0eYKUM3EC3Yt+Q8Lxvpe01PfDfUfP4EWK4cU8QdDHTpi+XuOzTtFio3BBBM= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.in; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=E31CqVB4Hu1oXV9Go/J2ycqeIjYPh9ou5HgusvNUWDFQDgsfZL9NPMIqv2+rXCF/nP4OkDM6v1NByRU+85zpAQD0uUMAPSlyOWJQow5iJCpuyBcKRBJ1l1kJ90YJokeQdQYojmIq5Npl4AtcEO21EbSLn7Y2pUkAuDjJ739WDpY=; X-YMail-OSG: 8XQ94P0VM1kr8Dp2FyzNeJ0sA2UholUHvfekxBTYgkmitqA 42vy4TGc8KpyXGdDQfJIbsnBZnl56Bb_m9NKq30Bp1mxZWzP_ZiQOjVyanLs YioZ24CGL83yrNVvsUtZmRX3FVG8gtFnGyConxcgcO3V8PoxV1dWehvr.14t Z4gLpItdlsNqJ7gQrlLN_T88ZQWgd22FdM6CM6wIkVfHOMqVcN9cZJ.sl.cU N9epQNC9eDOplo0h98hyuMkfhJnVZbleDr.2TpN_7wOT51F_cZXPb1Ake6_. 5JiXk4E7X9SPfpSMpqZc66QDbyIsyXO2oMEo7hKKD5.CeqMZWyR2WwNwooEQ tCclOjvVbLKiNflz4GyzPura04clujbDsB0s6CVrlNw9x4NFxhSwo7IjRJlL _d2w_h2Wvn1HnJBDCD4jhVAGwVBK1tyystJ1rhuU7802AVd3S Received: from [115.242.118.243] by web194702.mail.sg3.yahoo.com via HTTP; Sun, 24 Feb 2013 19:53:38 SGT X-Rocket-MIMEInfo: 001.001,TWFueSBUaGFua3MgTml0aW4gZm9yIHlvdXIgcXVpY2sgcmVwbHkuCgpIZXJlcyB3aGF0IGkgaGF2ZSBpbiBteSBob3N0cyBmaWxlIGFuZCBpIGFtIHJ1bm5pbmcgaW4gVk0gaSBtIGFzc3VtaW5nIGl0IGlzIHRoZSBwc2V1ZG8gbW9kZToKCioqKioqKioqKioqKioqKioqKioqKgoxMjcuMC4wLjHCoMKgwqAgbG9jYWxob3N0LmxvY2FsZG9tYWluwqDCoMKgIGxvY2FsaG9zdAojOjoxwqDCoMKgIHVidW50dcKgwqDCoCBsb2NhbGhvc3Q2LmxvY2FsZG9tYWluNsKgwqDCoCBsb2NhbGhvc3Q2CiMxMjcuMC4xLjHCoAEwAQEBAQ-- X-Mailer: YahooMailWebService/0.8.134.513 References: <1361612254.36060.YahooMailNeo@web194705.mail.sg3.yahoo.com> <1361623966.2874.YahooMailNeo@web194705.mail.sg3.yahoo.com> <1361706030.51309.YahooMailNeo@web194706.mail.sg3.yahoo.com> Message-ID: <1361706818.42489.YahooMailNeo@web194702.mail.sg3.yahoo.com> Date: Sun, 24 Feb 2013 19:53:38 +0800 (SGT) From: Sai Sai Reply-To: Sai Sai Subject: Re: Trying to copy file to Hadoop file system from a program To: "user@hadoop.apache.org" In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="-924933940-1558606288-1361706818=:42489" X-Virus-Checked: Checked by ClamAV on apache.org ---924933940-1558606288-1361706818=:42489 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Many Thanks Nitin for your quick reply.=0A=0AHeres what i have in my hosts = file and i am running in VM i m assuming it is the pseudo mode:=0A=0A******= ***************=0A127.0.0.1=A0=A0=A0 localhost.localdomain=A0=A0=A0 localho= st=0A#::1=A0=A0=A0 ubuntu=A0=A0=A0 localhost6.localdomain6=A0=A0=A0 localho= st6=0A#127.0.1.1=A0=A0=A0 ubuntu=0A127.0.0.1=A0=A0 ubuntu=0A=0A# The follow= ing lines are desirable for IPv6 capable hosts=0A::1=A0=A0=A0=A0 localhost = ip6-localhost ip6-loopback=0Afe00::0 ip6-localnet=0Aff00::0 ip6-mcastprefix= =0Aff02::1 ip6-allnodes=0Aff02::2 ip6-allrouters=0Aff02::3 ip6-allhosts=0A*= ********************=0AIn my masters i have:=0Aubuntu=0AIn my slaves i have= :=0Alocalhost=0A***********************=0AMy question is in my variable bel= ow:=0Apublic static String fsURI =3D "hdfs://master:9000";=0A=0Awhat would = be the right value so i can connect to Hadoop successfully.=0APlease let me= know if you need more info.=0AThanks=0ASai=0A=0A=0A=0A=0A=0A=0A=0A________= ________________________=0A From: Nitin Pawar =0AT= o: user@hadoop.apache.org; Sai Sai =0ASent: Sunday, 24 = February 2013 3:42 AM=0ASubject: Re: Trying to copy file to Hadoop file sys= tem from a program=0A =0A=0Aif you want to use master as your hostname then= make such entry in your /etc/hosts file=A0=0A=0Aor change the hdfs://maste= r to hdfs://localhost=A0=0A=0A=0A=0AOn Sun, Feb 24, 2013 at 5:10 PM, Sai Sa= i wrote:=0A=0A=0A>=0A>Greetings,=0A>=0A>=0A>Below is th= e program i am trying to run and getting this exception:=0A>***************= ************************=0A>=0A>Test Start.....=0A>java.net.UnknownHostExce= ption: unknown host: master=0A>=A0=A0=A0 at org.apache.hadoop.ipc.Client$Co= nnection.(Client.java:214)=0A>=A0=A0=A0 at org.apache.hadoop.ipc.Clie= nt.getConnection(Client.java:1196)=0A>=A0=A0=A0 at org.apache.hadoop.ipc.Cl= ient.call(Client.java:1050)=0A>=A0=A0=A0 at org.apache.hadoop.ipc.RPC$Invok= er.invoke(RPC.java:225)=0A>=A0=A0=A0 at $Proxy1.getProtocolVersion(Unknown = Source)=0A>=A0=A0=A0 at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)=0A= >=A0=A0=A0 at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)=0A>=A0=A0=A0= at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:119)= =0A>=A0=A0=A0 at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:238= )=0A>=A0=A0=A0 at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:20= 3)=0A>=A0=A0=A0 at=0A org.apache.hadoop.hdfs.DistributedFileSystem.initiali= ze(DistributedFileSystem.java:89)=0A>=A0=A0=A0 at kelly.hadoop.hive.test.Ha= doopTest.main(HadoopTest.java:54)=0A>=0A>=0A>=0A>=0A>********************= =0A>=0A>=0A>=0A>public class HdpTest {=0A>=A0=A0=A0 =0A>=A0=A0=A0 public st= atic String fsURI =3D "hdfs://master:9000";=0A>=0A>=A0=A0=A0 =0A>=A0=A0=A0 = public static void copyFileToDFS(FileSystem fs, String srcFile,=0A>=A0=A0= =A0=0A =A0=A0=A0 =A0=A0=A0 String dstFile) throws IOException {=0A>=A0=A0= =A0 =A0=A0=A0 try {=0A>=A0=A0=A0 =A0=A0=A0 =A0=A0=A0 System.out.println("In= itialize copy...");=0A>=A0=A0=A0 =A0=A0=A0 =A0=A0=A0 URI suri =3D new URI(s= rcFile);=0A>=A0=A0=A0 =A0=A0=A0 =A0=A0=A0 URI duri =3D new URI(fsURI + "/" = + dstFile);=0A>=A0=A0=A0 =A0=A0=A0 =A0=A0=A0 Path dst =3D new Path(duri.toS= tring());=0A>=A0=A0=A0 =A0=A0=A0 =A0=A0=A0 Path src =3D new Path(suri.toStr= ing());=0A>=A0=A0=A0 =A0=A0=A0 =A0=A0=A0 System.out.println("Start copy..."= );=0A>=A0=A0=A0 =A0=A0=A0 =A0=A0=A0 fs.copyFromLocalFile(src, dst);=0A>=A0= =A0=A0 =A0=A0=A0 =A0=A0=A0 System.out.println("End copy...");=0A>=A0=A0=A0 = =A0=A0=A0 } catch (Exception e)=0A {=0A>=A0=A0=A0 =A0=A0=A0 =A0=A0=A0 e.pri= ntStackTrace();=0A>=A0=A0=A0 =A0=A0=A0 }=0A>=A0=A0=A0 }=0A>=0A>=A0=A0=A0 pu= blic static void main(String[] args) {=0A>=A0=A0=A0 =A0=A0=A0 try {=0A>=A0= =A0=A0 =A0=A0=A0 =A0=A0=A0 System.out.println("Test Start.....");=0A>=A0=A0= =A0 =A0=A0=A0 =A0=A0=A0 Configuration conf =3D new Configuration();=0A>=A0= =A0=A0 =A0=A0=A0 =A0=A0=A0 DistributedFileSystem fs =3D new DistributedFile= System();=0A>=A0=A0=A0 =A0=A0=A0 =A0=A0=A0 URI duri =3D new URI(fsURI);=0A>= =A0=A0=A0 =A0=A0=A0 =A0=A0=A0 fs.initialize(duri, conf); // Here is the xce= ption occuring=0A>=A0=A0=A0 =A0=A0=A0 =A0=A0=A0 long start =3D 0, end =3D 0= ;=0A>=A0=A0=A0 =A0=A0=A0=0A =A0=A0=A0 start =3D System.nanoTime();=0A>=A0= =A0=A0 =A0=A0=A0 =A0=A0=A0 //writing data from local to HDFS=0A>=A0=A0=A0 = =A0=A0=A0 =A0=A0=A0 copyFileToDFS(fs, "/home/kosmos/Work/input/wordpair.txt= ",=0A>=A0=A0=A0 =A0=A0=A0 =A0=A0=A0 =A0=A0=A0 =A0=A0=A0 "/input/raptor/trad= e1.txt");=0A>=A0=A0=A0 =A0=A0=A0 =A0=A0=A0 //Writing data from HDFS to Loca= l=0A>// =A0=A0 =A0=A0=A0 =A0=A0=A0=A0 copyFileFromDFS(fs, "/input/raptor/tr= ade1.txt", "/home/kosmos/Work/input/wordpair1.txt");=0A>=A0=A0=A0 =A0=A0=A0= =A0=A0=A0 end =3D System.nanoTime();=0A>=A0=A0=A0 =A0=A0=A0 =A0=A0=A0 Syst= em.out.println("Total Execution times: " + (end - start));=0A>=A0=A0=A0 =A0= =A0=A0 =A0=A0=A0 fs.close();=0A>=A0=A0=A0 =A0=A0=A0 } catch=0A (Throwable t= ) {=0A>=A0=A0=A0 =A0=A0=A0 =A0=A0=A0 t.printStackTrace();=0A>=A0=A0=A0 =A0= =A0=A0 }=0A>=A0=A0=A0 }=0A>=0A>******************************=0A>I am tryin= g to access in FireFox this url: =0A>=0A>hdfs://master:9000=0A>=0A>=0A>Get = an error msg FF does not know how to display this message.=0A>=0A>=0A>I can= successfully access my admin page:=0A>=0A>=0A>http://localhost:50070/dfshe= alth.jsp=0A>=0A>=0A>Just wondering if anyone can give me any suggestions, y= our help will be really appreciated.=0A>ThanksSai=0A>=0A>=0A>=0A=0A=0A-- = =0ANitin Pawar ---924933940-1558606288-1361706818=:42489 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Many Thanks Nitin for= your quick reply.

Heres what i have in my hosts file and i am runni= ng in VM i m assuming it is the pseudo mode:

*********************127.0.0.1    localhost.localdomain    local= host
#::1    ubuntu    localhost6.localdom= ain6    localhost6
#127.0.1.1    ubuntu127.0.0.1   ubuntu

# The following lines are desirable fo= r IPv6 capable hosts
::1     localhost ip6-localhost= ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1= ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
********= *************
In my masters i have:
ubuntu
In my slaves i have:localhost
***********************
My question is in my variable below:
public static String fsURI =3D "hdfs://master:9000";
<= br>what would be the right value so i can connect to Hadoop successfully.Please let me know if you need more info.
Thanks
Sai

=



From: Nitin Pawar <nitinpawar= 432@gmail.com>
To: = user@hadoop.apache.org; Sai Sai <saigraph@yahoo.in>
Sent: Sunday, 24 February 2013 3:42 AM=
Subject: Re: Trying t= o copy file to Hadoop file system from a program

if you want to use master as your hostname then make= such entry in your /etc/hosts file 

or = change the hdfs://master to hdfs://localhost 


=0A
On Sun, Feb 24, 2013 at 5:10 PM, Sai Sai <saigraph@yahoo.in> wrote:
=0A

G= reetings,
=0A
Below is the program i am trying to run and getting this exception:
<= div style=3D"font-style:normal;font-size:16px;background-color:transparent;= font-family:times new roman, new york, times, serif;">=0A******************= *********************
Test=0A Start.....
java.net.UnknownHostException: unknown hos= t: master
    at org.apache.hadoop.ipc.Client$Connection.= <init>(Client.java:214)
    at org.apache.hadoop.ip= c.Client.getConnection(Client.java:1196)
=0A    at org.ap= ache.hadoop.ipc.Client.call(Client.java:1050)
    at org.= apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
    at= $Proxy1.getProtocolVersion(Unknown Source)
    at org.ap= ache.hadoop.ipc.RPC.getProxy(RPC.java:396)
=0A    at org.= apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
    at org.a= pache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:119)
 =    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.jav= a:238)
    at org.apache.hadoop.hdfs.DFSClient.<init&g= t;(DFSClient.java:203)
=0A    at=0A org.apache.hadoop.hdf= s.DistributedFileSystem.initialize(DistributedFileSystem.java:89)
 =    at kelly.hadoop.hive.test.HadoopTest.main(HadoopTest.java:54)<= br>
=0A
=
**************= ******

=0Apublic class HdpTest {
   
 =    public static String fsURI =3D "hdfs://master:9000";

&n= bsp;  
    public static void copyFileToDFS(Fi= leSystem fs, String srcFile,
   =0A     &n= bsp;   String dstFile) throws IOException {
   =     try {
         &n= bsp;  System.out.println("Initialize copy...");
    =         URI suri =3D new URI(srcFile);
&nb= sp;           URI duri =3D new URI(= fsURI + "/" + dstFile);
=0A         &= nbsp;  Path dst =3D new Path(duri.toString());
    &= nbsp;       Path src =3D new Path(suri.toString())= ;
            System.out.pr= intln("Start copy...");
         &nbs= p;  fs.copyFromLocalFile(src, dst);
      =       System.out.println("End copy...");
=0A &n= bsp;      } catch (Exception e)=0A {
  &nb= sp;         e.printStackTrace();
 &nb= sp;      }
    }

  &= nbsp; public static void main(String[] args) {
     =    try {
          &nb= sp; System.out.println("Test Start.....");
     &nbs= p;      Configuration conf =3D new Configuration();
= =0A            DistributedFile= System fs =3D new DistributedFileSystem();
     &nbs= p;      URI duri =3D new URI(fsURI);
  &nb= sp;        = fs.initialize(duri, conf); // Here is the xception occuring
=0A&= nbsp;           long start =3D 0, e= nd =3D 0;
       =0A     st= art =3D System.nanoTime();
         &= nbsp;  //writing data from local to HDFS
     &= nbsp;      copyFileToDFS(fs, "/home/kosmos/Work/input/w= ordpair.txt",
            &= nbsp;       "/input/raptor/trade1.txt");
=0A&nb= sp;           //Writing data from H= DFS to Local
//            = copyFileFromDFS(fs, "/input/raptor/trade1.txt", "/home/kosmos/Work/input/w= ordpair1.txt");
           = end =3D System.nanoTime();
         =    System.out.println("Total Execution times: " + (end - start));=
=0A            fs.close();=
        } catch=0A (Throwable t) {
&nb= sp;           t.printStackTrace();<= br>        }
    }
=0A******************************
=0A
I am trying to= access in FireFox this url:
=0A
=0A
hdfs://master:9000
=0A

=0A
= =0A
Get an erro= r msg FF does not know how to display this message.
=0A

=0A
= =0A
I can succe= ssfully access my admin page:
=0A

=0A
=0A=0A=0A

=0AJust wondering if anyone can give me any sugges= tions, your help will be really appreciated.
=0AThanks
Sai
=0A



-- Nitin Pawar
=0A
=0A


---924933940-1558606288-1361706818=:42489--