Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2F87B10A1D for ; Tue, 5 Nov 2013 11:59:04 +0000 (UTC) Received: (qmail 48694 invoked by uid 500); 5 Nov 2013 11:58:54 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 48258 invoked by uid 500); 5 Nov 2013 11:58:47 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 48251 invoked by uid 99); 5 Nov 2013 11:58:45 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Nov 2013 11:58:45 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [130.238.7.73] (HELO lyra.its.uu.se) (130.238.7.73) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Nov 2013 11:58:41 +0000 X-Virus-Scanned: amavisd-new at uu.se X-DKIM: Sendmail DKIM Filter v2.8.3 lyra.its.uu.se ECAE4398C2 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=uu.se; s=centralsmtp; t=1383652697; i=@uu.se; bh=xkyJqd/m55IJOXx9WdH1ytfVGsw5khIcFndjxby+9zs=; h=From:Mime-Version:Content-Type:Subject:Date:In-Reply-To:To: References:Message-Id; b=UkueLUZFRH6MEGeayOKXUzlMOaOADIAP659Xb5Mi5ybF3B9dX5148h4EYscPQJ7kN HjGFd2zigg3yEFKa+W5c00IqnV5cY1/9cf+NQyrYxhq2ZIvb7zqi0fgof34EUnNGrp TYCFcYzpv6JJcsx2c+DTOaelVALKxMZA01o1qh/k= Received: from [192.168.0.14] (c83-253-249-32.bredband.comhem.se [83.253.249.32]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by lyra.its.uu.se (Postfix) with ESMTPSA id ECAE4398C2 for ; Tue, 5 Nov 2013 12:58:14 +0100 (CET) From: Salman Toor Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: multipart/alternative; boundary="Apple-Mail=_52501C75-C759-40D5-AD1A-74E8A62A8673" Subject: Re: C++ example for hadoop-2.2.0 Date: Tue, 5 Nov 2013 12:58:08 +0100 In-Reply-To: To: user@hadoop.apache.org References: <31CE858D-E7A4-4338-B942-D50407DFBC84@it.uu.se> <8B8D3BAF-314C-4D40-80FA-312ACB19755B@it.uu.se> <65D4BF50-33D5-49B0-93A0-E7120E195972@it.uu.se> <23082995-F21E-4717-AF35-A5C0251A5D75@it.uu.se> Message-Id: <63311911-5F71-45B8-863B-4CF3560E6F46@it.uu.se> X-Mailer: Apple Mail (2.1283) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_52501C75-C759-40D5-AD1A-74E8A62A8673 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Hi, I have solved the problem! Have downloaded the source, compiled it on = the machine and now can successfully link to the library.=20 Thank you everyone for you quick responses. Regards.. Salman.=20 Salman Toor, PhD salman.toor@it.uu.se On Nov 4, 2013, at 3:21 PM, Andre Kelpe wrote: > No, because I was trying to set up a cluster automatically with the > tarballs from apache.org. >=20 > - Andr=E9 >=20 > On Mon, Nov 4, 2013 at 3:05 PM, Salman Toor = wrote: >> Hi, >>=20 >> Did you tried to compile with source? >>=20 >> /Salman. >>=20 >>=20 >> Salman Toor, PhD >> salman.toor@it.uu.se >>=20 >>=20 >>=20 >> On Nov 4, 2013, at 2:55 PM, Andre Kelpe wrote: >>=20 >> I reported the 32bit/64bit problem a few weeks ago. There hasn't been >> much activity around it though: >> https://issues.apache.org/jira/browse/HADOOP-9911 >>=20 >> - Andr=E9 >>=20 >> On Mon, Nov 4, 2013 at 2:20 PM, Salman Toor = wrote: >>=20 >> Hi, >>=20 >>=20 >> Ok so 2.x is not a new version its another branch. Good to know! = Actually >>=20 >> 32bit will be difficult as the code I got have already some = dependencies on >>=20 >> 64 bit. >>=20 >>=20 >> Otherwise I will continue with 1.x version. Can you suggest some = version >>=20 >> with 1.x series which is stable and work on the cluster environment? >>=20 >> especially with C++ ... >>=20 >>=20 >> Regards.. >>=20 >> Salman. >>=20 >>=20 >> Salman Toor, PhD >>=20 >> salman.toor@it.uu.se >>=20 >>=20 >>=20 >>=20 >> On Nov 4, 2013, at 1:54 PM, Amr Shahin wrote: >>=20 >>=20 >> Well, the 2 series isn't exactly the "next version". It's a = continuation of >>=20 >> branch .2. >>=20 >> Also, the error message from the gcc indicates that the library = you're >>=20 >> trying to link to isn't compatible which made me suspicious. check = the >>=20 >> documentation to see if hadoop has 64 libraries, or otherwise compile >>=20 >> against the 32 ones >>=20 >>=20 >>=20 >> On Mon, Nov 4, 2013 at 4:51 PM, Salman Toor = wrote: >>=20 >>=20 >> Hi, >>=20 >>=20 >> Thanks for your answer! >>=20 >>=20 >> But are you sure about it? Actually Hadoop version 1.2 have both 32 = and 64 >>=20 >> bit libraries so I believe the the next version should have both... = But I am >>=20 >> not sure just a random guess :-( >>=20 >>=20 >> Regards.. >>=20 >> Salman. >>=20 >>=20 >> Salman Toor, PhD >>=20 >> salman.toor@it.uu.se >>=20 >>=20 >>=20 >>=20 >> On Nov 4, 2013, at 1:38 PM, Amr Shahin wrote: >>=20 >>=20 >> I believe hadoop isn't compatible with 64 architecture. Try = installing the >>=20 >> 32 libraries and compile against them. >>=20 >> This error (skipping incompatible >>=20 >> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching >>=20 >> -lhadooppipes) indicates so >>=20 >>=20 >>=20 >> On Mon, Nov 4, 2013 at 2:44 PM, Salman Toor = wrote: >>=20 >>=20 >> Hi, >>=20 >>=20 >> Can someone give a pointer? >>=20 >>=20 >>=20 >> Thanks in advance. >>=20 >>=20 >> Regards.. >>=20 >> Salman. >>=20 >>=20 >>=20 >> Salman Toor, PhD >>=20 >> salman.toor@it.uu.se >>=20 >>=20 >>=20 >>=20 >> On Nov 3, 2013, at 11:31 PM, Salman Toor wrote: >>=20 >>=20 >> Hi, >>=20 >>=20 >> I am quite new to the Hadoop world, previously was running = hadoop-1.2.0 >>=20 >> stable version on my small cluster and encountered some strange = problems >>=20 >> like the local path to the mapper file didn't copy to the hdfs .... = It >>=20 >> works fine on the single node setup but on multiple node simple = word-count >>=20 >> python example didn't work... I read on the blog that it might be = the >>=20 >> problem in the version I am using. So I thought to change the version = and >>=20 >> downloaded the Hadoop 2.2.0. This version has yarn together with many = new >>=20 >> features that I hope I will learn in the future. Now simple wordcount >>=20 >> example works without any problem on the multi-node setup. I am using = simple >>=20 >> python example. >>=20 >>=20 >> Now I would like to compile my C++ code. Since the directory = structure >>=20 >> together with other things have been changed. I have started to get = the >>=20 >> following error: >>=20 >>=20 >> ---------------- >>=20 >> /urs/bin/ld: skipping incompatible >>=20 >> /home/sztoor/hadoop-2.2.0/lib/native/libhadooputils.a when searching >>=20 >> -lhadooputils >>=20 >> cannot find -lhadooputils >>=20 >>=20 >> /urs/bin/ld: skipping incompatible >>=20 >> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching >>=20 >> -lhadooppipes >>=20 >> cannot find -lhadooppipes >>=20 >> ------------------ >>=20 >>=20 >> I have managed to run the c++ example successfully with the 1.2.0 = version >>=20 >> on single node setup. >>=20 >>=20 >> I am having 64bit Ubuntu machine. previously I was using = Linux-amd64-64 >>=20 >>=20 >> Now in new version, "lib" and "include" directories are in the >>=20 >> hadoop-2.2.0 directory. No build.xml is available... >>=20 >>=20 >> Can someone please give me an example of a makefile based on the = version >>=20 >> 2.2.0? Or suggest me which version I should go for? Or if there are = some >>=20 >> prerequisites that I should do before compiling my code? >>=20 >>=20 >> Thanks in advance. >>=20 >>=20 >> Regards.. >>=20 >> Salman. >>=20 >>=20 >>=20 >>=20 >> Salman Toor, PhD >>=20 >> salman.toor@it.uu.se >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >> -- >> Andr=E9 Kelpe >> andre@concurrentinc.com >> http://concurrentinc.com >>=20 >>=20 >=20 >=20 >=20 > --=20 > Andr=E9 Kelpe > andre@concurrentinc.com > http://concurrentinc.com --Apple-Mail=_52501C75-C759-40D5-AD1A-74E8A62A8673 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1
Salman = Toor, PhD


On Nov 4, 2013, at 3:21 PM, Andre Kelpe wrote:

No, = because I was trying to set up a cluster automatically with = the
tarballs from apache.org.

- Andr=E9

On = Mon, Nov 4, 2013 at 3:05 PM, Salman Toor <salman.toor@it.uu.se> = wrote:
Hi,

Did you tried = to compile with source?

/Salman.


Salman Toor, = PhD
salman.toor@it.uu.se



On Nov 4, 2013, = at 2:55 PM, Andre Kelpe wrote:

I reported the = 32bit/64bit problem a few weeks ago. There hasn't = been
much activity around it = though:
https://issues.= apache.org/jira/browse/HADOOP-9911

- = Andr=E9

On Mon, Nov 4, = 2013 at 2:20 PM, Salman Toor <salman.toor@it.uu.se> = wrote:

Hi,


Ok so 2.x is = not a new version its another branch. Good to know! = Actually

32bit will be = difficult as the code I got have already some dependencies = on

64 bit.


Otherwise I = will continue with 1.x version. Can you suggest some = version

with 1.x series = which is stable and work on the cluster = environment?

especially with = C++ ...


Regards..

Salman.


Salman Toor, = PhD

salman.toor@it.uu.se




On Nov 4, 2013, = at 1:54 PM, Amr Shahin wrote:


Well, the 2 = series isn't exactly the "next version". It's a continuation = of

branch .2.

Also, the error = message from the gcc indicates that the library = you're

trying to link = to isn't compatible which made me suspicious. check = the

documentation to see if hadoop has 64 libraries, or = otherwise compile

against the 32 = ones



On Mon, Nov 4, = 2013 at 4:51 PM, Salman Toor <salman.toor@it.uu.se> = wrote:


Hi,


Thanks for your = answer!


But are you = sure about it? Actually Hadoop version 1.2 have both 32 and = 64

bit libraries so I believe the the next version should = have both... But I am

not sure just a = random guess :-(


Regards..

Salman.


Salman Toor, = PhD

salman.toor@it.uu.se




On Nov 4, 2013, = at 1:38 PM, Amr Shahin wrote:


I believe = hadoop isn't compatible with 64 architecture. Try installing = the

32 libraries and compile against = them.

This error = (skipping incompatible

/home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when = searching

-lhadooppipes) = indicates so



On Mon, Nov 4, = 2013 at 2:44 PM, Salman Toor <salman.toor@it.uu.se> = wrote:


Hi,


Can someone = give a pointer?



Thanks in = advance.


Regards..

Salman.



Salman Toor, = PhD

salman.toor@it.uu.se




On Nov 3, 2013, = at 11:31 PM, Salman Toor wrote:


Hi,


I am quite new = to the Hadoop world, previously was running = hadoop-1.2.0

stable version = on my small cluster and encountered some strange = problems

like the local = path to the mapper  file didn't copy to the hdfs .... = It

works fine on the single node setup but on multiple node = simple word-count

python example = didn't work...  I read on the blog that it might be = the

problem in the version I am using. So I thought to change = the version and

downloaded the = Hadoop 2.2.0. This version has yarn together with many = new

features that I hope I will learn in the future. Now = simple wordcount

example works = without any problem on the multi-node setup. I am using = simple

python = example.


Now I would = like to compile my C++ code. Since the directory = structure

together with = other things have been changed. I have started to get = the

following error:


----------------

/urs/bin/ld: =  skipping incompatible

/home/sztoor/hadoop-2.2.0/lib/native/libhadooputils.a when = searching

-lhadooputils

cannot find = -lhadooputils


/urs/bin/ld: =  skipping incompatible

/home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when = searching

-lhadooppipes

cannot find = -lhadooppipes

------------------


I have managed = to run the c++ example successfully with the 1.2.0 = version

on single node = setup.


I am having = 64bit Ubuntu machine. previously I was using = Linux-amd64-64


Now in new = version, "lib" and "include" directories are in = the

hadoop-2.2.0 directory. No build.xml is = available...


Can someone = please give me an example of a makefile based on the = version

2.2.0? Or = suggest me which version I should go for? Or if there are = some

prerequisites = that I should do before compiling my code?


Thanks in = advance.


Regards..

Salman.




Salman Toor, = PhD

salman.toor@it.uu.se












--
Andr=E9 = Kelpe
andre@concurrentinc.com
http://concurrentinc.com





--
Andr=E9 Kelpe
andre@concurrentinc.com
htt= p://concurrentinc.com

= --Apple-Mail=_52501C75-C759-40D5-AD1A-74E8A62A8673--