Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D2FD110951 for ; Thu, 5 Dec 2013 23:59:07 +0000 (UTC) Received: (qmail 94461 invoked by uid 500); 5 Dec 2013 23:59:07 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 94426 invoked by uid 500); 5 Dec 2013 23:59:07 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 94418 invoked by uid 99); 5 Dec 2013 23:59:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 05 Dec 2013 23:59:07 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of rohitbhat.nitk@gmail.com designates 209.85.223.173 as permitted sender) Received: from [209.85.223.173] (HELO mail-ie0-f173.google.com) (209.85.223.173) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 05 Dec 2013 23:58:59 +0000 Received: by mail-ie0-f173.google.com with SMTP id to1so30831500ieb.18 for ; Thu, 05 Dec 2013 15:58:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=1xpVKCV3j3qge0u+kGI/u1XMMb8iIHx2KvRII1DMqBg=; b=xUWAHQqoZthNzkh5lo7WSqszWM91+EMVIHbDb3SvMVEYdFrX6B2i6YAEFgPbLkN7Sx OrvIE7v9X1/2bnJbmqJ9RMX8ipl+Qpn4jNW0tKx+Wlcs29OLvx+5qe7ql+2eetTYb0j8 c8iTkIwpEazd1XP4dK3/BWA2fGVVH5PhdhKkeF0l54E2IrYv2ITocuBC4DntTSt7XGTO 0x4+UR0sG0POIIXnumRrbrOBSFnzPHHHI1Ft9aas9hk29wvEb/Lubrk6rOoiPlO6Zc5V xSqXlz0cbtVucVaHa0JvDBICpQ4ZrnF1761glhWxYWx31tC5GFwIpH/3n+8SZLgGODvu NJpg== MIME-Version: 1.0 X-Received: by 10.50.45.73 with SMTP id k9mr437004igm.38.1386287918193; Thu, 05 Dec 2013 15:58:38 -0800 (PST) Received: by 10.64.14.148 with HTTP; Thu, 5 Dec 2013 15:58:38 -0800 (PST) In-Reply-To: References: Date: Thu, 5 Dec 2013 17:58:38 -0600 Message-ID: Subject: Re: Issue while executing SimpleShortestPathsComputation From: Rohit Bhat To: user@giraph.apache.org Content-Type: multipart/alternative; boundary=089e010d9e72eba61004ecd24e03 X-Virus-Checked: Checked by ClamAV on apache.org --089e010d9e72eba61004ecd24e03 Content-Type: text/plain; charset=ISO-8859-1 Can someone please confirm if giraph 1.0.0 works with hadoop-2.2.0? On Thu, Dec 5, 2013 at 5:58 PM, Rohit Bhat wrote: > I am using giraph 1.0.0 but with yarn (hadoop 2.2.0) > > > On Thu, Dec 5, 2013 at 5:49 PM, Yong Guo wrote: > >> Hi, >> >> I think it might be the Giraph version problem. When I tried to build >> giraph*1.0.0* on hadoop2.0.3-alpha, I can compile successfully by "*mvn >> -Phadoop_yarn -Dhadoop.version=2.0.3-alpha clean package -DskipTests*". >> >> The command failed when I used giraph*1.1.0*. >> >> Thanks, >> Yong >> >> >> >> 2013/12/6 Yong Guo >> >>> Hi, >>> >>> I also got this error when tried to build Giraph on top of hadoop-2.2.0 >>> using similar commands. Can anyone help? >>> >>> Thanks, >>> Yong >>> >>> >>> 2013/12/6 Rohit Bhat >>> >>>> I already have yarn (hadoop_2.2.0) on my machine. I tried the following >>>> 2 commands and got similar errors in both the cases: >>>> >>>> 1. "mvn -Phadoop_yarn -Dhadoop.version=2.2.0 -X clean package - >>>> DskipTests" >>>> >>>> 2. "mvn -P2.2.0 -DskipTests -Dmaven.javadoc.skip=true clean install >>>> package" >>>> >>>> >>>> [INFO] Checking dependency versions >>>> >>>> [ERROR] Found a problem with the direct dependency >>>> commons-httpclient:commons-httpclient of the current project >>>> >>>> Expected version is 3.0.1 >>>> >>>> Resolved version is 3.0.1 >>>> >>>> Version 3.1 was expected by artifact: org.apache.hadoop:hadoop-common >>>> >>>> >>>> [ERROR] Found a problem with the dependency commons-lang:commons-lang >>>> >>>> Resolved version is 2.4 >>>> >>>> Version 2.4 was expected by artifact: >>>> commons-configuration:commons-configuration >>>> >>>> Version 2.5 was expected by artifact: org.apache.hadoop:hadoop-common >>>> >>>> >>>> [ERROR] Found a problem with the dependency >>>> commons-logging:commons-logging >>>> >>>> Resolved version is 1.0.4 >>>> >>>> Version 1.1.1 was expected by artifact: org.apache. >>>> hadoop:hadoop-common >>>> >>>> [ERROR] Found a problem with the dependency io.netty:netty >>>> >>>> Resolved version is 3.5.3.Final >>>> >>>> Version 3.6.2.Final was expected by artifacts: org.apache. >>>> hadoop:hadoop-mapreduce-client-common, org.apache. >>>> hadoop:hadoop-mapreduce-client-core >>>> >>>> [ERROR] Found a problem with the dependency org.apache. >>>> zookeeper:zookeeper >>>> >>>> Resolved version is 3.3.3 >>>> >>>> Version 3.4.5 was expected by artifacts: org.apache. >>>> hadoop:hadoop-common, org.apache.hadoop:hadoop-mapreduce-client-common >>>> >>>> >>>> Am i missing something? >>>> >>>> >>>> On Mon, Dec 2, 2013 at 7:21 PM, Ameya Vilankar < >>>> ameya.vilankar@gmail.com> wrote: >>>> >>>>> Yes. Use this: >>>>> >>>>> mvn -Phadoop_cdh4.4.0 -DskipTests -Dmaven.javadoc.skip=true package >>>>> >>>>> >>>>> On Mon, Dec 2, 2013 at 5:18 PM, Manuel Lagang wrote: >>>>> >>>>>> I get the same error when I compile Giraph against the default hadoop >>>>>> version (0.20.203.0), but my project that uses Giraph uses a more >>>>>> recent hadoop version. Did you set the hadoop version via a maven profile >>>>>> when compiling Giraph (e.g. mvn -Phadoop_1.0 compile for hadoop 1.0)? >>>>>> >>>>>> >>>>>> Presumably, there's a way to compile Giraph for Yarn. There's even a >>>>>> Yarn profile in the pom.xml. I personally haven't had success building it >>>>>> though. >>>>>> >>>>>> >>>>>> On Mon, Dec 2, 2013 at 4:30 PM, Rohit Bhat wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I am new to Giraph. I have installed Yarn (hadoop 2.2.0) and have >>>>>>> installed Giraph on top of it. I have been trying to >>>>>>> execute SimpleShortestPathsComputation example but i am getting the error: >>>>>>> >>>>>>> "java.lang.IllegalArgumentException: >>>>>>> checkLocalJobRunnerConfiguration: When using LocalJobRunner, you cannot run >>>>>>> in split master / worker mode since there is only 1 task at a time!" >>>>>>> >>>>>>> I fixed this by adding an entry in mapred-site.xml for >>>>>>> mapreduce.jobtracker.address but now i am getting a different error: >>>>>>> >>>>>>> *Exception in thread "main" java.lang.IncompatibleClassChangeError: >>>>>>> Found interface org.apache.hadoop.mapreduce.JobContext, but class was >>>>>>> expected* >>>>>>> at >>>>>>> org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:43) >>>>>>> at >>>>>>> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:456) >>>>>>> at >>>>>>> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:342) >>>>>>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268) >>>>>>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265) >>>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>>> at javax.security.auth.Subject.doAs(Subject.java:415) >>>>>>> at >>>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491) >>>>>>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265) >>>>>>> at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:250) >>>>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94) >>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >>>>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) >>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>>>> at >>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>>>>>> at >>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:212) >>>>>>> >>>>>>> *Is Giraph compatible with yarn? If yes, am i missing anything?* >>>>>>> >>>>>>> Thanks, >>>>>>> Rohit >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> > --089e010d9e72eba61004ecd24e03 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Can someone please confirm if giraph 1.0.0 works with hado= op-2.2.0?


On Thu, Dec 5, 2013 at 5:58 PM, Rohit Bhat <rohitbhat.nitk@gmail= .com> wrote:
I am using giraph 1.0.0 but= with yarn (hadoop 2.2.0)


On Thu, Dec 5, 2013 at 5:49 PM, Yong Guo= <guoyongnudt@gmail.com> wrote:
Hi,=A0

I= think it might be the Giraph version problem. When I tried to build giraph= 1.0.0 on hadoop2.0.3-alpha, I can compile successfully by "m= vn -Phadoop_yarn -Dhadoop.version=3D2.0.3-alpha clean package -DskipTests".

The command failed when I used giraph1.1.0.

Thanks,
Yong

=


2013/12/6 Yong Guo <guoyongnudt@gmail.com>
Hi,

I al= so got this error when tried to build Giraph on top of hadoop-2.2.0 using s= imilar commands. Can anyone help?

Thanks,
Yong


2013/12/6 Rohit Bhat &= lt;rohitbhat.= nitk@gmail.com>
I already have yarn (hadoop_2.2.0) on my machine. I t= ried the following 2 commands and got similar errors in both the cases:

1.=A0"mvn=A0-Phadoop_yarn=A0-Dhadoop.versi= on= =3D2.2.0 -X clean package -Ds= kipTests"=A0

2.=A0"mvn=A0-P2.2.0 -= DskipTests=A0-Dmaven.javadoc.skip= =3Dt= rue clean install package"=A0


[INFO] Checking dependency ver= sions=A0

[ERROR] Found a proble= m with the direct dependency=A0commons-httpclient:commons-httpclient=A0of the current project<= span style=3D"margin:0px;padding:0px;font-size:11pt;line-height:18px">=A0

=A0 Expected version i= s 3.0.1=A0

=A0 Resolved version i= s 3.0.1=A0

=A0 Version 3.1 was ex= pected by artifact: org.apache.hadoop:hadoop-common=A0

[ERROR] Found a proble= m with the dependency=A0commo= ns-lang:commons-lang=A0

=A0 Resolved version i= s 2.4=A0

=A0 Version 2.4 was ex= pected by artifact:=A0commons= -configuration:commons-configuration=A0

=A0 Version 2.5 was ex= pected by artifact: org.apache.hadoop:hadoop-common=A0

[ERROR] Found a proble= m with the dependency=A0commo= ns-logging:commons-logging=A0

=A0 Resolved version i= s 1.0.4=A0

=A0 Version 1.1.1 was = expected by artifact: org.apache.hadoop:hadoop-common=A0

[ERROR] Found a proble= m with the dependency io.netty:netty=A0

=A0 Resolved version i= s 3.5.3.Final=A0

=A0 Version 3.6.2.Fina= l was expected by artifacts: org.apache.hadoop:hadoop-mapreduce-client-common, org.apache.hadoop:hadoop-mapreduce-client-core= =A0

[ERROR] Found a proble= m with the dependency org.apache.zookeeper:zookeeper=A0

=A0 Resolved version i= s 3.3.3=A0

=A0 Version 3.4.5 was expected= by artifacts: org.apache.had= oop:hadoop-common, org.apache.had= oop:hadoop-mapreduce-client-common=A0


Am= i missing something?



On Mon, Dec 2, 2013 at 7:21 PM, Ameya Vilankar <<= a href=3D"mailto:ameya.vilankar@gmail.com" target=3D"_blank">ameya.vilankar= @gmail.com> wrote:
Yes. Use this:

mvn -Phadoop_cdh4.4.0 -DskipTests -Dmaven.javadoc.skip=3Dtrue = package


On Mon, Dec 2, 2013 at 5:18 PM, Manuel Lagang <manuellagang@gmail.= com> wrote:
I get the same error when I= compile Giraph against the default hadoop version (0.20.203.0), but my project t= hat uses Giraph uses a more recent hadoop version. Did you set the hadoop v= ersion via a maven profile when compiling Giraph (e.g.=A0mvn -Phadoop_1.0 compile= for hadoop 1.0

On Mon, Dec 2, 2013 at 4:30 PM, Rohit Bhat <= span dir=3D"ltr"><rohitbhat.nitk@gmail.com> wrote:
Hi,

=
I am new t= o Giraph. I have installed Yarn (hadoop 2.2.0) and have installed Giraph on= top of it. I have been trying to execute=A0SimpleShortestPathsComputation = example but i am getting the error:

<= div>"java.lang.IllegalArgumentException: checkLocalJobRunnerConfigurat= ion: When using LocalJobRunner, you cannot run in split master / worker mod= e since there is only 1 task at a time!"

I fixed this by adding an entry in mapred-site.xml for mapreduce= .jobtracker.address but now i am getting a different error:

<= font face=3D"arial, sans-serif">Exception in thread "main" jav= a.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapr= educe.JobContext, but class was expected
= at org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutput= Format.java:43)
at org.apache.hadoop.mapreduce.JobSubmi= tter.checkSpecs(JobSubmitter.java:456)
= at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSu= bmitter.java:342)
at org.apache.hadoop.mapreduce.Job$10= .run(Job.java:1268)
= at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
at java.security.AccessController.doPrivileged(Native Method)
= at javax.security.auth.Subject.doAs(Subject.java:415)
<= div> = at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInf= ormation.java:1491)
= at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:250)=
= at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94)<= /div>
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
= at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)=
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)=
= at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethod= AccessorImpl.java:57)
= at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethod= AccessorImpl.java:43)
at java.lang.reflect.Method.invok= e(Method.java:606)
= at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
=

Is Giraph compatible with yarn? If yes, am i missing anything?
=

Thanks,
Rohit







--089e010d9e72eba61004ecd24e03--