giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Laird <lai...@sfu.ca>
Subject Re: Giraph & yarn build
Date Mon, 07 Oct 2013 02:04:53 GMT
Hmm, is there a recommendation on a particular version which might be 
good for building/testing against Hadoop 2.0.5?  I realize I'm playing 
with the bleeding edge here...

I threw a try/catch around   public static void printHaltInfo() just 
like private static boolean waitForHaltInfo() has.  However next I had 
issues with the tests, I'm guessing because I'm running the hadoop head 
node on this machine, it was a socket bind error, so I threw in a 
-DskipTests.

Next I ran in to a org.apache.rat issue, I added an exclude for *.txt, 
*.xml, and *.py.  Unfortunately next was:

[INFO] --- findbugs-maven-plugin:2.5.1:check (default) @ giraph-core ---
[INFO] BugInstance size is 2
[INFO] Error size is 0
[INFO] Total bugs: 2
[INFO] 
org.apache.giraph.yarn.GiraphYarnTask$OverrideExceptionHandler.uncaughtException(Thread, 
Throwable) invokes System.exit(...), which shuts down the entire virtual 
machine 
["org.apache.giraph.yarn.GiraphYarnTask$OverrideExceptionHandler"] At 
GiraphYarnTask.java:[lines 171-178]
[INFO] exceptional return value of java.io.File.delete() ignored in 
org.apache.giraph.yarn.YarnUtils.exportGiraphConfiguration(GiraphConfiguration, 
ApplicationId) ["org.apache.giraph.yarn.YarnUtils"] At 
YarnUtils.java:[lines 47-240]

I guess this is what I get for playing with the trunk. :)  But, what 
version was it that is mentioned in the previous mailing list threads 
that you can apply the patch to and get it compiled?

Thanks!

On 06/10/13 18:19, Matthew Laird wrote:
> Thanks for the reply, so I just grabbed that I assume is the latest 
> version, the snapshot link from git:
>
> https://git-wip-us.apache.org/repos/asf?p=giraph.git;a=tree;h=refs/heads/trunk;hb=trunk

>
>
> And ran the command:
>
> mvn -Dhadoop.version=2.0.5-alpha -Phadoop_yarn clean install
>
> The error I receive in this situation is:
>
> [INFO] 2 errors
> [INFO] -------------------------------------------------------------
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Giraph Parent .............................. SUCCESS 
> [1.593s]
> [INFO] Apache Giraph Core ................................ FAILURE 
> [23.522s]
> [INFO] Apache Giraph Examples ............................ SKIPPED
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] Total time: 25.574s
> [INFO] Finished at: Sun Oct 06 18:17:21 PDT 2013
> [INFO] Final Memory: 37M/692M
> [INFO] 
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.0:compile 
> (default-compile) on project giraph-core: Compilation failure: 
> Compilation failure:
> [ERROR] 
> /opt/giraph/giraph-core/target/munged/main/org/apache/giraph/job/HaltApplicationUtils.java:[78,49]

> unreported exception java.lang.InterruptedException; must be caught or 
> declared to be thrown
> [ERROR] 
> /opt/giraph/giraph-core/target/munged/main/org/apache/giraph/job/HaltApplicationUtils.java:[81,50]

> unreported exception java.lang.InterruptedException; must be caught or 
> declared to be thrown
>
> That looks like a bug in the code?
>
> Thanks.
>
> On 06/10/13 18:13, Milinda Pathirage wrote:
>> AFAIK, you need to specify hadoop.version system property. So command
>> to build giraph with YARN (2.0.5) should be something like following.
>>
>> mvn -Phadoop_yarn -Dhadoop.version=2.0.5-alpha clean install
>>
>> Hope this helps.
>>
>> Thanks
>> Milinda
>>
>>
>> On Sun, Oct 6, 2013 at 9:04 PM, Matthew Laird <lairdm@sfu.ca> wrote:
>>> Further, I let it go and after two hours I see the file:
>>>
>>> /giraph-core/target/munged/main/org/apache/giraph/yarn/GiraphYarnTask.java.rej

>>>
>>>
>>> is now 295GB in size.  I'm not sure what it's stuck doing after 
>>> applying
>>> this patch...
>>>
>>> Thanks.
>>>
>>>
>>> On 06/10/13 16:13, Matthew Laird wrote:
>>>> Hello,
>>>>
>>>> I'm trying to build giraph against Hadoop 2.0.5, and I've been 
>>>> browsing
>>>> the mailing list archives but am still having an issue. I found the 
>>>> previous
>>>> thread where Eli points to
>>>>
>>>> https://issues.apache.org/jira/browse/GIRAPH-688
>>>>
>>>> I've grabbed the patch and applied it against a clone from git
>>>> (http://git-wip-us.apache.org/repos/asf/giraph.git, is that the 
>>>> correct
>>>> place to get the latest trunk?)  Just like the other fellow asking 
>>>> a few
>>>> months ago, patch complains the patch has been applied before.
>>>>
>>>> Once applied I run the suggested build command:
>>>>
>>>> mvn -U -Phadoop_yarn clean install
>>>>
>>>> It reaches
>>>>
>>>> [INFO] --- munge-maven-plugin:1.0:munge (munge) @ giraph-core ---
>>>>
>>>> And so far it's been sitting there about 20 minutes at this step, in
>>>> another window I can see it's working hard doing something, but I'm 
>>>> assuming
>>>> it's stuck doing... I'm not sure what.
>>>>
>>>> Without the patch I receive the same error the other fellow was having
>>>> about dependancies for hadoop not being satisfied.
>>>>
>>>> Any advice on what might be going on, or is there a better place to 
>>>> get a
>>>> snapshot that has the needed patches applied? (I see a lot of 
>>>> commits since
>>>> the original thread)
>>>>
>>>> Thanks!
>>>>
>>
>>
>


Mime
View raw message