giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milinda Pathirage <mpath...@umail.iu.edu>
Subject Re: Giraph & yarn build
Date Mon, 07 Oct 2013 02:01:35 GMT
Yep, this is a bug in the code. Adding InterruptedException to method
signature in HaltApplicationUtils will fix that problem. then there
will be some checkstyle errors and findbug errors. You need to add
following exclusions to findbugs-exclude.xml in root directory of
giraph source.

<Match>
    <Class name="org.apache.giraph.yarn.GiraphYarnTask$OverrideExceptionHandler"/>
    <Bug pattern="DM_EXIT"/>
  </Match>
  <Match>
    <Class name="org.apache.giraph.yarn.YarnUtils"/>
    <Bug pattern="RV_RETURN_VALUE_IGNORED_BAD_PRACTICE"/>
  </Match>

Also skip checkstyle checks using -Dcheckstyle.skip=true system
property. I was able to get hadoop_yarn profile built following above
steps.

Thanks
Milinda

On Sun, Oct 6, 2013 at 9:19 PM, Matthew Laird <lairdm@sfu.ca> wrote:
> Thanks for the reply, so I just grabbed that I assume is the latest version,
> the snapshot link from git:
>
> https://git-wip-us.apache.org/repos/asf?p=giraph.git;a=tree;h=refs/heads/trunk;hb=trunk
>
> And ran the command:
>
> mvn -Dhadoop.version=2.0.5-alpha -Phadoop_yarn clean install
>
> The error I receive in this situation is:
>
> [INFO] 2 errors
> [INFO] -------------------------------------------------------------
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Giraph Parent .............................. SUCCESS [1.593s]
> [INFO] Apache Giraph Core ................................ FAILURE [23.522s]
> [INFO] Apache Giraph Examples ............................ SKIPPED
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 25.574s
> [INFO] Finished at: Sun Oct 06 18:17:21 PDT 2013
> [INFO] Final Memory: 37M/692M
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile)
> on project giraph-core: Compilation failure: Compilation failure:
> [ERROR]
> /opt/giraph/giraph-core/target/munged/main/org/apache/giraph/job/HaltApplicationUtils.java:[78,49]
> unreported exception java.lang.InterruptedException; must be caught or
> declared to be thrown
> [ERROR]
> /opt/giraph/giraph-core/target/munged/main/org/apache/giraph/job/HaltApplicationUtils.java:[81,50]
> unreported exception java.lang.InterruptedException; must be caught or
> declared to be thrown
>
> That looks like a bug in the code?
>
> Thanks.
>
>
> On 06/10/13 18:13, Milinda Pathirage wrote:
>>
>> AFAIK, you need to specify hadoop.version system property. So command
>> to build giraph with YARN (2.0.5) should be something like following.
>>
>> mvn -Phadoop_yarn -Dhadoop.version=2.0.5-alpha clean install
>>
>> Hope this helps.
>>
>> Thanks
>> Milinda
>>
>>
>> On Sun, Oct 6, 2013 at 9:04 PM, Matthew Laird <lairdm@sfu.ca> wrote:
>>>
>>> Further, I let it go and after two hours I see the file:
>>>
>>>
>>> /giraph-core/target/munged/main/org/apache/giraph/yarn/GiraphYarnTask.java.rej
>>>
>>> is now 295GB in size.  I'm not sure what it's stuck doing after applying
>>> this patch...
>>>
>>> Thanks.
>>>
>>>
>>> On 06/10/13 16:13, Matthew Laird wrote:
>>>>
>>>> Hello,
>>>>
>>>> I'm trying to build giraph against Hadoop 2.0.5, and I've been browsing
>>>> the mailing list archives but am still having an issue. I found the
>>>> previous
>>>> thread where Eli points to
>>>>
>>>> https://issues.apache.org/jira/browse/GIRAPH-688
>>>>
>>>> I've grabbed the patch and applied it against a clone from git
>>>> (http://git-wip-us.apache.org/repos/asf/giraph.git, is that the correct
>>>> place to get the latest trunk?)  Just like the other fellow asking a few
>>>> months ago, patch complains the patch has been applied before.
>>>>
>>>> Once applied I run the suggested build command:
>>>>
>>>> mvn -U -Phadoop_yarn clean install
>>>>
>>>> It reaches
>>>>
>>>> [INFO] --- munge-maven-plugin:1.0:munge (munge) @ giraph-core ---
>>>>
>>>> And so far it's been sitting there about 20 minutes at this step, in
>>>> another window I can see it's working hard doing something, but I'm
>>>> assuming
>>>> it's stuck doing... I'm not sure what.
>>>>
>>>> Without the patch I receive the same error the other fellow was having
>>>> about dependancies for hadoop not being satisfied.
>>>>
>>>> Any advice on what might be going on, or is there a better place to get
>>>> a
>>>> snapshot that has the needed patches applied? (I see a lot of commits
>>>> since
>>>> the original thread)
>>>>
>>>> Thanks!
>>>>
>>
>>
>



-- 
Milinda Pathirage

twitter: milindalakmal
skype: milinda.pathirage
blog: http://milinda.pathirage.org

Mime
View raw message