giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Reisman (JIRA)" <>
Subject [jira] [Updated] (GIRAPH-13) Port Giraph to YARN
Date Mon, 25 Feb 2013 06:46:16 GMT


Eli Reisman updated GIRAPH-13:

    Attachment: GIRAPH-13-6.patch

Lots of updates in this patch. The YARN and Giraph components are going swimmingly. However,
I have discovered a strange Maven behavior and I thought I would put up another placeholder
patch in case any of you know whats going on here.

The way I have this version of hadoop_yarn profile set up, the compile plugin (by default
on all profiles) does not compile anything in the o.a.g.yarn directory tree (or package) and
this seems to work as expected. But, when I compile using -Phadoop_yarn, I reinclude the yarn
dir from the profile definition in the pom where I declare the profile configuration of the
compiler plugin. This only partially works.

For some reason I cannot fathon as of yet, only a couple of the files in my yarn package compile
and make it into the final build products. This is even though mvn verify (etc) all passes
and classes themselves seem to compile. I thought at first I forgot to declare the package
for all my files, or maybe a problem with my file or something. But nothing
obvious. Just several of my YARN files don't end up in the jar with their companions. Anyone
know what I could be doing wrong here? All the changes to pom files that set this up are included
in the patch for viewing.

If anyone has an insight (or good Maven tricks to try) let me know. Thanks!

> Port Giraph to YARN
> -------------------
>                 Key: GIRAPH-13
>                 URL:
>             Project: Giraph
>          Issue Type: New Feature
>            Reporter: Jakob Homan
>            Assignee: Eli Reisman
>         Attachments: GIRAPH-13-1.patch, GIRAPH-13-2.patch, GIRAPH-13-3.patch, GIRAPH-13-4.patch,
GIRAPH-13-5.patch, GIRAPH-13-6.patch
> Now that YARN (aka MR2 aka MAPREDUCE-279) has been merged into the Hadoop trunk, we should
think about what it would take to separate out the graph processing bits of Giraph from the
MR1-specific code so as to take advantage of the less-MR centric aspects of YARN, while still
supporting both over the medium term.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message