giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Reisman (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (GIRAPH-13) Port Giraph to YARN
Date Fri, 08 Mar 2013 01:34:14 GMT

     [ https://issues.apache.org/jira/browse/GIRAPH-13?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eli Reisman updated GIRAPH-13:
------------------------------

    Attachment: GIRAPH-13-9.patch

This is an update, sorry I have waited to post it so long. Works like a charm, only 2 issues
left:

1. finish writing the integration test (partly included in patch here)

2. debug strange latency in launching job during YARN setup. This is why I have posted on
review board a bit early (I'll ping you guys when its ready for review.) My non-Giraphing
YARN expert colleagues are going to take a peek on RB and tell me what I did to slow the plumbing
down.

3. Remove the hardcoded convenience dependency on giraph-examples jar with deps. The idea
will be to include your own CSV list of jars to include in the job under the new -yj command
line opt.

Thanks! The RB link is:
https://reviews.apache.org/r/9811/diff/#index_header
                
> Port Giraph to YARN
> -------------------
>
>                 Key: GIRAPH-13
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-13
>             Project: Giraph
>          Issue Type: New Feature
>            Reporter: Jakob Homan
>            Assignee: Eli Reisman
>         Attachments: GIRAPH-13-1.patch, GIRAPH-13-2.patch, GIRAPH-13-3.patch, GIRAPH-13-4.patch,
GIRAPH-13-5.patch, GIRAPH-13-6.patch, GIRAPH-13-7.patch, GIRAPH-13-8.patch, GIRAPH-13-9.patch
>
>
> Now that YARN (aka MR2 aka MAPREDUCE-279) has been merged into the Hadoop trunk, we should
think about what it would take to separate out the graph processing bits of Giraph from the
MR1-specific code so as to take advantage of the less-MR centric aspects of YARN, while still
supporting both over the medium term.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message