giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Reisman (JIRA)" <>
Subject [jira] [Commented] (GIRAPH-13) Port Giraph to YARN
Date Sun, 06 Jan 2013 21:46:16 GMT


Eli Reisman commented on GIRAPH-13:

Jakob has handed this off to me for the time being, and I'm going to be logging some hours
on this very soon. The idea is a "Pure YARN" implementation so that we can submit our resource
needs and Giraph code to the cluster without doing anything MR specific. The idea is to make
this behavior a pluggable option so we can run on MRv2, YARN, or eventually any cluster framework
we like rather than lumping all the MRv1 behavior into GiraphMapper/GiraphJob/GiraphRunner
with the more generic Giraph behaviors.

As it stands, when you set up a MRv2 cluster on YARN, compile Giraph as 'mvn -Phadoop_2.0.2
package' (for instance) you can run Giraph on MRv2-enabled YARN cluster such as Hadoop-2.0.2-alpha
right now. This still utilizes the existing Hadoop/MR mapper-centric API but works just fine
for me so far. So yes, Giraph is fully functional on YARN clusters but still depends on MapReduce
as it stands today.

> Port Giraph to YARN
> -------------------
>                 Key: GIRAPH-13
>                 URL:
>             Project: Giraph
>          Issue Type: New Feature
>            Reporter: Jakob Homan
>            Assignee: Eli Reisman
> Now that YARN (aka MR2 aka MAPREDUCE-279) has been merged into the Hadoop trunk, we should
think about what it would take to separate out the graph processing bits of Giraph from the
MR1-specific code so as to take advantage of the less-MR centric aspects of YARN, while still
supporting both over the medium term.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message