giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Reisman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-13) Port Giraph to YARN
Date Sun, 06 Jan 2013 21:46:16 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13545504#comment-13545504
] 

Eli Reisman commented on GIRAPH-13:
-----------------------------------

Jakob has handed this off to me for the time being, and I'm going to be logging some hours
on this very soon. The idea is a "Pure YARN" implementation so that we can submit our resource
needs and Giraph code to the cluster without doing anything MR specific. The idea is to make
this behavior a pluggable option so we can run on MRv2, YARN, or eventually any cluster framework
we like rather than lumping all the MRv1 behavior into GiraphMapper/GiraphJob/GiraphRunner
with the more generic Giraph behaviors.

As it stands, when you set up a MRv2 cluster on YARN, compile Giraph as 'mvn -Phadoop_2.0.2
package' (for instance) you can run Giraph on MRv2-enabled YARN cluster such as Hadoop-2.0.2-alpha
right now. This still utilizes the existing Hadoop/MR mapper-centric API but works just fine
for me so far. So yes, Giraph is fully functional on YARN clusters but still depends on MapReduce
as it stands today.

                
> Port Giraph to YARN
> -------------------
>
>                 Key: GIRAPH-13
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-13
>             Project: Giraph
>          Issue Type: New Feature
>            Reporter: Jakob Homan
>            Assignee: Eli Reisman
>
> Now that YARN (aka MR2 aka MAPREDUCE-279) has been merged into the Hadoop trunk, we should
think about what it would take to separate out the graph processing bits of Giraph from the
MR1-specific code so as to take advantage of the less-MR centric aspects of YARN, while still
supporting both over the medium term.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message