giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Reisman <apache.mail...@gmail.com>
Subject Re: [jira] [Commented] (GIRAPH-13) Port Giraph to YARN
Date Mon, 04 Mar 2013 22:15:21 GMT
I think I will use Yarn MiniCluster to verify my AM comes up, and maybe
some no-op 1 node job. I will not test any BSP code as that is all running
against the MRv1 interface still (I didn't have to change or munge anything
but GiraphRunner!) so those tests verify that BSP works.

The cool thing about our implementation is that it will run on a local YARN
setup on one machine with any division of labor and heap you feel like. So
if the cluster has 6GB available, you can run Giraph on a bit less than
that (each app uses some heap for YARN management stuff) as 5 1GB nodes (1
master 4 workers) or 2 2GB (1

On Mon, Mar 4, 2013 at 9:35 AM, Hyunsik Choi (JIRA) <jira@apache.org> wrote:

>
>     [
> https://issues.apache.org/jira/browse/GIRAPH-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13592389#comment-13592389]
>
> Hyunsik Choi commented on GIRAPH-13:
> ------------------------------------
>
> Eli,
>
> This patch looks like a well-written reference code. Actually, I have
> learned good usage of Yarn from your patch. I'm looking forward to the
> complete work.
>
> How about the plan for unit test? Are you planning to use MiniYarnCluster
> for integration test?
>
> > Port Giraph to YARN
> > -------------------
> >
> >                 Key: GIRAPH-13
> >                 URL: https://issues.apache.org/jira/browse/GIRAPH-13
> >             Project: Giraph
> >          Issue Type: New Feature
> >            Reporter: Jakob Homan
> >            Assignee: Eli Reisman
> >         Attachments: GIRAPH-13-1.patch, GIRAPH-13-2.patch,
> GIRAPH-13-3.patch, GIRAPH-13-4.patch, GIRAPH-13-5.patch, GIRAPH-13-6.patch,
> GIRAPH-13-7.patch, GIRAPH-13-8.patch
> >
> >
> > Now that YARN (aka MR2 aka MAPREDUCE-279) has been merged into the
> Hadoop trunk, we should think about what it would take to separate out the
> graph processing bits of Giraph from the MR1-specific code so as to take
> advantage of the less-MR centric aspects of YARN, while still supporting
> both over the medium term.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message