giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Reisman (JIRA)" <j...@apache.org>
Subject [jira] [Created] (GIRAPH-572) The o.a.g.yarn package could be the top-level of a source tree of packages that miorrors core
Date Sat, 16 Mar 2013 19:54:12 GMT
Eli Reisman created GIRAPH-572:
----------------------------------

             Summary: The o.a.g.yarn package could be the top-level of a source tree of packages
that miorrors core
                 Key: GIRAPH-572
                 URL: https://issues.apache.org/jira/browse/GIRAPH-572
             Project: Giraph
          Issue Type: Improvement
    Affects Versions: 0.2.0
            Reporter: Eli Reisman


This might be a bad idea. But here goes:

There are possibilities to move all sorts of functionality out of the Giraph/BSP parts of
the code and into the YARN AppMaster, or into separately-managed containers launched from
the AppMaster.

For each functionality we decide to re-implement in YARN, it will need to live in the yarn
package tree to be selectively compiled and to use YARN-only imports.


One possibility to begin doing this is to use GIRAPH-13's Configuration#isPureYarnJob. We
will use the isPureYarnJob in Giraph to selectively "no-op" each functionality we replace.
Then, we re-implement the YARN way in our yarn package tree.

If we do this, we should begin early by mirroring the core source tree in subpackages of yarn.
So if we moved a functionality out of o.a.g.graph package we would reimplement it in o.a.g.yarn.graph
package.

I don't suggest doing it all at once, but as we add files to o.a.g.yarn, just to get the idea
out there before the files start to pile up. Anything that uses YARN imports will have to
choose between munge flags and being in the o.a.g.yarn package, one way or another.

If we don't like this idea, mark it won't fix. I'm not attached to it, just an idea.



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message