giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Reisman" <initialcont...@gmail.com>
Subject Re: Review Request 14575: Giraph Application Master: move to new and stable YARN API
Date Fri, 11 Oct 2013 01:52:42 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14575/#review26918
-----------------------------------------------------------

Ship it!


I'm +1 for moving to the new YARN APIs. I think with Hadoop 2 on the beta line now, supporting
alpha is no longer such a priority.

- Eli Reisman


On Oct. 10, 2013, 7:50 a.m., Mohammad Islam wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14575/
> -----------------------------------------------------------
> 
> (Updated Oct. 10, 2013, 7:50 a.m.)
> 
> 
> Review request for giraph.
> 
> 
> Bugs: GIRAPH-737
>     https://issues.apache.org/jira/browse/GIRAPH-737
> 
> 
> Repository: giraph-git
> 
> 
> Description
> -------
> 
> WIP patch. Please give your early comments.
> 
> Key points:
> 
> * Giraph AM using new API and asynchronous/handler model.
> * Adding Kerberos support.
> 
> Copied from the JIRA:
> 
> Giraph was the early adopter of Hadoop YARN AM! Eli successfully wrote a Giraph AM based
on Hadoop 2.0.x_alpha. However, in last few months, Yarn significantly overhauled its APIs
and associated coding patterns. The new beta version is 2.1.x and I was told by Yarn-dev that
current APIs will not change much.
> In the above circumstances, we need to substantially overhaul Giraph AM as well to accommodate
with the new Yarn API. Moreover, in newer YARN API, supporting kerberos security in AM becomes
easier and more transparent.
> Potential impact:
> The upcoming Girpah AM will not work with earlier alpha Hadoop versions such as 2.0.3.
I'm not sure if anyone is using Giraph AM in production. However, the more prevalent way of
Giraph processing (MR-based) should continue to work.
> 
> 
> Diffs
> -----
> 
>   giraph-core/src/main/java/org/apache/giraph/comm/netty/SaslNettyServer.java 00a802f

>   giraph-core/src/main/java/org/apache/giraph/comm/netty/handler/SaslServerHandler.java
922f373 
>   giraph-core/src/main/java/org/apache/giraph/yarn/GiraphApplicationMaster.java c2b88a0

>   giraph-core/src/main/java/org/apache/giraph/yarn/GiraphYarnClient.java 341db0e 
>   giraph-core/src/main/java/org/apache/giraph/yarn/YarnUtils.java aa042e8 
>   giraph-hive/src/main/java/org/apache/giraph/hive/output/package-info.java 65d87e3 
>   giraph-hive/src/test/java/org/apache/giraph/hive/input/package-info.java c2327ca 
>   pom.xml 41b6bb1 
> 
> Diff: https://reviews.apache.org/r/14575/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Mohammad Islam
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message