incubator-giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyunsik Choi <hyun...@apache.org>
Subject Re: hadoop version profiles
Date Wed, 21 Mar 2012 10:46:17 GMT
This approach looks good.
+1

On Wed, Mar 21, 2012 at 3:53 PM, Avery Ching <aching@apache.org> wrote:

> I agree with this approach, although munge is kinda hacky.  It is easy
> though. =)
>
> Avery
>
>
> On 3/20/12 5:52 PM, Eugene Koontz wrote:
>
>> Hi Giraphers,
>>    I think it might be good to look at how we can add support for new
>> hadoop versions. Currently we have hadoop_facebook (
>> https://issues.apache.org/**jira/browse/GIRAPH-14<https://issues.apache.org/jira/browse/GIRAPH-14>).
>> I am considering adding new ones such as hadoop_0.24. Looking at the code,
>> it seems that the main hadoop variation between the stock hadoop used
>> (0.203.0) versus facebook has to do with the new security-related APIs in
>> the latter that is, fortunately, also available in hadoop 0.23 and 0.24.
>> So, hopefully we can make use of the existing work that Avery has done for
>> hadoop_facebook and apply it to other hadoop versions. Therefore I would
>> propose that:
>>
>> 1. a new munge flag HADOOP_SECURE to be used in RPCCommunication.java and
>> a few other places, where we are currently checking for HADOOP_FACEBOOK and
>> HADOOP.
>> 2. we make a new profile called hadoop_secure, which, as with
>> hadoop_facebook, will use the above munge flag.
>> 3. we make a new profile "hadoop_0.20.203" for the existing default
>> hadoop and make it the default profile (activeByDefault=true). This will
>> makes it easier to handle the differences in the hadoop library dependency
>> set that have happened between 0.20.203 and hadoop trunk.
>>
>> Please see https://github.com/ekoontz/**giraph/tree/security-profile<https://github.com/ekoontz/giraph/tree/security-profile>for
my branch that implements the above.
>>
>> Thanks,
>>
>> -Eugene
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message