hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <saint....@gmail.com>
Subject Re: Proposed feature branch for HBase security
Date Sat, 03 Jul 2010 01:44:23 GMT
+1 on branch integrating security. If I can help just say.  

On b.) I am game moving to nio avro but not of it gets in the way of a.). 



On Jul 2, 2010, at 19:15, Gary Helmling <ghelmling@gmail.com> wrote:

> Hi folks,
> 
> With Yahoo's latest security release on github (
> http://github.com/yahoo/hadoop-common/tree/yahoo-hadoop-0.20.104), it looks
> like we now have a real-world usable version of secure Hadoop, based on
> 0.20.  This is exciting stuff, because now we have something solid to start
> working towards implementing similar security controls in HBase (HBASE-1697,
> HBASE-2014, HBASE-2016, HBASE-2420)!
> 
> However, this is going to be a large undertaking, with a strong dependency
> on the secure Hadoop branch (more on that in a bit --  unfortunately the
> fragmented hadoop-0.20 world is already leaking through).  So I'd like to
> propose a feature branch in the HBase svn repo for security work, to:
> 
> 1) ensure that changes towards implementing secure HBase have an ASF home
> 2) provide more visibility and granularity for review (esp. JIRA &
> reviewboard usage)
> 3) ease interaction/integration with other branched changes underway (master
> rewrite)
> 
> I've already started pushing some preliminary changes up to github (
> http://github.com/ghelmling/hbase/tree/security), and will continue to do
> so, but I'd like to avoid both massive patch sets accumulating too many
> changes and making interested committers & contributors go digging to see
> what the current state is.
> 
> On the secure Hadoop branch dependency -- I've integrated the
> org.apache.hadoop.ipc changes into o.a.h.hbase.ipc.* (HBASE-2742) and run
> into a couple complications:
> 
> * Hadoop RPC version rolled from 3 to 4 (apparently 0.20-append also does
> this!)
> * various bits in the updated HBaseClient, HBaseServer, etc. now depend on
> the security implementation, so building and running on top of non-secure
> Hadoop will not be possible.
> 
> I'd like to post the diff on review.hbase.org for more review and feedback,
> but that begs the question of where the changes should go?
> 
> Longer term, I think we need to dump Hadoop RPC (AVRO-405 seems promising in
> this) so that HBase internals aren't so intertwined with Hadoop
> implementation details, but that's it's own large scale project which we
> shouldn't couple to security.
> 
> So, to sum up, thoughts on:
> 
> a) creating a "security" feature branch in svn?
> b) RPC related changes, specifically cross Hadoop branch incompatibility due
> to version increment and Hadoop security dependencies?
> 
> Thanks,
> Gary

Mime
View raw message