Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
References: 
 <CADcMMgHKyOT9_eOW4dymKf_9roc1-1nt0UGknbxQrsr4Wu+4xg@mail.gmail.com>
 <CADfYSXTpiJivy5XFqGbPxmj3iM8BpA_aq13-xTFF1hTp6WHGtQ@mail.gmail.com>
 <CADY20s4mdE61+qCYCgJcj1N1D+Bt3VRpLEr5dxqQ9y75qsrjFA@mail.gmail.com>
 <CAEGKEUPjjvdSD43sTBwXhC8Y0++ZmFxoiwPzLA-w98Rrb5ddkw@mail.gmail.com>
Message-ID: <1315015577.5129.YahooMailNeo@web65504.mail.ac4.yahoo.com>
Date: Fri, 2 Sep 2011 19:06:17 -0700 (PDT)
From: Andrew Purtell <apurtell@apache.org>
Reply-To: Andrew Purtell <apurtell@apache.org>
Subject: Re: [DISCUSSION] Accumulo, another BigTable clone,
 has shown up on Apache Incubator as a proposal
To: "dev@hbase.apache.org" <dev@hbase.apache.org>
In-Reply-To: 
 <CAEGKEUPjjvdSD43sTBwXhC8Y0++ZmFxoiwPzLA-w98Rrb5ddkw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="0-1767787493-1315015577=:5129"

--0-1767787493-1315015577=:5129
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

> I think there is precedence for competing and/or "duplicate" Apache=0A> p=
rojects, Avro/Thrift and HBase/Cassandra come to mind. =0A=0A=A0=0AThat arg=
ument isn't helping you make your case.=0A=0A=0ABest regards,=0A=0A=0A=A0=
=A0=A0    - Andy=0A=0AProblems worthy of attack prove their worth by hittin=
g back. - Piet Hein (via Tom White)=0A=0A=0A>______________________________=
__=0A>From: Joey Echeverria <joey@cloudera.com>=0A>To: dev@hbase.apache.org=
=0A>Sent: Saturday, September 3, 2011 3:30 AM=0A>Subject: Re: [DISCUSSION] =
Accumulo, another BigTable clone, has shown up on Apache Incubator as a pro=
posal=0A>=0A>To add to what Todd said, I actually worked with those guys fo=
r the=0A>last 3 years and have used Accumulo in production. It's true that =
it=0A>would have been better if they had been able to contribute to HBase=
=0A>rather than go on their own, but it's not easy to contribute to open=0A=
>source, either officially or unofficially when you work at NSA. I=0A>think=
 there is precedence for competing and/or "duplicate" Apache=0A>projects, A=
vro/Thrift and HBase/Cassandra come to mind. I'm mostly=0A>interested in th=
is project setting a precedent for other work at NSA=0A>to be developed as =
open source.=0A>=0A>-Joey=0A>=0A>On Fri, Sep 2, 2011 at 3:09 PM, Todd Lipco=
n <todd@cloudera.com> wrote:=0A>> Hey folks,=0A>>=0A>> <wearing my Todd hat=
 and not my Cloudera hat!>=0A>>=0A>> I've been in touch with this team for =
the last 18 months or so.=0A>> They're good people, smart, and have a healt=
hy respect for HBase and=0A>> our team. Though they haven't contributed cod=
e or participated on the=0A>> lists, I can vouch that they do follow our de=
velopment and generally=0A>> do understand HBase as well as what makes thei=
r system different. In=0A>> the context of the incubator proposal, they're =
trying to explain why=0A>> their system is different than HBase, and not tr=
ying to knock our=0A>> project. They do borrow our ideas, and in the future=
 we'll be able to=0A>> borrow some of theirs. Iterator trees, for example, =
are distinct from=0A>> coprocessors and have some really nice capabilities =
which I'm looking=0A>> forward to adapting into HBase.=0A>>=0A>> There are =
a couple things to keep in mind about the story here:=0A>> - they first eva=
luated HBase 3 years ago. HBase at that point was not=0A>> usable for their=
 application - I think several of us here remember the=0A>> state of HBase =
at the time and might have made the same decision. So,=0A>> they started th=
eir own project with an internal team of 5-6 people.=0A>> - contributing to=
 open source from within the NSA is not easy, for=0A>> obvious reasons. The=
y've jumped through many hoops to open source=0A>> this, and we should be t=
hankful for that. Now that they're out in open=0A>> source land, I think we=
'll see them collaborating with us much more=0A>> openly.=0A>>=0A>> I for o=
ne look forward to working with these folks, and maybe merging=0A>> the pro=
jects some time down the road as the feature lists converge.=0A>>=0A>> -Tod=
d=0A>>=0A>> On Fri, Sep 2, 2011 at 11:40 AM, Gary Helmling <ghelmling@gmail=
.com> wrote:=0A>>> Some comments on the proposal and differentiation vs HBa=
se:=0A>>>=0A>>> Access Labels:=0A>>>=0A>>> The proposal claims that this is=
 "unlikely to be adopted [in HBase]". =A0This=0A>>> is completely untrue. =
=A0This has been discussed many times in the past in=0A>>> relation to our =
security implementation. =A0It's just been deferred at the=0A>>> moment due=
 to a need to focus on the initial implementation. =A0But it's=0A>>> certai=
nly viewed as a potentially important feature for a future iteration.=0A>>>=
 Contributions always welcome!=0A>>>=0A>>> see HBASE-3435: Provide per-colu=
mn-qualifier and per-key-value security for=0A>>> HBASE-3025=0A>>>=0A>>>=0A=
>>> Iterators:=0A>>>=0A>>> What do these provide that RegionObservers don't=
? =A0I'm speculating since the=0A>>> proposal provides little in the way of=
 details, but if these are "unlikely=0A>>> to be adopted" it's only because=
 coprocessors already offer more extensive=0A>>> functionality.=0A>>>=0A>>>=
=0A>>> "Flexibility" aka online schema changes and locality groups=0A>>>=0A=
>>> Locality groups seem to be the only meaningful differentiation in this=
=0A>>> entire comparison.=0A>>>=0A>>>=0A>>> Testing=0A>>>=0A>>> Performance=
 under "some configurations and conditions" and unsubstantiated=0A>>> "grea=
ter data integrity" is not meaningful differentiation.=0A>>>=0A>>>=0A>>> Ap=
ache Brand=0A>>>=0A>>> Claims a relationship with HBase. =A0Is there overla=
pping code or is this just=0A>>> the duplication of functionality? =A0There=
's no community relationship that=0A>>> I'm aware of. =A0I haven't seen any=
 of the proposed committers on the HBase=0A>>> user and dev lists to this p=
oint, so that doesn't set much of a precedent=0A>>> for community interacti=
on.=0A>>>=0A>>>=0A>>> Overall I see no meaningful differentiation vs HBase =
as an existing project,=0A>>> no past attempts to interact with the most re=
levant Apache community, and=0A>>> only an, until now, private "community" =
of government users. =A0I think it's=0A>>> great that they want to open sou=
rce this. =A0I don't want to discourage that=0A>>> -- go for it! =A0But I d=
on't see what the benefit is of ASF incubating this.=0A>>> I only see the p=
otential for community fragmentation and market confusion=0A>>> over such c=
losely similar projects.=0A>>>=0A>>>=0A>>> Gary=0A>>>=0A>>>=0A>>> On Fri, S=
ep 2, 2011 at 11:06 AM, Stack <stack@duboce.net> wrote:=0A>>>=0A>>>> See he=
re for the incubator proposal:=0A>>>> http://wiki.apache.org/incubator/Accu=
muloProposal=0A>>>>=0A>>>> Reactions probably better belong over on the inc=
ubator mailing list=0A>>>> but I thought a discussion here first might be u=
seful developing a=0A>>>> stance.=0A>>>>=0A>>>> Initial reaction, not havin=
g seen the code, is that it seems to be close to=0A>>>> HBase; so close, th=
ey call HBase out explicitly in their proposal.=0A>>>>=0A>>>> The cell base=
d 'access labels' seem like a matter of adding=0A>>>> an extra field to KV =
and their Iterators seem like a specialization on=0A>>>> Coprocessors. =A0T=
he ability to add column families on the fly seems too=0A>>>> minor a diffe=
rence to call out especially if online schema edits are=0A>>>> now (soon) s=
upported. =A0They talk of locality group like functionality=0A>>>> too -- t=
hat=0A>>>> could be a significant difference. =A0We would have to see the c=
ode but at=0A>>>> first blush, differences look small.=0A>>>>=0A>>>> Yet an=
other BT implementation further divides this contended space.=0A>>>> If the=
re were to be an effort integrating HBase into Accumulo or vice=0A>>>> vers=
a, its likely to distract significantly from project forward motion (If=0A>=
>>> the Accumulo fellows were interested in integrating the two projects,=
=0A>>>> I'd have thought they'd have tried to talk to us before this so tha=
ts=0A>>>> probably not their intent).=0A>>>>=0A>>>> On other hand, if their=
 once-secret project is out in the open, we can=0A>>>> steal the Apache-lic=
ensed good bits and....=0A>>>>=0A>>>> What do folks think?=0A>>>>=0A>>>> St=
.Ack=0A>>>>=0A>>>=0A>>=0A>>=0A>>=0A>> --=0A>> Todd Lipcon=0A>> Software Eng=
ineer, Cloudera=0A>>=0A>=0A>=0A>=0A>-- =0A>Joseph Echeverria=0A>Cloudera, I=
nc.=0A>443.305.9434=0A>=0A>=0A>
--0-1767787493-1315015577=:5129--