Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of jdcryans@gmail.com designates
 209.85.213.41 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=ic322PUP+qKZCv1/aNSGtcW/YFiV7h1wijVPKEOuirmv13U/ex78h5Gf9Kun0PwIpd
         4x9ihOEmmN1S29Gh2TKrqCeTRtZ3AoMa4NY6JCfzhQtCZwv2pGwc6frjbl6oUJ/0vapm
         GhxvdtDbfS+uWAp8Xk7IU4jKvKjbAnbjGW6Hk=
MIME-Version: 1.0
Sender: jdcryans@gmail.com
In-Reply-To: 
 <5A76F6CE309AD049AAF9A039A39242820F016764@sc-mbx04.TheFacebook.com>
References: <AANLkTi=Hr3v95As8hf+tCCf5mzsQMC8Y92rT2sBD1HhM@mail.gmail.com>
	<AANLkTimR1j7q8+2ehO38npoZOKveYDN1SXyzr_PYmTMC@mail.gmail.com>
	<AANLkTi=xYoxUy2C_v0EWs6czHGUPkiwT-Vj3jaP3Kfa7@mail.gmail.com>
	<AANLkTimkjVOG0fPCEjES850v=uuD_Q=v7ynfbSYo+iK8@mail.gmail.com>
	<AANLkTimKrj8gZBAzKSa3bT8Vu01N5rDcpE62vhtAwr+R@mail.gmail.com>
	<AANLkTimXGj0HovoMfsgH2-DPyxNjcKRZSv_6GDbNyEwO@mail.gmail.com>
	<AANLkTi=Q4AQBfzSXKfHQ53yb+YJYAaZ7=abGYzGEoQDt@mail.gmail.com>
	<5A76F6CE309AD049AAF9A039A39242820F016764@sc-mbx04.TheFacebook.com>
Date: Tue, 21 Sep 2010 09:28:42 -0700
Message-ID: <AANLkTimwyjR6YS27DfN36WsgYNtfyG8jUfvegn1YHbg2@mail.gmail.com>
Subject: Re: old master to 0.90 and new master to 0.92? (was RE: Millions of
 photos into Hbase)
From: Jean-Daniel Cryans <jdcryans@apache.org>
To: dev@hbase.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

So while you were in the plane a few things were discussed :)

In the thread "[VOTE] Release 'development release' HBase 0.89.2010830
rc2", it was decided that we sink RC2 in favor of a new RC with a
rollback of the zk-based assignment. This is what we're running here,
and Ryan published our repo yesterday
http://github.com/stumbleupon/hbase  (see the top of the CHANGES file
for what we added)

Also some of your arguments correlate those of Todd, expressed in the
thread "Next release" that was posted on 09/15. Stack and Andrew
expressed their opinions in favor of Todd's option #1 (although Stack
said that he would decide tomorrow if he changes his opinion after
some more work on the master).

My personal opinion leans towards Todd's option #2. 0.20.0 is now more
than a year old, we are able to release almost production-ready code
that is durable, so why not move forward with it (this is what we did
here). There are a few issues to fix, but it's small compared to
getting the new master in shape plus getting the rest working (like
the replication code's interaction with ZooKeeper that needs to be
redone).

J-D

On Tue, Sep 21, 2010 at 4:49 AM, Jonathan Gray <jgray@facebook.com> wrote:
> Deep within my soul I do not want to do this.
>
> But it might be practical.
>
> FB is going into prod w/ the old master and we've been doing work stabili=
zing it in ways that do not apply at all to the new one (especially around =
zk). =A0That'll make having two different active branches a bit of a nightm=
are but right now some of these patches are not even available as we've bee=
n operating under the assumption that the release would be on the new maste=
r (and the patches don't apply on trunk).
>
> I guess SU is also putting an 0.89 old master release into prod.
>
> What's a little unfortunate is that the 0.89 releases include the unneces=
sary move of some transition communication into ZK. =A0That was put in as a=
 first step towards new master before we decided to branch it off.
>
> The question for me would be whether we think it's at all feasible to get=
 the new master 0.90 released in time for HW. =A0If not, maybe we should co=
nsider taking an 0.89 branch, making it an 0.90 branch, and focus on stabil=
izing for release in time for HW, as Todd suggests.
>
> There are other ramifications of this. =A0It would be very difficult for =
me to get my flush/compact/split improvements in to the old master as the n=
ew implementation I've been working on relies completely on the new stuff. =
=A0But maybe better to punt that for 0.92 as well so can really nail it?
>
> The other factor is the enormous amount of ZK improvements that were done=
 in the new master branch. =A0It's a real fuckin mess in 0.89 releases, tho=
ugh I've done a bit of cleanup already towards making it at least tenable f=
or production.
>
> My main concern is that this move will push back any release of the new m=
aster significantly. =A0There are countless improvements in the codebase th=
at came along with the rewrite, well beyond just zk transitions. =A0But doi=
ng a production release in time for HW is probably the most important thing=
.
>
> JG
>
>> -----Original Message-----
>> From: Todd Lipcon [mailto:todd@cloudera.com]
>> Sent: Tuesday, September 21, 2010 8:10 AM
>> To: dev
>> Subject: Re: Millions of photos into Hbase
>>
>> On Mon, Sep 20, 2010 at 5:47 PM, Ryan Rawson <ryanobjc@gmail.com>
>> wrote:
>> > Hey,
>> >
>> > There is actually only 1 active branch of hbase, that being the 0.89
>> > release, which is based on 'trunk'. =A0We have snapshotted a series of
>> > 0.89 "developer releases" in hopes that people would try them our and
>> > start thinking about the next major version. =A0One of these is what S=
U
>> > is running prod on.
>> >
>> > At this point tracking 0.89 and which ones are the 'best' peach sets
>> > to run is a bit of a contact sport, but if you are serious about not
>> > losing data it is worthwhile. =A0SU is based on the most recent DR wit=
h
>> > a few minor patches of our own concoction brought in. =A0If current
>> > works, but some Master ops are slow, and there are a few patches on
>> > top of that. =A0I'll poke about and see if its possible to publish to =
a
>> > github branch or something.
>> >
>>
>> This is why I kind of want to release the latest 0.89 dev release as
>> 0.90, and push off the new master stuff as 0.92 :)
>>
>> -Todd
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>