Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of saint.ack@gmail.com designates
 209.85.214.52 as permitted sender)
MIME-Version: 1.0
Sender: saint.ack@gmail.com
In-Reply-To: 
 <CADcMMgG-0Me1fiz2=9sA=tmeUp0jVmDY5O1koPqcPeoh5TqEkA@mail.gmail.com>
References: 
 <CADcMMgG-0Me1fiz2=9sA=tmeUp0jVmDY5O1koPqcPeoh5TqEkA@mail.gmail.com>
Date: Sat, 3 Aug 2013 13:35:56 -0700
Message-ID: 
 <CADcMMgHwmZKoj6XT1Ooe5hYZDsGAOv-Yc1vR1rMYVoSjAeFkqA@mail.gmail.com>
Subject: Re: [UPDATE] Finishing up 0.96 --> WAS Re: 0.95 and 0.96 remaining
 issues
From: Stack <stack@duboce.net>
To: HBase Dev List <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary=047d7b66f937b23b3604e311057a

--047d7b66f937b23b3604e311057a
Content-Type: text/plain; charset=UTF-8

On Sat, Jul 27, 2013 at 2:52 PM, Stack <stack@duboce.net> wrote:

> The end of July is upon us.  I intend to cut a 0.95.2 next weekend and
> 0.95.2 will be 0.96.0 but for bug fixes and migration polish.  No more
> 'features' will be allowed after next weekend.
>
>
>
> I have done some moving around of hbase 0.95.2 issues [1] to reflect what
> the priorities are for this week (to my mind).  It is all about polish, bad
> bugs, unit test failures, and packaging/publishing/build issues.
>
> I'll be working on the blockers this week [2].  A few are in need of
> reviews; e.g. "HBASE-3787 Increment is non-idempotent but client retries
> RPC".  Feel free to take over any blockers if you'd like to help out: e.g.
> "HBASE-7386 Investigate providing some supervisor support for znode
> deletion"
>
> Criticals [3] are mostly just unit test issues that are probably fixed by
> now and another few that are patches in need of test/review: e.g.
> "HBASE-8874 PutCombiner is skipping KeyValues while combining puts of same
> row during bulkload" and "HBASE-8778 Region assigments scan table directory
> making them slow for huge tables".
>  "HBASE-6127 TestAtomicOperation#testMultiRowMutationMultiThreads
> occasionally fails" is a bad one in need of attention.  Again, if up for
> helping out, be our guest.
>
> There are some great patches hanging out in the priority major issues
> section that are patch available that could be committed but for want of
> review: e.g. "HBASE-8369 MapReduce over snapshot files".
>
> Regards the big features that are racing to make the 0.96 cutoff -- namely
> namespaces, tags, and serialization lib -- as I see it, Francis needs
> reviews if namespaces are to make it, tags ditto, and the serialization
> libs are nice-to-have auxillaries that can come in at any time.  If not
> done by the end of this week, then as I see it, these features do not make
> the cut.
>
> Unit tests are mostly passing.  The problematics are being worked on (e.g.
> JD is on the replication set -- it likely a real problem rather than a
> flakey test).
>
> How's the above sound?
> St.Ack
>
> P.S. It is late but if folks want to meet this week to hack in patches
> together, just say.  I could organize an afternoon or day if you all think
> it would be a good idea.
>
> 1.
> https://issues.apache.org/jira/browse/HBASE/fixforversion/12320040#selectedTab=com.atlassian.jira.plugin.system.project%3Aversion-issues-panel
> 2.
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20fixVersion%20%3D%20%220.95.2%22%20AND%20resolution%20%3D%20Unresolved%20AND%20priority%20%3D%20Blocker%20ORDER%20BY%20key%20DESC
> 3.
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20fixVersion%20%3D%20%220.95.2%22%20AND%20resolution%20%3D%20Unresolved%20AND%20priority%20%3D%20Critical%20ORDER%20BY%20key%20DESC
>
>
>
UPDATE:

I was to cut 0.95.2 this weekend.  I am pushing out the cut-date to Weds or
so of next week.  The gating factor is namespaces.  It needs a few more
days of patch cycling.  I'll cut 0.95.2 after it goes in.  It'll be the
last feature on the 0.95/0.96 branch.  Thereafter, only bug fixes,
migration cleanup, and doc additions will be allowed (0.96.0RC will follow
soon after the 0.95.2 developer release goes out).

It looks like the serialization ilb will make the cut, API+Ordering; Nick
has signed-on some consumers it seems.

On KV tags, we have an outstanding -1 but -1s can change; lets see what the
new posted patch looks like.

Elliott has fancy-pants tracing that he should be able to get in before
0.95.2.

On the dodgy-looking outstanding blockers:

"HBASE-3787 Increment is non-idempotent but client retries RPC" still needs
reviews.  It is a difficult problem well-researched by our Sergey; perhaps
this does not make it?

On "HBASE-7386 Investigate providing some supervisor support for znode
deletion" we could doc. the "ugly" wrapper/watcher process with why it
exists and suggest it should be supervise instead (with a template) but
this could be post release?

Criticals seem to be all in good hands wanting a bit of testing or a last
review.  Lets get them in.

Reviewing the major issues, I do not see anything we should hold up the
release; please speak up if you think different (How about HBASE-7667, the
stripe compaction work or the fsync work in HBASE-5954?.

On unit tests, we are mostly passing now as reported a few days ago.  There
are a few flakies that would be sweet to purge; e.g:

HBASE-7980 TestZKInterProcessReadWriteLock fails occasionally in QA test run
HBASE-9023 TestIOFencing.testFencingAroundCompactionAfterWALSync

TestDistributedLogSplitting can go 'invisibile' on occasion; i.e. tests
fail it is only one absent from list of completed tests.

But unit tests are looking good on 0.95 and trunk:
https://builds.apache.org/view/H-L/view/HBase/

Your RM,
St.Ack


>
> On Tue, Jul 9, 2013 at 1:14 PM, Stack <stack@duboce.net> wrote:
>
>> I am shooting for end of July for 0.96 being 'complete'.   I would like
>> to make a 0.96 release in August.  We have some criticals outstanding but I
>> think we could ship even if these are not fixed in time (excepting
>> migration polish and of course remaining build fixes).  See [1.] for the
>> current list of issues.  Please re-prioritize issues as you see fit (or
>> better, move issues out of 0.95.2 if you do not think they will be done in
>> time).
>>
>> What to do with namespaces -- the last 0.96 'feature' -- given the above
>> timeline?  Currently it is a massive patch out on a branch.  It is still
>> not done, in want of review, and the author is going on holidays for a few
>> weeks soon.  My thinking as of now, going by the rate of change over the
>> last few weeks and estimating what is yet to be done, is that namespaces
>> will not make it.  I am willing to be convinced otherwise but that is how
>> it looks to me currently.
>>
>> I am going to start just disabling flakey unit tests in 0.95 from here on
>> out.  When folks get the itch, they can fix at leisure first on trunk and
>> then over in 0.95.
>>
>> What else?
>>
>> Thanks,
>> St.Ack
>>
>> 1.
>> https://issues.apache.org/jira/browse/HBASE/fixforversion/12320040#selectedTab=com.atlassian.jira.plugin.system.project%3Aversion-issues-panel
>>
>>
>>
>> On Mon, Jun 24, 2013 at 8:45 PM, Stack <stack@duboce.net> wrote:
>>
>>> (Changed the subject)
>>>
>>> On Mon, Jun 24, 2013 at 4:04 PM, Nick Dimiduk <ndimiduk@gmail.com>wrote:
>>>
>>>> I want to see initial data type APIs ship out with 0.95.2. A patch for
>>>> ordered byte serialization is up (HBASE-8201) and is nearing
>>>> steady-state.
>>>> However, sershe is the only person who's left feedback. I just posted an
>>>> early patch for the data type API itself (HBASE-8693). It should get
>>>> some
>>>> eyes from all manor of interested parties, but I'll settle for folk from
>>>> Phoenix for now.
>>>>
>>>>
>>> It would be cool if Phoenix and Kiji fellows and any one else interested
>>> would weigh in and take a look see.
>>>
>>> This does not strike me as something we should hold up the release for
>>> though.  It looks like something that could go in at any time?
>>>
>>>
>>>
>>>> Should these tasks be escalated to criticals in order to grab attention?
>>>>
>>>>
>>> I don't think that works going by past experience (and I don't think
>>> this a blocker on 0.96)
>>>
>>>
>>>
>>>> Additional comments inline.
>>>>
>>>>
>>> ...
>>>
>>>
>>>
>>>>  Namespaces is the long pole and progress seems slow.  Do we hold up the
>>>> > release for them?  How can we hurry this effort along?  Swat team
>>>> descends
>>>> > on Y!?
>>>> >
>>>>
>>>> It would be a shame to not get a decision on this in for 0.96.
>>>>
>>>>
>>> Agree.  We need to get 0.96 out though.  It has been too long.
>>>
>>>
>>>
>>>>  + Is anyone testing?  Integration tests fail on ec2 build from time to
>>>> time
>>>> > [2].  Our Elliott dug in on one of the failures a few days back and
>>>> found
>>>> > legit issue w/ no retry on admin tasks (I heart hbase-it tests).  Our
>>>> unit
>>>> > test story is better [3] but there are still the odd failures.
>>>> >
>>>>
>>>> With the creation of the new list, noticing these issues is going to
>>>> push
>>>> further back-burner. Nannying this stuff should retain focus. I'll
>>>> volunteer to track on these issues as I see them.
>>>>
>>>>
>>> Thank you Nick.
>>>
>>>
>>>
>>>> Notice though that some of the more recent failures are caused by lack
>>>> of
>>>> disk space on the Jenkins build host.
>>>>
>>>>
>>> Oh.  Missed that.  Let me dig in.
>>>
>>>
>>> St.Ack
>>>
>>
>>
>

--047d7b66f937b23b3604e311057a--