hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Moving 2.0 forward
Date Sat, 28 Oct 2017 21:48:52 GMT
Yup, that was going to be my plan, Mike!

Making a pass now, and will check back later tonight again. I see others 
have already done some work today on this front.

On 10/27/17 11:38 PM, Mike Drob wrote:
> Josh - Do you want to kick off a bunch of QA runs? (Do you know how to do
> it directly on the jenkins job, so you don't have to bother with JIRA
> uploads)
> 
> If you're busy, then I can make time tomorrow or Sunday to kick off jobs,
> but I want to make sure we're not duplicating effort and jenkins cycles.
> 
> On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser <elserj@apache.org> wrote:
> 
>> My turn to bump ;)
>>
>> By my take: HBASE-18770 and HBASE-19092 are the only issues that remain
>> needing some more work. The rest are just awaiting a good QA run.
>>
>> Unless I hear otherwise, I'll try to keep an eye on things over the
>> weekend, bump them along as necessary, and get them committed. Would be
>> great to be able get a vote up on Monday.
>>
>>
>> On 10/24/17 6:03 PM, Stack wrote:
>>
>>> Chatting with my coworker Mr. Mike Drob, we were batting back and forth
>>> what remains to be done. Surfacing our thoughts here so you all clued
>>> in....Or if you think otherwise, please speak up.
>>>
>>> We have ~13 issues to land:
>>> https://issues.apache.org/jira/projects/HBASE/versions/12341594 About two
>>> are meta-issues that are about process which leaves 11.
>>>
>>> Duo and Zheng Hu are to merge the FilterList fixes improvements
>>> (HBASE-19057, HBASE-18410 et al.). These are blocker because some changed
>>> API/semantic that we need to get out earlier rather than later.
>>>
>>> Once the above is merged, HBASE-13346, change of Filter method names to
>>> mention Cell instead of KeyValue can land.
>>>
>>> HBASE-199048 needs a review (Anoop will probably do it), removing
>>> IA.Private objects as params to MasterObserver... Hopefully this goes in
>>> soon.
>>>
>>> Duo is hard at work on trackers for flush and compaction for CPs
>>> (HBASE-18905). How is HBASE-19033 looking Duo (facility for Tephra)?
>>>
>>> I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate after Duo
>>> is
>>> done w/ his work above.
>>>
>>> I'm on HBASE-18770 bypass and HBASE-19077 restore some parity after all
>>> the
>>> purges allowing CPs do direct calls against Regions in same Host.
>>>
>>> Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil.
>>>
>>> Another day or two?
>>>
>>> St.Ack
>>>
>>>
>>>
>>> On Mon, Oct 23, 2017 at 2:52 PM, Stack <stack@duboce.net> wrote:
>>>
>>>
>>>> On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <elserj@apache.org> wrote:
>>>>
>>>> +1
>>>>>
>>>>> I was trying to work on helping out on the outstanding alpha-4 stuff
>>>>> last
>>>>> week -- will be continuing to try to do the same this week.
>>>>>
>>>>> If you need any help, Stack, or if others need reviews where I haven't
>>>>> noticed on my own: feel free to @mention me.
>>>>>
>>>>>
>>>>> Thanks for the offer Josh. All items seem assigned and are being
>>>> actively
>>>> worked on. If you get a moment, reviews by you (or anyone else) helps
>>>> move
>>>> the process along.
>>>>
>>>> We need to merge in HBASE-18410 branch to pick up Filter improvements.
>>>> Then HBASE-13346 can go in.
>>>>
>>>> You are already helping out on HBASE-18906, thanks. Looks like that will
>>>> be addressed by other alpha-4s about to land.
>>>>
>>>> St.Ack
>>>> TODOs: https://issues.apache.org/jira/projects/HBASE/versions/12341594
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 10/23/17 12:53 PM, Stack wrote:
>>>>>
>>>>> (Reviving this thread)
>>>>>>
>>>>>> Lets push out alpha-4 this week. Alpha-4 is the release that has
the
>>>>>> refactor of the Coprocessor API shutting down access to internals
>>>>>> marked
>>>>>> InterfaceAudience.Private.
>>>>>>
>>>>>> The outstanding list is here:
>>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12341594
>>>>>>
>>>>>> Please push in anything marked alpha-4 that belongs to you.
>>>>>>
>>>>>> If issue, talk out loud on this thread. If you need a review to land
an
>>>>>> item, shout on the issue and here; we'll help you out.
>>>>>>
>>>>>> As is, items are coming along nicely I'd say. We need to merge the
>>>>>> filter
>>>>>> branch -- HBASE-18410 -- so APIs are finished for hbase2.
>>>>>>
>>>>>> Post alpha-4, we'll have to hunt down our downstreamers and help
them
>>>>>> test
>>>>>> on top of alpha-4 so rolling into beta-1, we have confidence our
>>>>>> downstreamers know what to expect (or we discover what we missed
BEFORE
>>>>>> we
>>>>>> beta-1).
>>>>>>
>>>>>> Thanks for time,
>>>>>> S
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Sep 8, 2017 at 2:04 PM, Stack <stack@duboce.net> wrote:
>>>>>>
>>>>>> I'll put up an alpha3 RC Monday, probably Monday night. That should
be
>>>>>>
>>>>>>> time, if we all sprint, for the public-facing API fixes to be
done.
>>>>>>>
>>>>>>> I had a bunch of Coprocessor refactor and fixup scheduled for
alpha3
>>>>>>> but
>>>>>>> it is plain that more time is needed (in spite of valiant effort
so
>>>>>>> far
>>>>>>> by
>>>>>>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4
whose
>>>>>>> theme is
>>>>>>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the
>>>>>>> following
>>>>>>> week.
>>>>>>>
>>>>>>> We should then be ready for beta (beta == no new features, no
API
>>>>>>> changes,
>>>>>>> just fixes).
>>>>>>>
>>>>>>> Thanks,
>>>>>>> St.Ack
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <stack@duboce.net>
wrote:
>>>>>>>
>>>>>>> I put up the hbase-2.0.0-alpha2 release candidate. Please vote
on it.
>>>>>>>
>>>>>>>>
>>>>>>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I hope
to get a
>>>>>>>> release out in the next week or so.
>>>>>>>>
>>>>>>>> I did a weeding of 2.0.0 issues over the last day. If folks
are
>>>>>>>> interested in helping out, below are the items I think we
need done
>>>>>>>> for
>>>>>>>> alpha3 (below are at least 'Critical' status, are API possibly
>>>>>>>> altering
>>>>>>>> items, and are absent those JIRAs that are making active
progress,
>>>>>>>> i.e. the
>>>>>>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that
needs
>>>>>>>> doing is
>>>>>>>> what Andrew did comparing 1.3. and 1.4 APIs
>>>>>>>>
>>>>>>>> * HBASE-18622 Mitigate compatibility concerns between branch-1
and
>>>>>>>> branch-2
>>>>>>>> This is to do what Andrew did between 1.3 and 1.4 branches
only do it
>>>>>>>> between branch-1 and branch-2.
>>>>>>>>
>>>>>>>> * HBASE-10462 Recategorize some of the client facing Public
/ Private
>>>>>>>> interfaces
>>>>>>>> This one is almost done. It could do with a finish, attention
to the
>>>>>>>> items in last comment, and then our codebase could do with
another
>>>>>>>> sweep
>>>>>>>> after the spirit of this issue since a bunch has gone in
since the
>>>>>>>> pass
>>>>>>>> that was the basis of this issue.
>>>>>>>>
>>>>>>>> * HBASE-10504 Define Replication Interface
>>>>>>>> I was going to take a crack at this as part of the revamp
forced by
>>>>>>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's
Service'
>>>>>>>> but if
>>>>>>>> anyone else is interested, be my guest.
>>>>>>>>
>>>>>>>> * HBASE-14996 Some more API cleanup for 2.0
>>>>>>>> Has a bunch of subtasks, some of which are being worked on.
Needs
>>>>>>>> finishing.
>>>>>>>>
>>>>>>>> * HBASE-14998 Unify synchronous and asynchronous methods
in Admin and
>>>>>>>> cleanup
>>>>>>>> Needs a pass. Small issue I think. Could also look at new
AsyncClient
>>>>>>>> and
>>>>>>>> make sure symmetry.
>>>>>>>>
>>>>>>>> * HBASE-15607 Remove PB references from Admin for 2.0
>>>>>>>> Predicated on result of an ongoing DISCUSSION thread but
needs to be
>>>>>>>> done.
>>>>>>>>
>>>>>>>> Rolling upgrade will have implications for our API. Would
be good to
>>>>>>>> try
>>>>>>>> it and figure what needs fixup (as said above, according
to trial by
>>>>>>>> Sean,
>>>>>>>> we might not be too bad here):
>>>>>>>> * HBASE-16060 1.x clients cannot access table state talking
to 2.0
>>>>>>>> cluster
>>>>>>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x
Master and
>>>>>>>> 1.x
>>>>>>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
>>>>>>>>
>>>>>>>> * HBASE-17442 Move most of the replication related classes
to
>>>>>>>> hbase-server package
>>>>>>>> The above would be good to do generally but it may make for
ripples
>>>>>>>> in
>>>>>>>> API so would be good to do now.
>>>>>>>>
>>>>>>>> * HBASE-18106 Redo ProcedureInfo and LockInfo
>>>>>>>> Balazs is working on this. The idea is that we avoid adding
two new
>>>>>>>> types
>>>>>>>> to our API, two types that are nought but curtailed, read-only
views
>>>>>>>> on
>>>>>>>> internals. Input if you have time appreciated.
>>>>>>>>
>>>>>>>> * HBASE-18596 A hbase1 cluster should be able to replicate
to a
>>>>>>>> hbase2
>>>>>>>> cluster; verify
>>>>>>>> Esteban is looking at this one
>>>>>>>>
>>>>>>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in core
>>>>>>>> * HBASE-17143 Scan improvement
>>>>>>>>
>>>>>>>> Our Coprocessor Interface needs a tough edit. It exposes
>>>>>>>> implementations
>>>>>>>> marked audience Private and returns implementations rather
than
>>>>>>>> Interfaces.
>>>>>>>> In a few locations, we allow returning an alternate implementation
>>>>>>>> altogether which is probably something we don't want a CP
doing. To
>>>>>>>> that
>>>>>>>> end, the following issues started by Duo and Anoop need to
be taken
>>>>>>>> to
>>>>>>>> the
>>>>>>>> finish line; ideally they'd have an owner:
>>>>>>>>
>>>>>>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0 release
<= The
>>>>>>>> umbrella issue.
>>>>>>>> * HBASE-18298 RegionServerServices Interface cleanup for
CP expose
>>>>>>>> * HBASE-16769 Deprecate/remove PB references from MasterObserver
and
>>>>>>>> RegionServerObserver
>>>>>>>>
>>>>>>>>
>>>>>>>> Nice-to-haves:
>>>>>>>>
>>>>>>>> * HBASE-15284 Make TimeRange constructors IA.Private and
remove
>>>>>>>> unused
>>>>>>>> TimeRange constructors
>>>>>>>>
>>>>>>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow() references
>>>>>>>> existing in the code
>>>>>>>> This is the end of an old long-running project moving up
on to Cell
>>>>>>>> Interface. We think it is done but for a few little items
(deprecate
>>>>>>>> KV
>>>>>>>> methods in MR and provide Cell versions instead...)
>>>>>>>>
>>>>>>>> * HBASE-13271 Table#puts(List<Put>) operation is indeterminate;
needs
>>>>>>>> fixing
>>>>>>>>
>>>>>>>> * HBASE-13346 Clean up Filter package for post 1.0
>>>>>>>>
>>>>>>>> * HBASE-14255 Simplify Cell creation post 1.0
>>>>>>>> * HBASE-14997
>>>>>>>> Move compareOp and Comparators out of filter to client package
>>>>>>>>
>>>>>>>> * HBASE-13740 Stop using Hadoop private interfaces
>>>>>>>>
>>>>>>>> What about:
>>>>>>>>
>>>>>>>> * HBASE-18601 Remove Htrace 3.2
>>>>>>>> As has been noted, the HTrace API is our 'trace' API.
>>>>>>>>
>>>>>>>> If interested in any of the above and you need a legup, just
ask in
>>>>>>>> the
>>>>>>>> issue and I'll be by....
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> St.Ack
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <stack@duboce.net>
wrote:
>>>>>>>>
>>>>>>>> Heads-up:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> I'm about to put up an hbase-2.0.0-alpha2 Release Candidate.
Theme
>>>>>>>>> is
>>>>>>>>> updated dependencies, reliance on relocated popular libs
(guava,
>>>>>>>>> netty,
>>>>>>>>> protobuf), purge of checked-in generated src, and
>>>>>>>>> master-carries-no-regions
>>>>>>>>> by default.
>>>>>>>>>
>>>>>>>>> alpha3 I hope will follow soon after (end-of-August?).
Its theme
>>>>>>>>> will
>>>>>>>>> be
>>>>>>>>> settling the APIs and compatibility (At first blush,
we are not
>>>>>>>>> looking too
>>>>>>>>> bad; our Sean ran some tests over weekend that have hbase-1
client
>>>>>>>>> running
>>>>>>>>> against an hbase-2 cluster....). The Coprocessor Interface
revamp
>>>>>>>>> should be
>>>>>>>>> done by alpha3 (i.e. returning Interfaces rather than
>>>>>>>>> Implementations, and
>>>>>>>>> our shutdown of CPs accessing classes in hbase marked
>>>>>>>>> InterfaceAudience).
>>>>>>>>> We'll also have purged thirdparty classes from our API;
e.g. guava
>>>>>>>>> 0.12
>>>>>>>>> Service showing through in our replication API and protobufs
in
>>>>>>>>> Admin
>>>>>>>>> Interface. On alpha3, we will have to do a bunch of outreach
to make
>>>>>>>>> sure
>>>>>>>>> our downstreamers are up on what is coming down the pipe.
>>>>>>>>>
>>>>>>>>> Beta1 in mid-September?
>>>>>>>>>
>>>>>>>>> I encourage you to check out the items marked for hbase2:
>>>>>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12327188
>>>>>>>>> Edit
>>>>>>>>> as
>>>>>>>>> you see appropriate. Punt if you know the JIRA will not
get any
>>>>>>>>> attention
>>>>>>>>> in next month or so.
>>>>>>>>>
>>>>>>>>> A bunch of issues marked blocker are unassigned. I'll
leave them as
>>>>>>>>> is
>>>>>>>>> another while but I'll boot them soon.
>>>>>>>>>
>>>>>>>>> While I have your attention:
>>>>>>>>>
>>>>>>>>> + I think we should leave thrift version at 0.9.3. Moving
hbase
>>>>>>>>> thrift
>>>>>>>>> to 0.10.0 will break existing clients. The change is
easy enough if
>>>>>>>>> folks
>>>>>>>>> need to upgrade their hbase thrift. See HBASE-18591.
>>>>>>>>> + Upgrade from 0.94 is disallowed. You have to get to
1.0 first
>>>>>>>>> (0.98?).
>>>>>>>>>
>>>>>>>>> St.Ack
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <stack@duboce.net>
wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <elserj@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On 7/31/17 9:00 AM, Stack wrote:
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser<elserj@apache.org>
>>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> ...
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> I like the idea of this also hitting
2.0 as it would make the
>>>>>>>>>>>>> feature a
>>>>>>>>>>>>> bit more "real", but am obviously a little
nervous (I have no
>>>>>>>>>>>>> reason
>>>>>>>>>>>>> to be
>>>>>>>>>>>>> nervous though). I am pretty happy with
the feature in terms of
>>>>>>>>>>>>> how
>>>>>>>>>>>>> much it
>>>>>>>>>>>>> is covered via testing.
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sounds good to me. Whats involved? Backport?
If so, +1 Josh.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> Last think on space quota says that need
doc too. See 'Space
>>>>>>>>>>>> Quota' in
>>>>>>>>>>>> here:
>>>>>>>>>>>> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9i
>>>>>>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5
>>>>>>>>>>>> Does this little section need an update Josh?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> S
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Yep, just a couple of cherry-picks. Good
test coverage and some
>>>>>>>>>>> docs
>>>>>>>>>>> already included for 17748.  Happy to put that
on my plate if
>>>>>>>>>>> you're good
>>>>>>>>>>> with it. I can reasonably assume that no one
is against it :)
>>>>>>>>>>>
>>>>>>>>>>> I think I had knocked out docs for the "phase
1" stuff before we
>>>>>>>>>>> merged it in from the original feature branch.
I'll double check
>>>>>>>>>>> and update
>>>>>>>>>>> the gdoc. Perhaps this was just a timing thing.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks Josh,
>>>>>>>>>> S
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
> 

Mime
View raw message