hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <els...@apache.org>
Subject Re: Moving 2.0 forward
Date Sat, 28 Oct 2017 00:10:39 GMT
My turn to bump ;)

By my take: HBASE-18770 and HBASE-19092 are the only issues that remain 
needing some more work. The rest are just awaiting a good QA run.

Unless I hear otherwise, I'll try to keep an eye on things over the 
weekend, bump them along as necessary, and get them committed. Would be 
great to be able get a vote up on Monday.

On 10/24/17 6:03 PM, Stack wrote:
> Chatting with my coworker Mr. Mike Drob, we were batting back and forth
> what remains to be done. Surfacing our thoughts here so you all clued
> in....Or if you think otherwise, please speak up.
> 
> We have ~13 issues to land:
> https://issues.apache.org/jira/projects/HBASE/versions/12341594 About two
> are meta-issues that are about process which leaves 11.
> 
> Duo and Zheng Hu are to merge the FilterList fixes improvements
> (HBASE-19057, HBASE-18410 et al.). These are blocker because some changed
> API/semantic that we need to get out earlier rather than later.
> 
> Once the above is merged, HBASE-13346, change of Filter method names to
> mention Cell instead of KeyValue can land.
> 
> HBASE-199048 needs a review (Anoop will probably do it), removing
> IA.Private objects as params to MasterObserver... Hopefully this goes in
> soon.
> 
> Duo is hard at work on trackers for flush and compaction for CPs
> (HBASE-18905). How is HBASE-19033 looking Duo (facility for Tephra)?
> 
> I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate after Duo is
> done w/ his work above.
> 
> I'm on HBASE-18770 bypass and HBASE-19077 restore some parity after all the
> purges allowing CPs do direct calls against Regions in same Host.
> 
> Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil.
> 
> Another day or two?
> 
> St.Ack
> 
> 
> 
> On Mon, Oct 23, 2017 at 2:52 PM, Stack <stack@duboce.net> wrote:
> 
>>
>> On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <elserj@apache.org> wrote:
>>
>>> +1
>>>
>>> I was trying to work on helping out on the outstanding alpha-4 stuff last
>>> week -- will be continuing to try to do the same this week.
>>>
>>> If you need any help, Stack, or if others need reviews where I haven't
>>> noticed on my own: feel free to @mention me.
>>>
>>>
>> Thanks for the offer Josh. All items seem assigned and are being actively
>> worked on. If you get a moment, reviews by you (or anyone else) helps move
>> the process along.
>>
>> We need to merge in HBASE-18410 branch to pick up Filter improvements.
>> Then HBASE-13346 can go in.
>>
>> You are already helping out on HBASE-18906, thanks. Looks like that will
>> be addressed by other alpha-4s about to land.
>>
>> St.Ack
>> TODOs: https://issues.apache.org/jira/projects/HBASE/versions/12341594
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>> On 10/23/17 12:53 PM, Stack wrote:
>>>
>>>> (Reviving this thread)
>>>>
>>>> Lets push out alpha-4 this week. Alpha-4 is the release that has the
>>>> refactor of the Coprocessor API shutting down access to internals marked
>>>> InterfaceAudience.Private.
>>>>
>>>> The outstanding list is here:
>>>> https://issues.apache.org/jira/projects/HBASE/versions/12341594
>>>>
>>>> Please push in anything marked alpha-4 that belongs to you.
>>>>
>>>> If issue, talk out loud on this thread. If you need a review to land an
>>>> item, shout on the issue and here; we'll help you out.
>>>>
>>>> As is, items are coming along nicely I'd say. We need to merge the filter
>>>> branch -- HBASE-18410 -- so APIs are finished for hbase2.
>>>>
>>>> Post alpha-4, we'll have to hunt down our downstreamers and help them
>>>> test
>>>> on top of alpha-4 so rolling into beta-1, we have confidence our
>>>> downstreamers know what to expect (or we discover what we missed BEFORE
>>>> we
>>>> beta-1).
>>>>
>>>> Thanks for time,
>>>> S
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Sep 8, 2017 at 2:04 PM, Stack <stack@duboce.net> wrote:
>>>>
>>>> I'll put up an alpha3 RC Monday, probably Monday night. That should be
>>>>> time, if we all sprint, for the public-facing API fixes to be done.
>>>>>
>>>>> I had a bunch of Coprocessor refactor and fixup scheduled for alpha3
but
>>>>> it is plain that more time is needed (in spite of valiant effort so far
>>>>> by
>>>>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4 whose
>>>>> theme is
>>>>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the following
>>>>> week.
>>>>>
>>>>> We should then be ready for beta (beta == no new features, no API
>>>>> changes,
>>>>> just fixes).
>>>>>
>>>>> Thanks,
>>>>> St.Ack
>>>>>
>>>>>
>>>>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <stack@duboce.net> wrote:
>>>>>
>>>>> I put up the hbase-2.0.0-alpha2 release candidate. Please vote on it.
>>>>>>
>>>>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I hope to get
a
>>>>>> release out in the next week or so.
>>>>>>
>>>>>> I did a weeding of 2.0.0 issues over the last day. If folks are
>>>>>> interested in helping out, below are the items I think we need done
for
>>>>>> alpha3 (below are at least 'Critical' status, are API possibly altering
>>>>>> items, and are absent those JIRAs that are making active progress,
>>>>>> i.e. the
>>>>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that needs
>>>>>> doing is
>>>>>> what Andrew did comparing 1.3. and 1.4 APIs
>>>>>>
>>>>>> * HBASE-18622 Mitigate compatibility concerns between branch-1 and
>>>>>> branch-2
>>>>>> This is to do what Andrew did between 1.3 and 1.4 branches only do
it
>>>>>> between branch-1 and branch-2.
>>>>>>
>>>>>> * HBASE-10462 Recategorize some of the client facing Public / Private
>>>>>> interfaces
>>>>>> This one is almost done. It could do with a finish, attention to
the
>>>>>> items in last comment, and then our codebase could do with another
>>>>>> sweep
>>>>>> after the spirit of this issue since a bunch has gone in since the
pass
>>>>>> that was the basis of this issue.
>>>>>>
>>>>>> * HBASE-10504 Define Replication Interface
>>>>>> I was going to take a crack at this as part of the revamp forced
by
>>>>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's Service'
>>>>>> but if
>>>>>> anyone else is interested, be my guest.
>>>>>>
>>>>>> * HBASE-14996 Some more API cleanup for 2.0
>>>>>> Has a bunch of subtasks, some of which are being worked on. Needs
>>>>>> finishing.
>>>>>>
>>>>>> * HBASE-14998 Unify synchronous and asynchronous methods in Admin
and
>>>>>> cleanup
>>>>>> Needs a pass. Small issue I think. Could also look at new AsyncClient
>>>>>> and
>>>>>> make sure symmetry.
>>>>>>
>>>>>> * HBASE-15607 Remove PB references from Admin for 2.0
>>>>>> Predicated on result of an ongoing DISCUSSION thread but needs to
be
>>>>>> done.
>>>>>>
>>>>>> Rolling upgrade will have implications for our API. Would be good
to
>>>>>> try
>>>>>> it and figure what needs fixup (as said above, according to trial
by
>>>>>> Sean,
>>>>>> we might not be too bad here):
>>>>>> * HBASE-16060 1.x clients cannot access table state talking to 2.0
>>>>>> cluster
>>>>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x Master
and
>>>>>> 1.x
>>>>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
>>>>>>
>>>>>> * HBASE-17442 Move most of the replication related classes to
>>>>>> hbase-server package
>>>>>> The above would be good to do generally but it may make for ripples
in
>>>>>> API so would be good to do now.
>>>>>>
>>>>>> * HBASE-18106 Redo ProcedureInfo and LockInfo
>>>>>> Balazs is working on this. The idea is that we avoid adding two new
>>>>>> types
>>>>>> to our API, two types that are nought but curtailed, read-only views
on
>>>>>> internals. Input if you have time appreciated.
>>>>>>
>>>>>> * HBASE-18596 A hbase1 cluster should be able to replicate to a hbase2
>>>>>> cluster; verify
>>>>>> Esteban is looking at this one
>>>>>>
>>>>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in core
>>>>>> * HBASE-17143 Scan improvement
>>>>>>
>>>>>> Our Coprocessor Interface needs a tough edit. It exposes
>>>>>> implementations
>>>>>> marked audience Private and returns implementations rather than
>>>>>> Interfaces.
>>>>>> In a few locations, we allow returning an alternate implementation
>>>>>> altogether which is probably something we don't want a CP doing.
To
>>>>>> that
>>>>>> end, the following issues started by Duo and Anoop need to be taken
to
>>>>>> the
>>>>>> finish line; ideally they'd have an owner:
>>>>>>
>>>>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0 release <=
The
>>>>>> umbrella issue.
>>>>>> * HBASE-18298 RegionServerServices Interface cleanup for CP expose
>>>>>> * HBASE-16769 Deprecate/remove PB references from MasterObserver
and
>>>>>> RegionServerObserver
>>>>>>
>>>>>>
>>>>>> Nice-to-haves:
>>>>>>
>>>>>> * HBASE-15284 Make TimeRange constructors IA.Private and remove unused
>>>>>> TimeRange constructors
>>>>>>
>>>>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow() references
>>>>>> existing in the code
>>>>>> This is the end of an old long-running project moving up on to Cell
>>>>>> Interface. We think it is done but for a few little items (deprecate
KV
>>>>>> methods in MR and provide Cell versions instead...)
>>>>>>
>>>>>> * HBASE-13271 Table#puts(List<Put>) operation is indeterminate;
needs
>>>>>> fixing
>>>>>>
>>>>>> * HBASE-13346 Clean up Filter package for post 1.0
>>>>>>
>>>>>> * HBASE-14255 Simplify Cell creation post 1.0
>>>>>> * HBASE-14997
>>>>>> Move compareOp and Comparators out of filter to client package
>>>>>>
>>>>>> * HBASE-13740 Stop using Hadoop private interfaces
>>>>>>
>>>>>> What about:
>>>>>>
>>>>>> * HBASE-18601 Remove Htrace 3.2
>>>>>> As has been noted, the HTrace API is our 'trace' API.
>>>>>>
>>>>>> If interested in any of the above and you need a legup, just ask
in the
>>>>>> issue and I'll be by....
>>>>>>
>>>>>> Thanks,
>>>>>> St.Ack
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <stack@duboce.net>
wrote:
>>>>>>
>>>>>> Heads-up:
>>>>>>>
>>>>>>> I'm about to put up an hbase-2.0.0-alpha2 Release Candidate.
Theme is
>>>>>>> updated dependencies, reliance on relocated popular libs (guava,
>>>>>>> netty,
>>>>>>> protobuf), purge of checked-in generated src, and
>>>>>>> master-carries-no-regions
>>>>>>> by default.
>>>>>>>
>>>>>>> alpha3 I hope will follow soon after (end-of-August?). Its theme
will
>>>>>>> be
>>>>>>> settling the APIs and compatibility (At first blush, we are not
>>>>>>> looking too
>>>>>>> bad; our Sean ran some tests over weekend that have hbase-1 client
>>>>>>> running
>>>>>>> against an hbase-2 cluster....). The Coprocessor Interface revamp
>>>>>>> should be
>>>>>>> done by alpha3 (i.e. returning Interfaces rather than
>>>>>>> Implementations, and
>>>>>>> our shutdown of CPs accessing classes in hbase marked
>>>>>>> InterfaceAudience).
>>>>>>> We'll also have purged thirdparty classes from our API; e.g.
guava
>>>>>>> 0.12
>>>>>>> Service showing through in our replication API and protobufs
in Admin
>>>>>>> Interface. On alpha3, we will have to do a bunch of outreach
to make
>>>>>>> sure
>>>>>>> our downstreamers are up on what is coming down the pipe.
>>>>>>>
>>>>>>> Beta1 in mid-September?
>>>>>>>
>>>>>>> I encourage you to check out the items marked for hbase2:
>>>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12327188
Edit
>>>>>>> as
>>>>>>> you see appropriate. Punt if you know the JIRA will not get any
>>>>>>> attention
>>>>>>> in next month or so.
>>>>>>>
>>>>>>> A bunch of issues marked blocker are unassigned. I'll leave them
as is
>>>>>>> another while but I'll boot them soon.
>>>>>>>
>>>>>>> While I have your attention:
>>>>>>>
>>>>>>> + I think we should leave thrift version at 0.9.3. Moving hbase
thrift
>>>>>>> to 0.10.0 will break existing clients. The change is easy enough
if
>>>>>>> folks
>>>>>>> need to upgrade their hbase thrift. See HBASE-18591.
>>>>>>> + Upgrade from 0.94 is disallowed. You have to get to 1.0 first
>>>>>>> (0.98?).
>>>>>>>
>>>>>>> St.Ack
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <stack@duboce.net>
wrote:
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <elserj@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 7/31/17 9:00 AM, Stack wrote:
>>>>>>>>>
>>>>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser<elserj@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> ...
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I like the idea of this also hitting 2.0 as it
would make the
>>>>>>>>>>> feature a
>>>>>>>>>>> bit more "real", but am obviously a little nervous
(I have no
>>>>>>>>>>> reason
>>>>>>>>>>> to be
>>>>>>>>>>> nervous though). I am pretty happy with the feature
in terms of
>>>>>>>>>>> how
>>>>>>>>>>> much it
>>>>>>>>>>> is covered via testing.
>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Sounds good to me. Whats involved? Backport?
If so, +1 Josh.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Last think on space quota says that need doc too.
See 'Space
>>>>>>>>>> Quota' in
>>>>>>>>>> here:
>>>>>>>>>> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9i
>>>>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5
>>>>>>>>>> Does this little section need an update Josh?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> S
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> Yep, just a couple of cherry-picks. Good test coverage
and some docs
>>>>>>>>> already included for 17748.  Happy to put that on my
plate if
>>>>>>>>> you're good
>>>>>>>>> with it. I can reasonably assume that no one is against
it :)
>>>>>>>>>
>>>>>>>>> I think I had knocked out docs for the "phase 1" stuff
before we
>>>>>>>>> merged it in from the original feature branch. I'll double
check
>>>>>>>>> and update
>>>>>>>>> the gdoc. Perhaps this was just a timing thing.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Thanks Josh,
>>>>>>>> S
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>
> 

Mime
View raw message