hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Drob <md...@apache.org>
Subject Re: Moving 2.0 forward
Date Sat, 28 Oct 2017 03:38:49 GMT
Josh - Do you want to kick off a bunch of QA runs? (Do you know how to do
it directly on the jenkins job, so you don't have to bother with JIRA
uploads)

If you're busy, then I can make time tomorrow or Sunday to kick off jobs,
but I want to make sure we're not duplicating effort and jenkins cycles.

On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser <elserj@apache.org> wrote:

> My turn to bump ;)
>
> By my take: HBASE-18770 and HBASE-19092 are the only issues that remain
> needing some more work. The rest are just awaiting a good QA run.
>
> Unless I hear otherwise, I'll try to keep an eye on things over the
> weekend, bump them along as necessary, and get them committed. Would be
> great to be able get a vote up on Monday.
>
>
> On 10/24/17 6:03 PM, Stack wrote:
>
>> Chatting with my coworker Mr. Mike Drob, we were batting back and forth
>> what remains to be done. Surfacing our thoughts here so you all clued
>> in....Or if you think otherwise, please speak up.
>>
>> We have ~13 issues to land:
>> https://issues.apache.org/jira/projects/HBASE/versions/12341594 About two
>> are meta-issues that are about process which leaves 11.
>>
>> Duo and Zheng Hu are to merge the FilterList fixes improvements
>> (HBASE-19057, HBASE-18410 et al.). These are blocker because some changed
>> API/semantic that we need to get out earlier rather than later.
>>
>> Once the above is merged, HBASE-13346, change of Filter method names to
>> mention Cell instead of KeyValue can land.
>>
>> HBASE-199048 needs a review (Anoop will probably do it), removing
>> IA.Private objects as params to MasterObserver... Hopefully this goes in
>> soon.
>>
>> Duo is hard at work on trackers for flush and compaction for CPs
>> (HBASE-18905). How is HBASE-19033 looking Duo (facility for Tephra)?
>>
>> I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate after Duo
>> is
>> done w/ his work above.
>>
>> I'm on HBASE-18770 bypass and HBASE-19077 restore some parity after all
>> the
>> purges allowing CPs do direct calls against Regions in same Host.
>>
>> Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil.
>>
>> Another day or two?
>>
>> St.Ack
>>
>>
>>
>> On Mon, Oct 23, 2017 at 2:52 PM, Stack <stack@duboce.net> wrote:
>>
>>
>>> On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <elserj@apache.org> wrote:
>>>
>>> +1
>>>>
>>>> I was trying to work on helping out on the outstanding alpha-4 stuff
>>>> last
>>>> week -- will be continuing to try to do the same this week.
>>>>
>>>> If you need any help, Stack, or if others need reviews where I haven't
>>>> noticed on my own: feel free to @mention me.
>>>>
>>>>
>>>> Thanks for the offer Josh. All items seem assigned and are being
>>> actively
>>> worked on. If you get a moment, reviews by you (or anyone else) helps
>>> move
>>> the process along.
>>>
>>> We need to merge in HBASE-18410 branch to pick up Filter improvements.
>>> Then HBASE-13346 can go in.
>>>
>>> You are already helping out on HBASE-18906, thanks. Looks like that will
>>> be addressed by other alpha-4s about to land.
>>>
>>> St.Ack
>>> TODOs: https://issues.apache.org/jira/projects/HBASE/versions/12341594
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 10/23/17 12:53 PM, Stack wrote:
>>>>
>>>> (Reviving this thread)
>>>>>
>>>>> Lets push out alpha-4 this week. Alpha-4 is the release that has the
>>>>> refactor of the Coprocessor API shutting down access to internals
>>>>> marked
>>>>> InterfaceAudience.Private.
>>>>>
>>>>> The outstanding list is here:
>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12341594
>>>>>
>>>>> Please push in anything marked alpha-4 that belongs to you.
>>>>>
>>>>> If issue, talk out loud on this thread. If you need a review to land
an
>>>>> item, shout on the issue and here; we'll help you out.
>>>>>
>>>>> As is, items are coming along nicely I'd say. We need to merge the
>>>>> filter
>>>>> branch -- HBASE-18410 -- so APIs are finished for hbase2.
>>>>>
>>>>> Post alpha-4, we'll have to hunt down our downstreamers and help them
>>>>> test
>>>>> on top of alpha-4 so rolling into beta-1, we have confidence our
>>>>> downstreamers know what to expect (or we discover what we missed BEFORE
>>>>> we
>>>>> beta-1).
>>>>>
>>>>> Thanks for time,
>>>>> S
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Sep 8, 2017 at 2:04 PM, Stack <stack@duboce.net> wrote:
>>>>>
>>>>> I'll put up an alpha3 RC Monday, probably Monday night. That should be
>>>>>
>>>>>> time, if we all sprint, for the public-facing API fixes to be done.
>>>>>>
>>>>>> I had a bunch of Coprocessor refactor and fixup scheduled for alpha3
>>>>>> but
>>>>>> it is plain that more time is needed (in spite of valiant effort
so
>>>>>> far
>>>>>> by
>>>>>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4 whose
>>>>>> theme is
>>>>>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the
>>>>>> following
>>>>>> week.
>>>>>>
>>>>>> We should then be ready for beta (beta == no new features, no API
>>>>>> changes,
>>>>>> just fixes).
>>>>>>
>>>>>> Thanks,
>>>>>> St.Ack
>>>>>>
>>>>>>
>>>>>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <stack@duboce.net>
wrote:
>>>>>>
>>>>>> I put up the hbase-2.0.0-alpha2 release candidate. Please vote on
it.
>>>>>>
>>>>>>>
>>>>>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I hope
to get a
>>>>>>> release out in the next week or so.
>>>>>>>
>>>>>>> I did a weeding of 2.0.0 issues over the last day. If folks are
>>>>>>> interested in helping out, below are the items I think we need
done
>>>>>>> for
>>>>>>> alpha3 (below are at least 'Critical' status, are API possibly
>>>>>>> altering
>>>>>>> items, and are absent those JIRAs that are making active progress,
>>>>>>> i.e. the
>>>>>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that
needs
>>>>>>> doing is
>>>>>>> what Andrew did comparing 1.3. and 1.4 APIs
>>>>>>>
>>>>>>> * HBASE-18622 Mitigate compatibility concerns between branch-1
and
>>>>>>> branch-2
>>>>>>> This is to do what Andrew did between 1.3 and 1.4 branches only
do it
>>>>>>> between branch-1 and branch-2.
>>>>>>>
>>>>>>> * HBASE-10462 Recategorize some of the client facing Public /
Private
>>>>>>> interfaces
>>>>>>> This one is almost done. It could do with a finish, attention
to the
>>>>>>> items in last comment, and then our codebase could do with another
>>>>>>> sweep
>>>>>>> after the spirit of this issue since a bunch has gone in since
the
>>>>>>> pass
>>>>>>> that was the basis of this issue.
>>>>>>>
>>>>>>> * HBASE-10504 Define Replication Interface
>>>>>>> I was going to take a crack at this as part of the revamp forced
by
>>>>>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's Service'
>>>>>>> but if
>>>>>>> anyone else is interested, be my guest.
>>>>>>>
>>>>>>> * HBASE-14996 Some more API cleanup for 2.0
>>>>>>> Has a bunch of subtasks, some of which are being worked on. Needs
>>>>>>> finishing.
>>>>>>>
>>>>>>> * HBASE-14998 Unify synchronous and asynchronous methods in Admin
and
>>>>>>> cleanup
>>>>>>> Needs a pass. Small issue I think. Could also look at new AsyncClient
>>>>>>> and
>>>>>>> make sure symmetry.
>>>>>>>
>>>>>>> * HBASE-15607 Remove PB references from Admin for 2.0
>>>>>>> Predicated on result of an ongoing DISCUSSION thread but needs
to be
>>>>>>> done.
>>>>>>>
>>>>>>> Rolling upgrade will have implications for our API. Would be
good to
>>>>>>> try
>>>>>>> it and figure what needs fixup (as said above, according to trial
by
>>>>>>> Sean,
>>>>>>> we might not be too bad here):
>>>>>>> * HBASE-16060 1.x clients cannot access table state talking to
2.0
>>>>>>> cluster
>>>>>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x Master
and
>>>>>>> 1.x
>>>>>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
>>>>>>>
>>>>>>> * HBASE-17442 Move most of the replication related classes to
>>>>>>> hbase-server package
>>>>>>> The above would be good to do generally but it may make for ripples
>>>>>>> in
>>>>>>> API so would be good to do now.
>>>>>>>
>>>>>>> * HBASE-18106 Redo ProcedureInfo and LockInfo
>>>>>>> Balazs is working on this. The idea is that we avoid adding two
new
>>>>>>> types
>>>>>>> to our API, two types that are nought but curtailed, read-only
views
>>>>>>> on
>>>>>>> internals. Input if you have time appreciated.
>>>>>>>
>>>>>>> * HBASE-18596 A hbase1 cluster should be able to replicate to
a
>>>>>>> hbase2
>>>>>>> cluster; verify
>>>>>>> Esteban is looking at this one
>>>>>>>
>>>>>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in core
>>>>>>> * HBASE-17143 Scan improvement
>>>>>>>
>>>>>>> Our Coprocessor Interface needs a tough edit. It exposes
>>>>>>> implementations
>>>>>>> marked audience Private and returns implementations rather than
>>>>>>> Interfaces.
>>>>>>> In a few locations, we allow returning an alternate implementation
>>>>>>> altogether which is probably something we don't want a CP doing.
To
>>>>>>> that
>>>>>>> end, the following issues started by Duo and Anoop need to be
taken
>>>>>>> to
>>>>>>> the
>>>>>>> finish line; ideally they'd have an owner:
>>>>>>>
>>>>>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0 release
<= The
>>>>>>> umbrella issue.
>>>>>>> * HBASE-18298 RegionServerServices Interface cleanup for CP expose
>>>>>>> * HBASE-16769 Deprecate/remove PB references from MasterObserver
and
>>>>>>> RegionServerObserver
>>>>>>>
>>>>>>>
>>>>>>> Nice-to-haves:
>>>>>>>
>>>>>>> * HBASE-15284 Make TimeRange constructors IA.Private and remove
>>>>>>> unused
>>>>>>> TimeRange constructors
>>>>>>>
>>>>>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow() references
>>>>>>> existing in the code
>>>>>>> This is the end of an old long-running project moving up on to
Cell
>>>>>>> Interface. We think it is done but for a few little items (deprecate
>>>>>>> KV
>>>>>>> methods in MR and provide Cell versions instead...)
>>>>>>>
>>>>>>> * HBASE-13271 Table#puts(List<Put>) operation is indeterminate;
needs
>>>>>>> fixing
>>>>>>>
>>>>>>> * HBASE-13346 Clean up Filter package for post 1.0
>>>>>>>
>>>>>>> * HBASE-14255 Simplify Cell creation post 1.0
>>>>>>> * HBASE-14997
>>>>>>> Move compareOp and Comparators out of filter to client package
>>>>>>>
>>>>>>> * HBASE-13740 Stop using Hadoop private interfaces
>>>>>>>
>>>>>>> What about:
>>>>>>>
>>>>>>> * HBASE-18601 Remove Htrace 3.2
>>>>>>> As has been noted, the HTrace API is our 'trace' API.
>>>>>>>
>>>>>>> If interested in any of the above and you need a legup, just
ask in
>>>>>>> the
>>>>>>> issue and I'll be by....
>>>>>>>
>>>>>>> Thanks,
>>>>>>> St.Ack
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <stack@duboce.net>
wrote:
>>>>>>>
>>>>>>> Heads-up:
>>>>>>>
>>>>>>>>
>>>>>>>> I'm about to put up an hbase-2.0.0-alpha2 Release Candidate.
Theme
>>>>>>>> is
>>>>>>>> updated dependencies, reliance on relocated popular libs
(guava,
>>>>>>>> netty,
>>>>>>>> protobuf), purge of checked-in generated src, and
>>>>>>>> master-carries-no-regions
>>>>>>>> by default.
>>>>>>>>
>>>>>>>> alpha3 I hope will follow soon after (end-of-August?). Its
theme
>>>>>>>> will
>>>>>>>> be
>>>>>>>> settling the APIs and compatibility (At first blush, we are
not
>>>>>>>> looking too
>>>>>>>> bad; our Sean ran some tests over weekend that have hbase-1
client
>>>>>>>> running
>>>>>>>> against an hbase-2 cluster....). The Coprocessor Interface
revamp
>>>>>>>> should be
>>>>>>>> done by alpha3 (i.e. returning Interfaces rather than
>>>>>>>> Implementations, and
>>>>>>>> our shutdown of CPs accessing classes in hbase marked
>>>>>>>> InterfaceAudience).
>>>>>>>> We'll also have purged thirdparty classes from our API; e.g.
guava
>>>>>>>> 0.12
>>>>>>>> Service showing through in our replication API and protobufs
in
>>>>>>>> Admin
>>>>>>>> Interface. On alpha3, we will have to do a bunch of outreach
to make
>>>>>>>> sure
>>>>>>>> our downstreamers are up on what is coming down the pipe.
>>>>>>>>
>>>>>>>> Beta1 in mid-September?
>>>>>>>>
>>>>>>>> I encourage you to check out the items marked for hbase2:
>>>>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12327188
>>>>>>>> Edit
>>>>>>>> as
>>>>>>>> you see appropriate. Punt if you know the JIRA will not get
any
>>>>>>>> attention
>>>>>>>> in next month or so.
>>>>>>>>
>>>>>>>> A bunch of issues marked blocker are unassigned. I'll leave
them as
>>>>>>>> is
>>>>>>>> another while but I'll boot them soon.
>>>>>>>>
>>>>>>>> While I have your attention:
>>>>>>>>
>>>>>>>> + I think we should leave thrift version at 0.9.3. Moving
hbase
>>>>>>>> thrift
>>>>>>>> to 0.10.0 will break existing clients. The change is easy
enough if
>>>>>>>> folks
>>>>>>>> need to upgrade their hbase thrift. See HBASE-18591.
>>>>>>>> + Upgrade from 0.94 is disallowed. You have to get to 1.0
first
>>>>>>>> (0.98?).
>>>>>>>>
>>>>>>>> St.Ack
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <stack@duboce.net>
wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <elserj@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On 7/31/17 9:00 AM, Stack wrote:
>>>>>>>>>>
>>>>>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser<elserj@apache.org>
>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> ...
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> I like the idea of this also hitting 2.0
as it would make the
>>>>>>>>>>>> feature a
>>>>>>>>>>>> bit more "real", but am obviously a little
nervous (I have no
>>>>>>>>>>>> reason
>>>>>>>>>>>> to be
>>>>>>>>>>>> nervous though). I am pretty happy with the
feature in terms of
>>>>>>>>>>>> how
>>>>>>>>>>>> much it
>>>>>>>>>>>> is covered via testing.
>>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Sounds good to me. Whats involved? Backport?
If so, +1 Josh.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Last think on space quota says that need doc
too. See 'Space
>>>>>>>>>>> Quota' in
>>>>>>>>>>> here:
>>>>>>>>>>> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9i
>>>>>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5
>>>>>>>>>>> Does this little section need an update Josh?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> S
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Yep, just a couple of cherry-picks. Good test
coverage and some
>>>>>>>>>> docs
>>>>>>>>>> already included for 17748.  Happy to put that on
my plate if
>>>>>>>>>> you're good
>>>>>>>>>> with it. I can reasonably assume that no one is against
it :)
>>>>>>>>>>
>>>>>>>>>> I think I had knocked out docs for the "phase 1"
stuff before we
>>>>>>>>>> merged it in from the original feature branch. I'll
double check
>>>>>>>>>> and update
>>>>>>>>>> the gdoc. Perhaps this was just a timing thing.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks Josh,
>>>>>>>>> S
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message