hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Moving 2.0 forward
Date Mon, 23 Oct 2017 21:52:12 GMT
On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <elserj@apache.org> wrote:

> +1
>
> I was trying to work on helping out on the outstanding alpha-4 stuff last
> week -- will be continuing to try to do the same this week.
>
> If you need any help, Stack, or if others need reviews where I haven't
> noticed on my own: feel free to @mention me.
>
>
Thanks for the offer Josh. All items seem assigned and are being actively
worked on. If you get a moment, reviews by you (or anyone else) helps move
the process along.

We need to merge in HBASE-18410 branch to pick up Filter improvements. Then
HBASE-13346 can go in.

You are already helping out on HBASE-18906, thanks. Looks like that will be
addressed by other alpha-4s about to land.

St.Ack
TODOs: https://issues.apache.org/jira/projects/HBASE/versions/12341594









> On 10/23/17 12:53 PM, Stack wrote:
>
>> (Reviving this thread)
>>
>> Lets push out alpha-4 this week. Alpha-4 is the release that has the
>> refactor of the Coprocessor API shutting down access to internals marked
>> InterfaceAudience.Private.
>>
>> The outstanding list is here:
>> https://issues.apache.org/jira/projects/HBASE/versions/12341594
>>
>> Please push in anything marked alpha-4 that belongs to you.
>>
>> If issue, talk out loud on this thread. If you need a review to land an
>> item, shout on the issue and here; we'll help you out.
>>
>> As is, items are coming along nicely I'd say. We need to merge the filter
>> branch -- HBASE-18410 -- so APIs are finished for hbase2.
>>
>> Post alpha-4, we'll have to hunt down our downstreamers and help them test
>> on top of alpha-4 so rolling into beta-1, we have confidence our
>> downstreamers know what to expect (or we discover what we missed BEFORE we
>> beta-1).
>>
>> Thanks for time,
>> S
>>
>>
>>
>>
>>
>> On Fri, Sep 8, 2017 at 2:04 PM, Stack <stack@duboce.net> wrote:
>>
>> I'll put up an alpha3 RC Monday, probably Monday night. That should be
>>> time, if we all sprint, for the public-facing API fixes to be done.
>>>
>>> I had a bunch of Coprocessor refactor and fixup scheduled for alpha3 but
>>> it is plain that more time is needed (in spite of valiant effort so far
>>> by
>>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4 whose theme
>>> is
>>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the following
>>> week.
>>>
>>> We should then be ready for beta (beta == no new features, no API
>>> changes,
>>> just fixes).
>>>
>>> Thanks,
>>> St.Ack
>>>
>>>
>>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <stack@duboce.net> wrote:
>>>
>>> I put up the hbase-2.0.0-alpha2 release candidate. Please vote on it.
>>>>
>>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I hope to get a
>>>> release out in the next week or so.
>>>>
>>>> I did a weeding of 2.0.0 issues over the last day. If folks are
>>>> interested in helping out, below are the items I think we need done for
>>>> alpha3 (below are at least 'Critical' status, are API possibly altering
>>>> items, and are absent those JIRAs that are making active progress, i.e.
>>>> the
>>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that needs
>>>> doing is
>>>> what Andrew did comparing 1.3. and 1.4 APIs
>>>>
>>>> * HBASE-18622 Mitigate compatibility concerns between branch-1 and
>>>> branch-2
>>>> This is to do what Andrew did between 1.3 and 1.4 branches only do it
>>>> between branch-1 and branch-2.
>>>>
>>>> * HBASE-10462 Recategorize some of the client facing Public / Private
>>>> interfaces
>>>> This one is almost done. It could do with a finish, attention to the
>>>> items in last comment, and then our codebase could do with another sweep
>>>> after the spirit of this issue since a bunch has gone in since the pass
>>>> that was the basis of this issue.
>>>>
>>>> * HBASE-10504 Define Replication Interface
>>>> I was going to take a crack at this as part of the revamp forced by
>>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's Service' but
>>>> if
>>>> anyone else is interested, be my guest.
>>>>
>>>> * HBASE-14996 Some more API cleanup for 2.0
>>>> Has a bunch of subtasks, some of which are being worked on. Needs
>>>> finishing.
>>>>
>>>> * HBASE-14998 Unify synchronous and asynchronous methods in Admin and
>>>> cleanup
>>>> Needs a pass. Small issue I think. Could also look at new AsyncClient
>>>> and
>>>> make sure symmetry.
>>>>
>>>> * HBASE-15607 Remove PB references from Admin for 2.0
>>>> Predicated on result of an ongoing DISCUSSION thread but needs to be
>>>> done.
>>>>
>>>> Rolling upgrade will have implications for our API. Would be good to try
>>>> it and figure what needs fixup (as said above, according to trial by
>>>> Sean,
>>>> we might not be too bad here):
>>>> * HBASE-16060 1.x clients cannot access table state talking to 2.0
>>>> cluster
>>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x Master and 1.x
>>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
>>>>
>>>> * HBASE-17442 Move most of the replication related classes to
>>>> hbase-server package
>>>> The above would be good to do generally but it may make for ripples in
>>>> API so would be good to do now.
>>>>
>>>> * HBASE-18106 Redo ProcedureInfo and LockInfo
>>>> Balazs is working on this. The idea is that we avoid adding two new
>>>> types
>>>> to our API, two types that are nought but curtailed, read-only views on
>>>> internals. Input if you have time appreciated.
>>>>
>>>> * HBASE-18596 A hbase1 cluster should be able to replicate to a hbase2
>>>> cluster; verify
>>>> Esteban is looking at this one
>>>>
>>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in core
>>>> * HBASE-17143 Scan improvement
>>>>
>>>> Our Coprocessor Interface needs a tough edit. It exposes implementations
>>>> marked audience Private and returns implementations rather than
>>>> Interfaces.
>>>> In a few locations, we allow returning an alternate implementation
>>>> altogether which is probably something we don't want a CP doing. To that
>>>> end, the following issues started by Duo and Anoop need to be taken to
>>>> the
>>>> finish line; ideally they'd have an owner:
>>>>
>>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0 release <= The
>>>> umbrella issue.
>>>> * HBASE-18298 RegionServerServices Interface cleanup for CP expose
>>>> * HBASE-16769 Deprecate/remove PB references from MasterObserver and
>>>> RegionServerObserver
>>>>
>>>>
>>>> Nice-to-haves:
>>>>
>>>> * HBASE-15284 Make TimeRange constructors IA.Private and remove unused
>>>> TimeRange constructors
>>>>
>>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow() references
>>>> existing in the code
>>>> This is the end of an old long-running project moving up on to Cell
>>>> Interface. We think it is done but for a few little items (deprecate KV
>>>> methods in MR and provide Cell versions instead...)
>>>>
>>>> * HBASE-13271 Table#puts(List<Put>) operation is indeterminate; needs
>>>> fixing
>>>>
>>>> * HBASE-13346 Clean up Filter package for post 1.0
>>>>
>>>> * HBASE-14255 Simplify Cell creation post 1.0
>>>> * HBASE-14997
>>>> Move compareOp and Comparators out of filter to client package
>>>>
>>>> * HBASE-13740 Stop using Hadoop private interfaces
>>>>
>>>> What about:
>>>>
>>>> * HBASE-18601 Remove Htrace 3.2
>>>> As has been noted, the HTrace API is our 'trace' API.
>>>>
>>>> If interested in any of the above and you need a legup, just ask in the
>>>> issue and I'll be by....
>>>>
>>>> Thanks,
>>>> St.Ack
>>>>
>>>>
>>>>
>>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <stack@duboce.net> wrote:
>>>>
>>>> Heads-up:
>>>>>
>>>>> I'm about to put up an hbase-2.0.0-alpha2 Release Candidate. Theme is
>>>>> updated dependencies, reliance on relocated popular libs (guava, netty,
>>>>> protobuf), purge of checked-in generated src, and
>>>>> master-carries-no-regions
>>>>> by default.
>>>>>
>>>>> alpha3 I hope will follow soon after (end-of-August?). Its theme will
>>>>> be
>>>>> settling the APIs and compatibility (At first blush, we are not
>>>>> looking too
>>>>> bad; our Sean ran some tests over weekend that have hbase-1 client
>>>>> running
>>>>> against an hbase-2 cluster....). The Coprocessor Interface revamp
>>>>> should be
>>>>> done by alpha3 (i.e. returning Interfaces rather than Implementations,
>>>>> and
>>>>> our shutdown of CPs accessing classes in hbase marked
>>>>> InterfaceAudience).
>>>>> We'll also have purged thirdparty classes from our API; e.g. guava 0.12
>>>>> Service showing through in our replication API and protobufs in Admin
>>>>> Interface. On alpha3, we will have to do a bunch of outreach to make
>>>>> sure
>>>>> our downstreamers are up on what is coming down the pipe.
>>>>>
>>>>> Beta1 in mid-September?
>>>>>
>>>>> I encourage you to check out the items marked for hbase2:
>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12327188 Edit
>>>>> as
>>>>> you see appropriate. Punt if you know the JIRA will not get any
>>>>> attention
>>>>> in next month or so.
>>>>>
>>>>> A bunch of issues marked blocker are unassigned. I'll leave them as is
>>>>> another while but I'll boot them soon.
>>>>>
>>>>> While I have your attention:
>>>>>
>>>>> + I think we should leave thrift version at 0.9.3. Moving hbase thrift
>>>>> to 0.10.0 will break existing clients. The change is easy enough if
>>>>> folks
>>>>> need to upgrade their hbase thrift. See HBASE-18591.
>>>>> + Upgrade from 0.94 is disallowed. You have to get to 1.0 first
>>>>> (0.98?).
>>>>>
>>>>> St.Ack
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <stack@duboce.net> wrote:
>>>>>
>>>>>
>>>>>>
>>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <elserj@apache.org>
wrote:
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> On 7/31/17 9:00 AM, Stack wrote:
>>>>>>>
>>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser<elserj@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> ...
>>>>>>>>
>>>>>>>>>
>>>>>>>>> I like the idea of this also hitting 2.0 as it would
make the
>>>>>>>>> feature a
>>>>>>>>> bit more "real", but am obviously a little nervous (I
have no
>>>>>>>>> reason
>>>>>>>>> to be
>>>>>>>>> nervous though). I am pretty happy with the feature in
terms of how
>>>>>>>>> much it
>>>>>>>>> is covered via testing.
>>>>>>>>>
>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Sounds good to me. Whats involved? Backport? If so, +1
Josh.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Last think on space quota says that need doc too. See 'Space
Quota'
>>>>>>>> in
>>>>>>>> here:
>>>>>>>> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9i
>>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5
>>>>>>>> Does this little section need an update Josh?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> S
>>>>>>>>
>>>>>>>>
>>>>>>> Yep, just a couple of cherry-picks. Good test coverage and some
docs
>>>>>>> already included for 17748.  Happy to put that on my plate if
you're
>>>>>>> good
>>>>>>> with it. I can reasonably assume that no one is against it :)
>>>>>>>
>>>>>>> I think I had knocked out docs for the "phase 1" stuff before
we
>>>>>>> merged it in from the original feature branch. I'll double check
and
>>>>>>> update
>>>>>>> the gdoc. Perhaps this was just a timing thing.
>>>>>>>
>>>>>>>
>>>>>> Thanks Josh,
>>>>>> S
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message