hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ramkrishna vasudevan <ramkrishna.s.vasude...@gmail.com>
Subject Re: Moving 2.0 forward
Date Tue, 24 Oct 2017 04:42:03 GMT
Hi Stack

Do you want HBASE-18995 before the alpha-4 (REmoving exposed internal APIs
from CellUtil)? Because you had mentioned no more API changes. If so I will
start making changes and put up a patch ASAP.

Regards
Ram

On Tue, Oct 24, 2017 at 3:22 AM, Stack <stack@duboce.net> wrote:

> On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <elserj@apache.org> wrote:
>
> > +1
> >
> > I was trying to work on helping out on the outstanding alpha-4 stuff last
> > week -- will be continuing to try to do the same this week.
> >
> > If you need any help, Stack, or if others need reviews where I haven't
> > noticed on my own: feel free to @mention me.
> >
> >
> Thanks for the offer Josh. All items seem assigned and are being actively
> worked on. If you get a moment, reviews by you (or anyone else) helps move
> the process along.
>
> We need to merge in HBASE-18410 branch to pick up Filter improvements. Then
> HBASE-13346 can go in.
>
> You are already helping out on HBASE-18906, thanks. Looks like that will be
> addressed by other alpha-4s about to land.
>
> St.Ack
> TODOs: https://issues.apache.org/jira/projects/HBASE/versions/12341594
>
>
>
>
>
>
>
>
>
> > On 10/23/17 12:53 PM, Stack wrote:
> >
> >> (Reviving this thread)
> >>
> >> Lets push out alpha-4 this week. Alpha-4 is the release that has the
> >> refactor of the Coprocessor API shutting down access to internals marked
> >> InterfaceAudience.Private.
> >>
> >> The outstanding list is here:
> >> https://issues.apache.org/jira/projects/HBASE/versions/12341594
> >>
> >> Please push in anything marked alpha-4 that belongs to you.
> >>
> >> If issue, talk out loud on this thread. If you need a review to land an
> >> item, shout on the issue and here; we'll help you out.
> >>
> >> As is, items are coming along nicely I'd say. We need to merge the
> filter
> >> branch -- HBASE-18410 -- so APIs are finished for hbase2.
> >>
> >> Post alpha-4, we'll have to hunt down our downstreamers and help them
> test
> >> on top of alpha-4 so rolling into beta-1, we have confidence our
> >> downstreamers know what to expect (or we discover what we missed BEFORE
> we
> >> beta-1).
> >>
> >> Thanks for time,
> >> S
> >>
> >>
> >>
> >>
> >>
> >> On Fri, Sep 8, 2017 at 2:04 PM, Stack <stack@duboce.net> wrote:
> >>
> >> I'll put up an alpha3 RC Monday, probably Monday night. That should be
> >>> time, if we all sprint, for the public-facing API fixes to be done.
> >>>
> >>> I had a bunch of Coprocessor refactor and fixup scheduled for alpha3
> but
> >>> it is plain that more time is needed (in spite of valiant effort so far
> >>> by
> >>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4 whose
> theme
> >>> is
> >>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the
> following
> >>> week.
> >>>
> >>> We should then be ready for beta (beta == no new features, no API
> >>> changes,
> >>> just fixes).
> >>>
> >>> Thanks,
> >>> St.Ack
> >>>
> >>>
> >>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <stack@duboce.net> wrote:
> >>>
> >>> I put up the hbase-2.0.0-alpha2 release candidate. Please vote on it.
> >>>>
> >>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I hope to get
a
> >>>> release out in the next week or so.
> >>>>
> >>>> I did a weeding of 2.0.0 issues over the last day. If folks are
> >>>> interested in helping out, below are the items I think we need done
> for
> >>>> alpha3 (below are at least 'Critical' status, are API possibly
> altering
> >>>> items, and are absent those JIRAs that are making active progress,
> i.e.
> >>>> the
> >>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that needs
> >>>> doing is
> >>>> what Andrew did comparing 1.3. and 1.4 APIs
> >>>>
> >>>> * HBASE-18622 Mitigate compatibility concerns between branch-1 and
> >>>> branch-2
> >>>> This is to do what Andrew did between 1.3 and 1.4 branches only do it
> >>>> between branch-1 and branch-2.
> >>>>
> >>>> * HBASE-10462 Recategorize some of the client facing Public / Private
> >>>> interfaces
> >>>> This one is almost done. It could do with a finish, attention to the
> >>>> items in last comment, and then our codebase could do with another
> sweep
> >>>> after the spirit of this issue since a bunch has gone in since the
> pass
> >>>> that was the basis of this issue.
> >>>>
> >>>> * HBASE-10504 Define Replication Interface
> >>>> I was going to take a crack at this as part of the revamp forced by
> >>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's Service'
> but
> >>>> if
> >>>> anyone else is interested, be my guest.
> >>>>
> >>>> * HBASE-14996 Some more API cleanup for 2.0
> >>>> Has a bunch of subtasks, some of which are being worked on. Needs
> >>>> finishing.
> >>>>
> >>>> * HBASE-14998 Unify synchronous and asynchronous methods in Admin and
> >>>> cleanup
> >>>> Needs a pass. Small issue I think. Could also look at new AsyncClient
> >>>> and
> >>>> make sure symmetry.
> >>>>
> >>>> * HBASE-15607 Remove PB references from Admin for 2.0
> >>>> Predicated on result of an ongoing DISCUSSION thread but needs to be
> >>>> done.
> >>>>
> >>>> Rolling upgrade will have implications for our API. Would be good to
> try
> >>>> it and figure what needs fixup (as said above, according to trial by
> >>>> Sean,
> >>>> we might not be too bad here):
> >>>> * HBASE-16060 1.x clients cannot access table state talking to 2.0
> >>>> cluster
> >>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x Master and
> 1.x
> >>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
> >>>>
> >>>> * HBASE-17442 Move most of the replication related classes to
> >>>> hbase-server package
> >>>> The above would be good to do generally but it may make for ripples
in
> >>>> API so would be good to do now.
> >>>>
> >>>> * HBASE-18106 Redo ProcedureInfo and LockInfo
> >>>> Balazs is working on this. The idea is that we avoid adding two new
> >>>> types
> >>>> to our API, two types that are nought but curtailed, read-only views
> on
> >>>> internals. Input if you have time appreciated.
> >>>>
> >>>> * HBASE-18596 A hbase1 cluster should be able to replicate to a hbase2
> >>>> cluster; verify
> >>>> Esteban is looking at this one
> >>>>
> >>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in core
> >>>> * HBASE-17143 Scan improvement
> >>>>
> >>>> Our Coprocessor Interface needs a tough edit. It exposes
> implementations
> >>>> marked audience Private and returns implementations rather than
> >>>> Interfaces.
> >>>> In a few locations, we allow returning an alternate implementation
> >>>> altogether which is probably something we don't want a CP doing. To
> that
> >>>> end, the following issues started by Duo and Anoop need to be taken
to
> >>>> the
> >>>> finish line; ideally they'd have an owner:
> >>>>
> >>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0 release <=
The
> >>>> umbrella issue.
> >>>> * HBASE-18298 RegionServerServices Interface cleanup for CP expose
> >>>> * HBASE-16769 Deprecate/remove PB references from MasterObserver and
> >>>> RegionServerObserver
> >>>>
> >>>>
> >>>> Nice-to-haves:
> >>>>
> >>>> * HBASE-15284 Make TimeRange constructors IA.Private and remove unused
> >>>> TimeRange constructors
> >>>>
> >>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow() references
> >>>> existing in the code
> >>>> This is the end of an old long-running project moving up on to Cell
> >>>> Interface. We think it is done but for a few little items (deprecate
> KV
> >>>> methods in MR and provide Cell versions instead...)
> >>>>
> >>>> * HBASE-13271 Table#puts(List<Put>) operation is indeterminate;
needs
> >>>> fixing
> >>>>
> >>>> * HBASE-13346 Clean up Filter package for post 1.0
> >>>>
> >>>> * HBASE-14255 Simplify Cell creation post 1.0
> >>>> * HBASE-14997
> >>>> Move compareOp and Comparators out of filter to client package
> >>>>
> >>>> * HBASE-13740 Stop using Hadoop private interfaces
> >>>>
> >>>> What about:
> >>>>
> >>>> * HBASE-18601 Remove Htrace 3.2
> >>>> As has been noted, the HTrace API is our 'trace' API.
> >>>>
> >>>> If interested in any of the above and you need a legup, just ask in
> the
> >>>> issue and I'll be by....
> >>>>
> >>>> Thanks,
> >>>> St.Ack
> >>>>
> >>>>
> >>>>
> >>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <stack@duboce.net> wrote:
> >>>>
> >>>> Heads-up:
> >>>>>
> >>>>> I'm about to put up an hbase-2.0.0-alpha2 Release Candidate. Theme
is
> >>>>> updated dependencies, reliance on relocated popular libs (guava,
> netty,
> >>>>> protobuf), purge of checked-in generated src, and
> >>>>> master-carries-no-regions
> >>>>> by default.
> >>>>>
> >>>>> alpha3 I hope will follow soon after (end-of-August?). Its theme
will
> >>>>> be
> >>>>> settling the APIs and compatibility (At first blush, we are not
> >>>>> looking too
> >>>>> bad; our Sean ran some tests over weekend that have hbase-1 client
> >>>>> running
> >>>>> against an hbase-2 cluster....). The Coprocessor Interface revamp
> >>>>> should be
> >>>>> done by alpha3 (i.e. returning Interfaces rather than
> Implementations,
> >>>>> and
> >>>>> our shutdown of CPs accessing classes in hbase marked
> >>>>> InterfaceAudience).
> >>>>> We'll also have purged thirdparty classes from our API; e.g. guava
> 0.12
> >>>>> Service showing through in our replication API and protobufs in
Admin
> >>>>> Interface. On alpha3, we will have to do a bunch of outreach to
make
> >>>>> sure
> >>>>> our downstreamers are up on what is coming down the pipe.
> >>>>>
> >>>>> Beta1 in mid-September?
> >>>>>
> >>>>> I encourage you to check out the items marked for hbase2:
> >>>>> https://issues.apache.org/jira/projects/HBASE/versions/12327188
Edit
> >>>>> as
> >>>>> you see appropriate. Punt if you know the JIRA will not get any
> >>>>> attention
> >>>>> in next month or so.
> >>>>>
> >>>>> A bunch of issues marked blocker are unassigned. I'll leave them
as
> is
> >>>>> another while but I'll boot them soon.
> >>>>>
> >>>>> While I have your attention:
> >>>>>
> >>>>> + I think we should leave thrift version at 0.9.3. Moving hbase
> thrift
> >>>>> to 0.10.0 will break existing clients. The change is easy enough
if
> >>>>> folks
> >>>>> need to upgrade their hbase thrift. See HBASE-18591.
> >>>>> + Upgrade from 0.94 is disallowed. You have to get to 1.0 first
> >>>>> (0.98?).
> >>>>>
> >>>>> St.Ack
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <stack@duboce.net> wrote:
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <elserj@apache.org>
> wrote:
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> On 7/31/17 9:00 AM, Stack wrote:
> >>>>>>>
> >>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser<elserj@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> ...
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> I like the idea of this also hitting 2.0 as it would
make the
> >>>>>>>>> feature a
> >>>>>>>>> bit more "real", but am obviously a little nervous
(I have no
> >>>>>>>>> reason
> >>>>>>>>> to be
> >>>>>>>>> nervous though). I am pretty happy with the feature
in terms of
> how
> >>>>>>>>> much it
> >>>>>>>>> is covered via testing.
> >>>>>>>>>
> >>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Sounds good to me. Whats involved? Backport? If
so, +1 Josh.
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> Last think on space quota says that need doc too. See
'Space
> Quota'
> >>>>>>>> in
> >>>>>>>> here:
> >>>>>>>> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9i
> >>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5
> >>>>>>>> Does this little section need an update Josh?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> S
> >>>>>>>>
> >>>>>>>>
> >>>>>>> Yep, just a couple of cherry-picks. Good test coverage and
some
> docs
> >>>>>>> already included for 17748.  Happy to put that on my plate
if
> you're
> >>>>>>> good
> >>>>>>> with it. I can reasonably assume that no one is against
it :)
> >>>>>>>
> >>>>>>> I think I had knocked out docs for the "phase 1" stuff before
we
> >>>>>>> merged it in from the original feature branch. I'll double
check
> and
> >>>>>>> update
> >>>>>>> the gdoc. Perhaps this was just a timing thing.
> >>>>>>>
> >>>>>>>
> >>>>>> Thanks Josh,
> >>>>>> S
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message