Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: <CADcMMgFi+BaQd14iMTTgSjW6sBDfK6sFM7U70VGeDV09rZ_6GQ@mail.gmail.com>
References: <64DC0B0F-E4F4-4057-B6D4-99F1AB870D19@gmail.com>
 <CA+RK=_D_8SiyqVT9RJt6zjOwdxU2TeRoKUzeRJGwBBRcNH2j=w@mail.gmail.com>
 <CADcMMgEdJvxKSR3wvru-Q+JQZfni3jK5t6DuBVj1GFn6F=fksg@mail.gmail.com>
 <CADcMMgE0_ASBM7JZcgPDLdhU6kCWVeD5JUKpjh4wrb-6xKd=xg@mail.gmail.com>
 <CAAAYAnO1uunOUuBu7PZ8=1dqF0oj8yM=e1CA-wgSXWyVQL5CVw@mail.gmail.com>
 <CADcMMgG6-fo5O13KdKk23fO4XWwttzg0c4WSV6NxUQP=4CYfQA@mail.gmail.com>
 <CADcMMgHvZDvPjfVBQOH9b0g8=0wCAfejMnM7Eut0gvnEKrwfew@mail.gmail.com>
 <5627f4ff-203e-8ef9-4915-68f63a8c18e5@apache.org> <CADcMMgHq_FZzbZ0i1-JO=8G-whTFUZ+R8GPn7wfb5-am=6Qx-Q@mail.gmail.com>
 <090b5156-0850-07e7-0c33-c252a917476b@apache.org> <CADcMMgHhuzyboizYiOV7r7dO+ZVbuzBwS2Z8vPZAekSkJ18W9w@mail.gmail.com>
 <CADcMMgGGrvGcoCiLqQ5kO_TEcDJ7YV=B26i6f39OqJMA6X9-MQ@mail.gmail.com>
 <CADcMMgEm+aJDLz2jx6FQVR5Kq4dVZKfer45Um895Rkzu9Y3kJw@mail.gmail.com>
 <CADcMMgFf9_e1vuSZYTGP6uA93DEVH086q+7OnTfzjxqufaxVvw@mail.gmail.com>
 <CADcMMgEPioyM+xQMhGSgjd5LBY478rtEHMeV7eb8Parun7cn5w@mail.gmail.com>
 <0988dc33-b7ee-42c3-d2b0-e3834d969f3c@apache.org> <CADcMMgGgJ-msZve6SOsVGy_So1g0WseSukuK+ecEyDzALHkh+w@mail.gmail.com>
 <CAAT7MkpGH0fNX0V9PwfxBFYJX2rtV7FiL5GtEFmG=SYyedDVdA@mail.gmail.com> <CADcMMgFi+BaQd14iMTTgSjW6sBDfK6sFM7U70VGeDV09rZ_6GQ@mail.gmail.com>
From: ramkrishna vasudevan <ramkrishna.s.vasudevan@gmail.com>
Date: Tue, 24 Oct 2017 12:40:19 +0530
Message-ID: <CAAT7Mkqoz4EOFyoihneSy3eZ079a79-ZPfeUNLWqmi8F2Xe5sQ@mail.gmail.com>
Subject: Re: Moving 2.0 forward
To: "dev@hbase.apache.org" <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary="94eb2c058680ca9796055c45a1d3"
archived-at: Tue, 24 Oct 2017 07:10:37 -0000

--94eb2c058680ca9796055c45a1d3
Content-Type: text/plain; charset="UTF-8"

Thanks Stack. Started working on it. Will post a patch ASAP.

On Tue, Oct 24, 2017 at 11:38 AM, Stack <stack@duboce.net> wrote:

> That'd be a good one to get in Ram.
> S
>
> On Mon, Oct 23, 2017 at 9:42 PM, ramkrishna vasudevan <
> ramkrishna.s.vasudevan@gmail.com> wrote:
>
> > Hi Stack
> >
> > Do you want HBASE-18995 before the alpha-4 (REmoving exposed internal
> APIs
> > from CellUtil)? Because you had mentioned no more API changes. If so I
> will
> > start making changes and put up a patch ASAP.
> >
> > Regards
> > Ram
> >
> > On Tue, Oct 24, 2017 at 3:22 AM, Stack <stack@duboce.net> wrote:
> >
> > > On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <elserj@apache.org>
> wrote:
> > >
> > > > +1
> > > >
> > > > I was trying to work on helping out on the outstanding alpha-4 stuff
> > last
> > > > week -- will be continuing to try to do the same this week.
> > > >
> > > > If you need any help, Stack, or if others need reviews where I
> haven't
> > > > noticed on my own: feel free to @mention me.
> > > >
> > > >
> > > Thanks for the offer Josh. All items seem assigned and are being
> actively
> > > worked on. If you get a moment, reviews by you (or anyone else) helps
> > move
> > > the process along.
> > >
> > > We need to merge in HBASE-18410 branch to pick up Filter improvements.
> > Then
> > > HBASE-13346 can go in.
> > >
> > > You are already helping out on HBASE-18906, thanks. Looks like that
> will
> > be
> > > addressed by other alpha-4s about to land.
> > >
> > > St.Ack
> > > TODOs: https://issues.apache.org/jira/projects/HBASE/versions/12341594
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > > On 10/23/17 12:53 PM, Stack wrote:
> > > >
> > > >> (Reviving this thread)
> > > >>
> > > >> Lets push out alpha-4 this week. Alpha-4 is the release that has the
> > > >> refactor of the Coprocessor API shutting down access to internals
> > marked
> > > >> InterfaceAudience.Private.
> > > >>
> > > >> The outstanding list is here:
> > > >> https://issues.apache.org/jira/projects/HBASE/versions/12341594
> > > >>
> > > >> Please push in anything marked alpha-4 that belongs to you.
> > > >>
> > > >> If issue, talk out loud on this thread. If you need a review to land
> > an
> > > >> item, shout on the issue and here; we'll help you out.
> > > >>
> > > >> As is, items are coming along nicely I'd say. We need to merge the
> > > filter
> > > >> branch -- HBASE-18410 -- so APIs are finished for hbase2.
> > > >>
> > > >> Post alpha-4, we'll have to hunt down our downstreamers and help
> them
> > > test
> > > >> on top of alpha-4 so rolling into beta-1, we have confidence our
> > > >> downstreamers know what to expect (or we discover what we missed
> > BEFORE
> > > we
> > > >> beta-1).
> > > >>
> > > >> Thanks for time,
> > > >> S
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On Fri, Sep 8, 2017 at 2:04 PM, Stack <stack@duboce.net> wrote:
> > > >>
> > > >> I'll put up an alpha3 RC Monday, probably Monday night. That should
> be
> > > >>> time, if we all sprint, for the public-facing API fixes to be done.
> > > >>>
> > > >>> I had a bunch of Coprocessor refactor and fixup scheduled for
> alpha3
> > > but
> > > >>> it is plain that more time is needed (in spite of valiant effort so
> > far
> > > >>> by
> > > >>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4 whose
> > > theme
> > > >>> is
> > > >>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the
> > > following
> > > >>> week.
> > > >>>
> > > >>> We should then be ready for beta (beta == no new features, no API
> > > >>> changes,
> > > >>> just fixes).
> > > >>>
> > > >>> Thanks,
> > > >>> St.Ack
> > > >>>
> > > >>>
> > > >>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <stack@duboce.net> wrote:
> > > >>>
> > > >>> I put up the hbase-2.0.0-alpha2 release candidate. Please vote on
> it.
> > > >>>>
> > > >>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I hope to
> get
> > a
> > > >>>> release out in the next week or so.
> > > >>>>
> > > >>>> I did a weeding of 2.0.0 issues over the last day. If folks are
> > > >>>> interested in helping out, below are the items I think we need
> done
> > > for
> > > >>>> alpha3 (below are at least 'Critical' status, are API possibly
> > > altering
> > > >>>> items, and are absent those JIRAs that are making active progress,
> > > i.e.
> > > >>>> the
> > > >>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that needs
> > > >>>> doing is
> > > >>>> what Andrew did comparing 1.3. and 1.4 APIs
> > > >>>>
> > > >>>> * HBASE-18622 Mitigate compatibility concerns between branch-1 and
> > > >>>> branch-2
> > > >>>> This is to do what Andrew did between 1.3 and 1.4 branches only do
> > it
> > > >>>> between branch-1 and branch-2.
> > > >>>>
> > > >>>> * HBASE-10462 Recategorize some of the client facing Public /
> > Private
> > > >>>> interfaces
> > > >>>> This one is almost done. It could do with a finish, attention to
> the
> > > >>>> items in last comment, and then our codebase could do with another
> > > sweep
> > > >>>> after the spirit of this issue since a bunch has gone in since the
> > > pass
> > > >>>> that was the basis of this issue.
> > > >>>>
> > > >>>> * HBASE-10504 Define Replication Interface
> > > >>>> I was going to take a crack at this as part of the revamp forced
> by
> > > >>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's
> Service'
> > > but
> > > >>>> if
> > > >>>> anyone else is interested, be my guest.
> > > >>>>
> > > >>>> * HBASE-14996 Some more API cleanup for 2.0
> > > >>>> Has a bunch of subtasks, some of which are being worked on. Needs
> > > >>>> finishing.
> > > >>>>
> > > >>>> * HBASE-14998 Unify synchronous and asynchronous methods in Admin
> > and
> > > >>>> cleanup
> > > >>>> Needs a pass. Small issue I think. Could also look at new
> > AsyncClient
> > > >>>> and
> > > >>>> make sure symmetry.
> > > >>>>
> > > >>>> * HBASE-15607 Remove PB references from Admin for 2.0
> > > >>>> Predicated on result of an ongoing DISCUSSION thread but needs to
> be
> > > >>>> done.
> > > >>>>
> > > >>>> Rolling upgrade will have implications for our API. Would be good
> to
> > > try
> > > >>>> it and figure what needs fixup (as said above, according to trial
> by
> > > >>>> Sean,
> > > >>>> we might not be too bad here):
> > > >>>> * HBASE-16060 1.x clients cannot access table state talking to 2.0
> > > >>>> cluster
> > > >>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x Master
> and
> > > 1.x
> > > >>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
> > > >>>>
> > > >>>> * HBASE-17442 Move most of the replication related classes to
> > > >>>> hbase-server package
> > > >>>> The above would be good to do generally but it may make for
> ripples
> > in
> > > >>>> API so would be good to do now.
> > > >>>>
> > > >>>> * HBASE-18106 Redo ProcedureInfo and LockInfo
> > > >>>> Balazs is working on this. The idea is that we avoid adding two
> new
> > > >>>> types
> > > >>>> to our API, two types that are nought but curtailed, read-only
> views
> > > on
> > > >>>> internals. Input if you have time appreciated.
> > > >>>>
> > > >>>> * HBASE-18596 A hbase1 cluster should be able to replicate to a
> > hbase2
> > > >>>> cluster; verify
> > > >>>> Esteban is looking at this one
> > > >>>>
> > > >>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in core
> > > >>>> * HBASE-17143 Scan improvement
> > > >>>>
> > > >>>> Our Coprocessor Interface needs a tough edit. It exposes
> > > implementations
> > > >>>> marked audience Private and returns implementations rather than
> > > >>>> Interfaces.
> > > >>>> In a few locations, we allow returning an alternate implementation
> > > >>>> altogether which is probably something we don't want a CP doing.
> To
> > > that
> > > >>>> end, the following issues started by Duo and Anoop need to be
> taken
> > to
> > > >>>> the
> > > >>>> finish line; ideally they'd have an owner:
> > > >>>>
> > > >>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0 release <=
> > The
> > > >>>> umbrella issue.
> > > >>>> * HBASE-18298 RegionServerServices Interface cleanup for CP expose
> > > >>>> * HBASE-16769 Deprecate/remove PB references from MasterObserver
> and
> > > >>>> RegionServerObserver
> > > >>>>
> > > >>>>
> > > >>>> Nice-to-haves:
> > > >>>>
> > > >>>> * HBASE-15284 Make TimeRange constructors IA.Private and remove
> > unused
> > > >>>> TimeRange constructors
> > > >>>>
> > > >>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow() references
> > > >>>> existing in the code
> > > >>>> This is the end of an old long-running project moving up on to
> Cell
> > > >>>> Interface. We think it is done but for a few little items
> (deprecate
> > > KV
> > > >>>> methods in MR and provide Cell versions instead...)
> > > >>>>
> > > >>>> * HBASE-13271 Table#puts(List<Put>) operation is indeterminate;
> > needs
> > > >>>> fixing
> > > >>>>
> > > >>>> * HBASE-13346 Clean up Filter package for post 1.0
> > > >>>>
> > > >>>> * HBASE-14255 Simplify Cell creation post 1.0
> > > >>>> * HBASE-14997
> > > >>>> Move compareOp and Comparators out of filter to client package
> > > >>>>
> > > >>>> * HBASE-13740 Stop using Hadoop private interfaces
> > > >>>>
> > > >>>> What about:
> > > >>>>
> > > >>>> * HBASE-18601 Remove Htrace 3.2
> > > >>>> As has been noted, the HTrace API is our 'trace' API.
> > > >>>>
> > > >>>> If interested in any of the above and you need a legup, just ask
> in
> > > the
> > > >>>> issue and I'll be by....
> > > >>>>
> > > >>>> Thanks,
> > > >>>> St.Ack
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <stack@duboce.net> wrote:
> > > >>>>
> > > >>>> Heads-up:
> > > >>>>>
> > > >>>>> I'm about to put up an hbase-2.0.0-alpha2 Release Candidate.
> Theme
> > is
> > > >>>>> updated dependencies, reliance on relocated popular libs (guava,
> > > netty,
> > > >>>>> protobuf), purge of checked-in generated src, and
> > > >>>>> master-carries-no-regions
> > > >>>>> by default.
> > > >>>>>
> > > >>>>> alpha3 I hope will follow soon after (end-of-August?). Its theme
> > will
> > > >>>>> be
> > > >>>>> settling the APIs and compatibility (At first blush, we are not
> > > >>>>> looking too
> > > >>>>> bad; our Sean ran some tests over weekend that have hbase-1
> client
> > > >>>>> running
> > > >>>>> against an hbase-2 cluster....). The Coprocessor Interface revamp
> > > >>>>> should be
> > > >>>>> done by alpha3 (i.e. returning Interfaces rather than
> > > Implementations,
> > > >>>>> and
> > > >>>>> our shutdown of CPs accessing classes in hbase marked
> > > >>>>> InterfaceAudience).
> > > >>>>> We'll also have purged thirdparty classes from our API; e.g.
> guava
> > > 0.12
> > > >>>>> Service showing through in our replication API and protobufs in
> > Admin
> > > >>>>> Interface. On alpha3, we will have to do a bunch of outreach to
> > make
> > > >>>>> sure
> > > >>>>> our downstreamers are up on what is coming down the pipe.
> > > >>>>>
> > > >>>>> Beta1 in mid-September?
> > > >>>>>
> > > >>>>> I encourage you to check out the items marked for hbase2:
> > > >>>>> https://issues.apache.org/jira/projects/HBASE/versions/12327188
> > Edit
> > > >>>>> as
> > > >>>>> you see appropriate. Punt if you know the JIRA will not get any
> > > >>>>> attention
> > > >>>>> in next month or so.
> > > >>>>>
> > > >>>>> A bunch of issues marked blocker are unassigned. I'll leave them
> as
> > > is
> > > >>>>> another while but I'll boot them soon.
> > > >>>>>
> > > >>>>> While I have your attention:
> > > >>>>>
> > > >>>>> + I think we should leave thrift version at 0.9.3. Moving hbase
> > > thrift
> > > >>>>> to 0.10.0 will break existing clients. The change is easy enough
> if
> > > >>>>> folks
> > > >>>>> need to upgrade their hbase thrift. See HBASE-18591.
> > > >>>>> + Upgrade from 0.94 is disallowed. You have to get to 1.0 first
> > > >>>>> (0.98?).
> > > >>>>>
> > > >>>>> St.Ack
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <stack@duboce.net> wrote:
> > > >>>>>
> > > >>>>>
> > > >>>>>>
> > > >>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <elserj@apache.org>
> > > wrote:
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>> On 7/31/17 9:00 AM, Stack wrote:
> > > >>>>>>>
> > > >>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser<elserj@apache.org
> >
> > > >>>>>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>> ...
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> I like the idea of this also hitting 2.0 as it would make the
> > > >>>>>>>>> feature a
> > > >>>>>>>>> bit more "real", but am obviously a little nervous (I have no
> > > >>>>>>>>> reason
> > > >>>>>>>>> to be
> > > >>>>>>>>> nervous though). I am pretty happy with the feature in terms
> of
> > > how
> > > >>>>>>>>> much it
> > > >>>>>>>>> is covered via testing.
> > > >>>>>>>>>
> > > >>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> Sounds good to me. Whats involved? Backport? If so, +1 Josh.
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> Last think on space quota says that need doc too. See 'Space
> > > Quota'
> > > >>>>>>>> in
> > > >>>>>>>> here:
> > > >>>>>>>> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9i
> > > >>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5
> > > >>>>>>>> Does this little section need an update Josh?
> > > >>>>>>>>
> > > >>>>>>>> Thanks,
> > > >>>>>>>> S
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>> Yep, just a couple of cherry-picks. Good test coverage and some
> > > docs
> > > >>>>>>> already included for 17748.  Happy to put that on my plate if
> > > you're
> > > >>>>>>> good
> > > >>>>>>> with it. I can reasonably assume that no one is against it :)
> > > >>>>>>>
> > > >>>>>>> I think I had knocked out docs for the "phase 1" stuff before
> we
> > > >>>>>>> merged it in from the original feature branch. I'll double
> check
> > > and
> > > >>>>>>> update
> > > >>>>>>> the gdoc. Perhaps this was just a timing thing.
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>> Thanks Josh,
> > > >>>>>> S
> > > >>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> >
>

--94eb2c058680ca9796055c45a1d3--