hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: Moving 2.0 forward
Date Fri, 10 Mar 2017 19:07:59 GMT
I agree AMv2/Pv2 is almost finished and this makes sense as 2.0.

Agreed, upgrade issues need to be addressed, but I have been assuming this
will be done after branching during the stabilization and polishing part.
Need the branch-2 first to stabilize and polish.

Most of the rest justifies a 3.0, assuming we are agreed that a 2.0 is
overdue. I think we have that. If we are lamenting the long time coming for
a 2.0, why not go the other way? Push out unfinished work to 3.0, and
attempt to apply the same desired effort at getting that out in a 3.0 ASAP
as opposed to landing them in a 2.0.


On Fri, Mar 10, 2017 at 9:54 AM, Stack <stack@duboce.net> wrote:

> On Mon, Mar 6, 2017 at 6:15 PM, Enis Söztutar <enis.soz@gmail.com> wrote:
>
> > Thanks Stack for the nice writeup.
> >
> > I think we should shoot for an alpha release sooner than 2 months. It
> gives
> > a test target, and will be a great way to test-drive and push for the
> > release vehicles (packing, hadoop3, license issues, etc) and also create
> > some well-deserved excitement. I can help with that.
> >
> >
> Looking at the list of items in the Core and Tasks (with an eye on the
> concurrent thread "A suggestion for releasing major versions faster...") ,
> it might be time for a branch -- end of next week or better, the
> end-of-the-month? We could push an Alpha soon after?
>
> As I see it, the blockers on hbase2 are:
>
> + AMv2/Pv2. Its been trickling in for a year or more now. We are close to
> throwing the switch on move up on to new AMv2, cornerstone of 1M-regions
> effort and fast assignment. There'll be fall-out but we'll be up on a more
> solid intent-log, no-zk basis. Could put this off to hbase-3 I suppose but
> its all-over the code base half-done; it'll rot if we just leave it.
> + Rolling Restart from branch-1 to branch-2. Has to work. Can't have a
> singularity. No work done.
> + Master carrying hbase:meta. Currently it does by default. We have a
> running thread on pros and cons still to finish. If master is to carry
> hbase:meta, there is work to do. If not, there is work to do.
> + Updating dependencies and shading the critical likely-clashing libs
> (netty, guava). No work done.
>
> Other super important stuff that we should fix (criticals) but that don't
> warrant hold-up of the release are:
>
> + Narrative around client operation timeout (Phil Yang doing great work
> here rationalizing our timeout mess)
> + Perf (async hdfs client, netty rpcserver, G1GC default, etc.) and
> updating defaults.
> + Hadoop3 (EC, etc.)
>
> I don't make mention of criticals in above list that I have confidence will
> land in time (inmemory compaction, the offheaping work). I leave aside
> criticals that are not getting love (hbase-replication, FS Redo, though it
> seems like hbase-spark might see some uptake -- thanks Jerry and crew).
>
> A major release is an opportunity for big changes. It'd be a pity if we
> missed this window to first-class sequenceid throughout or come up on HLC,
> at least for new tables, or split hbase:meta but as seems to be the push
> over in the concurrent thread, these can wait for hbase3.
>
> St.Ack
> 1.
> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_
> ktczrlKHK8N4SZzs/edit#heading=h.jxxznc91m047
>
>
>
> > Enis
> >
> > On Mon, Mar 6, 2017 at 2:54 PM, Stack <stack@duboce.net> wrote:
> >
> > > On Mon, Mar 6, 2017 at 2:50 PM, Stack <stack@duboce.net> wrote:
> > >
> > > > ...
> > > > + No recent work on core decision tasks (clean-up narrative around
> RPC
> > > > timeout, hbase:meta on master or not, batch vs partial semantic,
> etc.)
> > > >
> > > >
> > > Correction. batch vs partial semantic is making goodprogress
> > (HBASE-15484).
> > > S
> > >
> > >
> > >
> > >
> > > > Non-criticals/Ancillaries
> > > >
> > > > + Async client and C++ client are both making good progress. Not
> done.
> > > > + Backup/Restore is making good progress
> > > > + RegionServer-based assignment got a bunch of scrutiny lately and is
> > now
> > > > 'done'.
> > > > + FileSystem Quotas making good progress.
> > > >
> > > > I'm seeing another month or two at least before branch and probably
> > > three.
> > > > See doc [1] for more detail.
> > > >
> > > > Yours,
> > > > St.Ack
> > > >
> > > > 1.  https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9
> > > > iEu_ktczrlKHK8N4SZzs/edit#
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Sun, Jan 29, 2017 at 9:16 PM, Stack <stack@duboce.net> wrote:
> > > >
> > > >> On Thu, Jan 26, 2017 at 11:49 PM, ramkrishna vasudevan <
> > > >> ramkrishna.s.vasudevan@gmail.com> wrote:
> > > >>
> > > >>> Hi All
> > > >>>
> > > >>> Thanks Stack. The doc looks great. The offheap write path/read
> path-
> > I
> > > >>> think from the read path perspective we have some good feedback
> from
> > > >>> Alibaba folks.
> > > >>>
> > > >>
> > > >> Agree.
> > > >>
> > > >>
> > > >>> The write path subtasks are all done. We are currently working
on
> > some
> > > >>> perf
> > > >>> results that would help us to come up with some docs that suggests
> > best
> > > >>> configs and tunings for the offheap write path configurations.
> > > >>>
> > > >>>
> > > >> Thanks Ram. Would be good to hear what configs you are looking to
> > > >> implement as default so those of us also starting to test can enable
> > > them
> > > >> to get you feedback.
> > > >>
> > > >> Also suggest you fill the above short status into the doc (You are
> > > >> keeping up full status elsewhere). I've been trying to add status
> as I
> > > see
> > > >> it popping up; e.g. Enis did a nice state-of-the-C++ client recently
> > up
> > > in
> > > >> JIRA and I added pointer to the 2.0 doc. Anyone else working on 2.0
> > > >> features, would be good if you kept a short state in this overview
> > doc;
> > > >> just ask for edit perms.
> > > >>
> > > >> Thanks,
> > > >> St.Ack
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>>
> > > >>> Regards
> > > >>> Ram
> > > >>>
> > > >>> On Thu, Jan 19, 2017 at 5:37 AM, Andrew Purtell <
> > > >>> andrew.purtell@gmail.com>
> > > >>> wrote:
> > > >>>
> > > >>> > I'm interested in both split meta and rsgroups. Good news.
I'd
> like
> > > to
> > > >>> > help test.
> > > >>> >
> > > >>> >
> > > >>> > > On Jan 18, 2017, at 2:53 PM, Stack <stack@duboce.net>
wrote:
> > > >>> > >
> > > >>> > >> On Wed, Jan 18, 2017 at 2:26 PM, Francis Liu <
> toffer@apache.org
> > >
> > > >>> wrote:
> > > >>> > >>
> > > >>> > >> Hi Stack,
> > > >>> > >> I'd like to get split meta (HBASE-112288) in 2.x
as well. I
> can
> > > >>> have a
> > > >>> > 2.x
> > > >>> > >> draft up next week. Was working on the 1.x version
internally.
> > > >>> > >> Also if you'd like I can be the owner for rsgroups
as well.
> > > >>> > >> Thanks,Francis
> > > >>> > >>
> > > >>> > >>
> > > >>> > >>
> > > >>> > >> I added splitting meta as a possible and had you
and I as
> owner
> > on
> > > >>> > > rsgroups (I was doing to do a bit of testing and doc
for this
> > > >>> feature).
> > > >>> > >
> > > >>> > > Would love to see splittable meta show up. Needs to
be rolling
> > > >>> > upgradeable
> > > >>> > > though. Lets chat up on the issue.
> > > >>> > > St.Ack
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > >>
> > > >>> > >>
> > > >>> > >>
> > > >>> > >>    On Wednesday, January 18, 2017 11:29 AM, Stack
<
> > > stack@duboce.net
> > > >>> >
> > > >>> > >> wrote:
> > > >>> > >>
> > > >>> > >>
> > > >>> > >> Done Thiruvel (And thanks Guanghao for adding
> > hbase-replication).
> > > >>> > >> St.Ack
> > > >>> > >>
> > > >>> > >> On Tue, Jan 17, 2017 at 6:11 PM, Thiruvel Thirumoolan
<
> > > >>> > >> thiruvel@yahoo-inc.com.invalid> wrote:
> > > >>> > >>
> > > >>> > >>> Hi Stack,
> > > >>> > >>> I would like to add Favored Nodes to the ancillary
section.
> > > >>> > >>> HBASE-15531: Favored Nodes EnhancementsStatus:
Active
> > > >>> > development.Owner:
> > > >>> > >>> Thiruvel Thanks!-Thiruvel
> > > >>> > >>>
> > > >>> > >>>   On Monday, January 16, 2017 2:10 PM, Stack
<
> stack@duboce.net
> > >
> > > >>> wrote:
> > > >>> > >>>
> > > >>> > >>>
> > > >>> > >>> On Mon, Jan 16, 2017 at 3:01 AM, Guanghao Zhang
<
> > > >>> zghaobac@gmail.com>
> > > >>> > >>> wrote:
> > > >>> > >>>
> > > >>> > >>>> For 6. Significant contirbs in master only,
there are some
> > > issues
> > > >>> > about
> > > >>> > >>>> replication operations routed through master.
They are
> > sub-task
> > > >>> > >>>> of HBASE-10504. And there are other umbrella
issue for
> > > >>> replication,
> > > >>> > >> like
> > > >>> > >>>> HBase-14379 Replication V2 and HBASE-15867
Moving HBase
> > > >>> Replication
> > > >>> > >>>> tracking from Zookeeper to HBase. So i thought
we can add a
> > new
> > > >>> > section
> > > >>> > >>>> named
> > > >>> > >>>> hbase-replication to possible 2.0.0s. This
will help us to
> > track
> > > >>> the
> > > >>> > >>> state.
> > > >>> > >>>> Thanks.
> > > >>> > >>>>
> > > >>> > >>>
> > > >>> > >>> Thanks Guanghao Zhang. I agree. I made you an
editor. If you
> > want
> > > >>> to
> > > >>> > >> have a
> > > >>> > >>> go at a first cut, be my guest. If nothing done
in the next
> day
> > > or
> > > >>> so,
> > > >>> > >> I'll
> > > >>> > >>> add this section Sir.
> > > >>> > >>> Thanks,
> > > >>> > >>> M
> > > >>> > >>>
> > > >>> > >>>
> > > >>> > >>>
> > > >>> > >>>
> > > >>> > >>
> > > >>> > >>
> > > >>> > >>
> > > >>> > >>
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >
> > >
> >
>



-- 
Best regards,

   - Andy

If you are given a choice, you believe you have acted freely. - Raymond
Teller (via Peter Watts)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message