hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: HBASE 2.0 release progress
Date Mon, 14 Nov 2016 23:05:50 GMT
Using this lens, backup / restore should come in 2.0 with client facing
APIs properly defined.

On Mon, Nov 14, 2016 at 2:57 PM, Enis Söztutar <enis.soz@gmail.com> wrote:

> Another way to look at whether a feature is a "blocker" or not for 2.0 is a
> simple test:
>  - Can this feature be committed for 2.1, 2.2, etc or not assuming it does
> not make into 2.0.
>
> It is a simple test, but affectively validates whether the feature is
> client-visible and have backwards compatibility implications. If it is ok
> to bring the feature in for 2.1 or later, it means that the release should
> never block on it. Otherwise, we should at least have the API / Client
> visible parts of it in 2.0 if not the full thing.
>
> Looking at the features under discussion through this lens reveals easier
> decision points I think. A couple of examples:
>  - Java async client / C++ Client: It is independent work, can come in 2.1
> or 2.2, etc. Whenever it is completed.
>  - Offheaping : transparent ti clients, so can come in incrementally in
> 2.0, 2.1, 2.2, etc.
>
> On the other hand, 1.x releases have been going on for >1.5 years, and will
> likely go on for another year at least. So the choices (dependency, APIs,
> etc) that we make now for 2.0 will likely live for >2 years. In that
> respect I would rather be a bit more aggressive in terms of dropping
> support for older stuff and updating wherever we can. For example require
> Hadoop-2.6+, Java-8, etc.
>
> Enis
>
> On Mon, Nov 14, 2016 at 12:15 PM, Mikhail Antonov <olorinbant@gmail.com>
> wrote:
>
> > Great to see progress here.
> >
> > As I'm reading through the list, here're some high-level questions I
> have:
> >
> >  -  Regarding the work on new AssignmentManager - any notes on
> > perf/stability testing? Are you guys running tips of master branch
> through
> > ITBLL setup?
> >  -  Anyone (Ram, Anoop?) wants to post a high-level writeup on the
> current
> > up-to-date state of offheaping?
> >  -  Logical clock - at this point is it more like a nice to have feature
> > than "need to be done before 2.0"? What are the features blocked or
> > affected by lack thereof?
> >
> > -Mikhail
> >
> > On Sun, Nov 13, 2016 at 3:57 PM, Stack <stack@duboce.net> wrote:
> >
> > > Thanks for the writeup Stephen.
> > >
> > > See below.
> > >
> > > On Fri, Nov 11, 2016 at 5:15 PM, Stephen Jiang <
> syuanjiangdev@gmail.com>
> > > wrote:
> > >
> > > > Hello, fellow HBASE developers,
> > > >
> > > > We are making progress towards HBASE 2.0 releases.  I am using the
> > > > following queries to search for on-going HBASE 2.0 feature work items
> > > > (project = HBase AND (fixVersion = 2.0.0 OR affectedVersion = 2.0.0)
> > AND
> > > > resolution is EMPTY AND (issuetype != Bug AND issuetype != Test AND
> > > > issuetype != Sub-task) ORDER BY issuetype DESC), at this time, we
> have
> > > > 247!  That is a lot.  At some time in near future, we definitely need
> > to
> > > > trim down the list; otherwise, 2.0 will never be released.
> > > >
> > > >
> > > Agree. Suggest we all do our own review but at a certain stage, it is
> up
> > to
> > > the RMs to make a call on what shape of 2.0 will be.
> > >
> > >
> > >
> > > > For now, Matteo and I are tracking some big projects that are
> on-going:
> > > >
> > > > (1). HBASE-14350 the stable Assignment Manager (using Procedure V2)
> > > > - This is a blocker to have branch-2 cut.  In the past few weeks, we
> > made
> > > > good progress and majority of implementation is done.  The patches
> are
> > > > under review and testing.  Matto is drafting a document for review.
> > > >
> > > > (2). HBASE-14414 Backup/Restore Phase 3
> > > > - Currently it is blocked by HBASE-14123.  The giant HBASE-14123
> > patches
> > > > was discussed and reviewed from the community (see
> > > > http://www.mail-archive.com/dev@hbase.apache.org/msg41090.html for
> the
> > > > long
> > > > discussion); and all feedback were taken care of in the latest patch.
> > > > Currently I marked HBASE-14123 as 2.0 blocker, as without it, the
> > further
> > > > develpment of backup/restore is blocked - the backup/restore is a key
> > > > enterprise feature for 2.0 release.  I think HBASE-14123 is ready for
> > > > another round of vote.
> > > >
> > > >
> > > IMO, not a blocker since it ancillary function but something we should
> > get
> > > in.
> > >
> > >
> > >
> > > > (3). HBASE-15179 Offheap
> > > > - This is another important feature that I think it is MUST for 2.0.
> > > Since
> > > > stack works closely with Intel developers, he has some insight on
> this
> > > > project: "Intel are betting on this. Alibaba are using the offheap
> read
> > > > path and interested in write path too. This work is still very much
> up
> > in
> > > > the air and being worked out as we go (especially the Y! Israel
> > inmemory
> > > > compaction component).  It is a little shakey dependent on mslab
> > pooling,
> > > > blockcache being on by default, async wal being default, and then
> > > dependent
> > > > on lots of perf and ITBLL testing."
> > > >
> > > >
> > > The above is from a private, hurried email that does not do the current
> > > state justice.
> > >
> > > Ram and Anoop are the authorities on offheaping and have been keeping
> up
> > > their state in an associated doc. The lads might be persuaded do a
> > summary
> > > of current state here.
> > >
> > > Ditto for inmemory compaction. Our Anastasia and Eshcar are working
> hard
> > at
> > > the moment doing laste pieces and perf testing. Maybe we can a status
> > > dumped here in dev.
> > >
> > >
> > >
> > > > (4). HBASE-16952 Protobuf3
> > > > - Good news, stack got the majority of work done already.  This is a
> > MUST
> > > > for 2.0 release.  Now we only have a small sub-task HBASE-16967
> > > (findbugs)
> > > > left.
> > > >
> > > >
> > > To be clear, pb3 is under the offheaping umbrella. Let's add async WAL
> to
> > > this list, also needed by offheaping.
> > >
> > > Will be back after taking a look at current 2.0 list.
> > >
> > > Agree that hbase2 needs to support hadoop3.
> > >
> > > St.Ack
> > >
> > >
> > >
> > >
> > > > (5). HBASE-14070 Logical Clock
> > > > - This needs to be done before 2.0 release.  At this time, seems not
> > > making
> > > > much progress.
> > > >
> > > > (6). HBASE-16833 JAVA Async Client
> > > > - A lot of progress was made by Duo and his associates.  We should be
> > in
> > > a
> > > > good shape in JAVA client when 2.0 release.
> > > >
> > > > (7). HBASE-14850 C++ Async Client
> > > > - Another project that is making good progress.  I know some customer
> > > wants
> > > > this.  This is long overdue project.  Hopefully we will make those
> > > customer
> > > > happy in 2.0 release with a good performed C++ client.
> > > >
> > > > Please let me know other on-going projects that needs to be in HBASE
> > 2.0
> > > > release (stack mentioned "logical file system tier", but I am not
> sure
> > > > whether it would make enough progress to have a realistic chance
> making
> > > > 2.0).  As I said at the beginning of this email, at some point soon,
> we
> > > > will be more picky and trim down the unfinished features in 2.0.
> > > >
> > > > Happy Weekend!
> > > > Stephen
> > > >
> > >
> >
> >
> >
> > --
> > Thanks,
> > Michael Antonov
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message