hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Enis Söztutar <enis....@gmail.com>
Subject Re: HBASE 2.0 release progress
Date Mon, 14 Nov 2016 22:57:18 GMT
Another way to look at whether a feature is a "blocker" or not for 2.0 is a
simple test:
 - Can this feature be committed for 2.1, 2.2, etc or not assuming it does
not make into 2.0.

It is a simple test, but affectively validates whether the feature is
client-visible and have backwards compatibility implications. If it is ok
to bring the feature in for 2.1 or later, it means that the release should
never block on it. Otherwise, we should at least have the API / Client
visible parts of it in 2.0 if not the full thing.

Looking at the features under discussion through this lens reveals easier
decision points I think. A couple of examples:
 - Java async client / C++ Client: It is independent work, can come in 2.1
or 2.2, etc. Whenever it is completed.
 - Offheaping : transparent ti clients, so can come in incrementally in
2.0, 2.1, 2.2, etc.

On the other hand, 1.x releases have been going on for >1.5 years, and will
likely go on for another year at least. So the choices (dependency, APIs,
etc) that we make now for 2.0 will likely live for >2 years. In that
respect I would rather be a bit more aggressive in terms of dropping
support for older stuff and updating wherever we can. For example require
Hadoop-2.6+, Java-8, etc.

Enis

On Mon, Nov 14, 2016 at 12:15 PM, Mikhail Antonov <olorinbant@gmail.com>
wrote:

> Great to see progress here.
>
> As I'm reading through the list, here're some high-level questions I have:
>
>  -  Regarding the work on new AssignmentManager - any notes on
> perf/stability testing? Are you guys running tips of master branch through
> ITBLL setup?
>  -  Anyone (Ram, Anoop?) wants to post a high-level writeup on the current
> up-to-date state of offheaping?
>  -  Logical clock - at this point is it more like a nice to have feature
> than "need to be done before 2.0"? What are the features blocked or
> affected by lack thereof?
>
> -Mikhail
>
> On Sun, Nov 13, 2016 at 3:57 PM, Stack <stack@duboce.net> wrote:
>
> > Thanks for the writeup Stephen.
> >
> > See below.
> >
> > On Fri, Nov 11, 2016 at 5:15 PM, Stephen Jiang <syuanjiangdev@gmail.com>
> > wrote:
> >
> > > Hello, fellow HBASE developers,
> > >
> > > We are making progress towards HBASE 2.0 releases.  I am using the
> > > following queries to search for on-going HBASE 2.0 feature work items
> > > (project = HBase AND (fixVersion = 2.0.0 OR affectedVersion = 2.0.0)
> AND
> > > resolution is EMPTY AND (issuetype != Bug AND issuetype != Test AND
> > > issuetype != Sub-task) ORDER BY issuetype DESC), at this time, we have
> > > 247!  That is a lot.  At some time in near future, we definitely need
> to
> > > trim down the list; otherwise, 2.0 will never be released.
> > >
> > >
> > Agree. Suggest we all do our own review but at a certain stage, it is up
> to
> > the RMs to make a call on what shape of 2.0 will be.
> >
> >
> >
> > > For now, Matteo and I are tracking some big projects that are on-going:
> > >
> > > (1). HBASE-14350 the stable Assignment Manager (using Procedure V2)
> > > - This is a blocker to have branch-2 cut.  In the past few weeks, we
> made
> > > good progress and majority of implementation is done.  The patches are
> > > under review and testing.  Matto is drafting a document for review.
> > >
> > > (2). HBASE-14414 Backup/Restore Phase 3
> > > - Currently it is blocked by HBASE-14123.  The giant HBASE-14123
> patches
> > > was discussed and reviewed from the community (see
> > > http://www.mail-archive.com/dev@hbase.apache.org/msg41090.html for the
> > > long
> > > discussion); and all feedback were taken care of in the latest patch.
> > > Currently I marked HBASE-14123 as 2.0 blocker, as without it, the
> further
> > > develpment of backup/restore is blocked - the backup/restore is a key
> > > enterprise feature for 2.0 release.  I think HBASE-14123 is ready for
> > > another round of vote.
> > >
> > >
> > IMO, not a blocker since it ancillary function but something we should
> get
> > in.
> >
> >
> >
> > > (3). HBASE-15179 Offheap
> > > - This is another important feature that I think it is MUST for 2.0.
> > Since
> > > stack works closely with Intel developers, he has some insight on this
> > > project: "Intel are betting on this. Alibaba are using the offheap read
> > > path and interested in write path too. This work is still very much up
> in
> > > the air and being worked out as we go (especially the Y! Israel
> inmemory
> > > compaction component).  It is a little shakey dependent on mslab
> pooling,
> > > blockcache being on by default, async wal being default, and then
> > dependent
> > > on lots of perf and ITBLL testing."
> > >
> > >
> > The above is from a private, hurried email that does not do the current
> > state justice.
> >
> > Ram and Anoop are the authorities on offheaping and have been keeping up
> > their state in an associated doc. The lads might be persuaded do a
> summary
> > of current state here.
> >
> > Ditto for inmemory compaction. Our Anastasia and Eshcar are working hard
> at
> > the moment doing laste pieces and perf testing. Maybe we can a status
> > dumped here in dev.
> >
> >
> >
> > > (4). HBASE-16952 Protobuf3
> > > - Good news, stack got the majority of work done already.  This is a
> MUST
> > > for 2.0 release.  Now we only have a small sub-task HBASE-16967
> > (findbugs)
> > > left.
> > >
> > >
> > To be clear, pb3 is under the offheaping umbrella. Let's add async WAL to
> > this list, also needed by offheaping.
> >
> > Will be back after taking a look at current 2.0 list.
> >
> > Agree that hbase2 needs to support hadoop3.
> >
> > St.Ack
> >
> >
> >
> >
> > > (5). HBASE-14070 Logical Clock
> > > - This needs to be done before 2.0 release.  At this time, seems not
> > making
> > > much progress.
> > >
> > > (6). HBASE-16833 JAVA Async Client
> > > - A lot of progress was made by Duo and his associates.  We should be
> in
> > a
> > > good shape in JAVA client when 2.0 release.
> > >
> > > (7). HBASE-14850 C++ Async Client
> > > - Another project that is making good progress.  I know some customer
> > wants
> > > this.  This is long overdue project.  Hopefully we will make those
> > customer
> > > happy in 2.0 release with a good performed C++ client.
> > >
> > > Please let me know other on-going projects that needs to be in HBASE
> 2.0
> > > release (stack mentioned "logical file system tier", but I am not sure
> > > whether it would make enough progress to have a realistic chance making
> > > 2.0).  As I said at the beginning of this email, at some point soon, we
> > > will be more picky and trim down the unfinished features in 2.0.
> > >
> > > Happy Weekend!
> > > Stephen
> > >
> >
>
>
>
> --
> Thanks,
> Michael Antonov
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message