hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Antonov <olorinb...@gmail.com>
Subject Re: HBASE 2.0 release progress
Date Mon, 14 Nov 2016 20:15:04 GMT
Great to see progress here.

As I'm reading through the list, here're some high-level questions I have:

 -  Regarding the work on new AssignmentManager - any notes on
perf/stability testing? Are you guys running tips of master branch through
ITBLL setup?
 -  Anyone (Ram, Anoop?) wants to post a high-level writeup on the current
up-to-date state of offheaping?
 -  Logical clock - at this point is it more like a nice to have feature
than "need to be done before 2.0"? What are the features blocked or
affected by lack thereof?

-Mikhail

On Sun, Nov 13, 2016 at 3:57 PM, Stack <stack@duboce.net> wrote:

> Thanks for the writeup Stephen.
>
> See below.
>
> On Fri, Nov 11, 2016 at 5:15 PM, Stephen Jiang <syuanjiangdev@gmail.com>
> wrote:
>
> > Hello, fellow HBASE developers,
> >
> > We are making progress towards HBASE 2.0 releases.  I am using the
> > following queries to search for on-going HBASE 2.0 feature work items
> > (project = HBase AND (fixVersion = 2.0.0 OR affectedVersion = 2.0.0) AND
> > resolution is EMPTY AND (issuetype != Bug AND issuetype != Test AND
> > issuetype != Sub-task) ORDER BY issuetype DESC), at this time, we have
> > 247!  That is a lot.  At some time in near future, we definitely need to
> > trim down the list; otherwise, 2.0 will never be released.
> >
> >
> Agree. Suggest we all do our own review but at a certain stage, it is up to
> the RMs to make a call on what shape of 2.0 will be.
>
>
>
> > For now, Matteo and I are tracking some big projects that are on-going:
> >
> > (1). HBASE-14350 the stable Assignment Manager (using Procedure V2)
> > - This is a blocker to have branch-2 cut.  In the past few weeks, we made
> > good progress and majority of implementation is done.  The patches are
> > under review and testing.  Matto is drafting a document for review.
> >
> > (2). HBASE-14414 Backup/Restore Phase 3
> > - Currently it is blocked by HBASE-14123.  The giant HBASE-14123 patches
> > was discussed and reviewed from the community (see
> > http://www.mail-archive.com/dev@hbase.apache.org/msg41090.html for the
> > long
> > discussion); and all feedback were taken care of in the latest patch.
> > Currently I marked HBASE-14123 as 2.0 blocker, as without it, the further
> > develpment of backup/restore is blocked - the backup/restore is a key
> > enterprise feature for 2.0 release.  I think HBASE-14123 is ready for
> > another round of vote.
> >
> >
> IMO, not a blocker since it ancillary function but something we should get
> in.
>
>
>
> > (3). HBASE-15179 Offheap
> > - This is another important feature that I think it is MUST for 2.0.
> Since
> > stack works closely with Intel developers, he has some insight on this
> > project: "Intel are betting on this. Alibaba are using the offheap read
> > path and interested in write path too. This work is still very much up in
> > the air and being worked out as we go (especially the Y! Israel inmemory
> > compaction component).  It is a little shakey dependent on mslab pooling,
> > blockcache being on by default, async wal being default, and then
> dependent
> > on lots of perf and ITBLL testing."
> >
> >
> The above is from a private, hurried email that does not do the current
> state justice.
>
> Ram and Anoop are the authorities on offheaping and have been keeping up
> their state in an associated doc. The lads might be persuaded do a summary
> of current state here.
>
> Ditto for inmemory compaction. Our Anastasia and Eshcar are working hard at
> the moment doing laste pieces and perf testing. Maybe we can a status
> dumped here in dev.
>
>
>
> > (4). HBASE-16952 Protobuf3
> > - Good news, stack got the majority of work done already.  This is a MUST
> > for 2.0 release.  Now we only have a small sub-task HBASE-16967
> (findbugs)
> > left.
> >
> >
> To be clear, pb3 is under the offheaping umbrella. Let's add async WAL to
> this list, also needed by offheaping.
>
> Will be back after taking a look at current 2.0 list.
>
> Agree that hbase2 needs to support hadoop3.
>
> St.Ack
>
>
>
>
> > (5). HBASE-14070 Logical Clock
> > - This needs to be done before 2.0 release.  At this time, seems not
> making
> > much progress.
> >
> > (6). HBASE-16833 JAVA Async Client
> > - A lot of progress was made by Duo and his associates.  We should be in
> a
> > good shape in JAVA client when 2.0 release.
> >
> > (7). HBASE-14850 C++ Async Client
> > - Another project that is making good progress.  I know some customer
> wants
> > this.  This is long overdue project.  Hopefully we will make those
> customer
> > happy in 2.0 release with a good performed C++ client.
> >
> > Please let me know other on-going projects that needs to be in HBASE 2.0
> > release (stack mentioned "logical file system tier", but I am not sure
> > whether it would make enough progress to have a realistic chance making
> > 2.0).  As I said at the beginning of this email, at some point soon, we
> > will be more picky and trim down the unfinished features in 2.0.
> >
> > Happy Weekend!
> > Stephen
> >
>



-- 
Thanks,
Michael Antonov

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message