hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Hsieh <...@cloudera.com>
Subject Re: Hadoop 3.x profile working for hbase 2.0 [Re: HBASE 2.0]
Date Thu, 06 Oct 2016 17:41:00 GMT
Yes.

The goal is to produce one set of hbase binaries that will work against
multiple hadoops such as the 2.7 and 3.0.0-alpha1 versions, but
preferentially tested against and likely including binaries from a stable
hadoop version.

Up until recently, compiling against the hadoop 3 profile fails because of
compilation issues and licensing issues,   Another issue, HBASE-16711 has
already landed which fixed compilation against hadoop2 and hadoop3. What
remains is on the short proposed list makes sure licensing enforcers are
satisfied correctly and getting build infrastructure precommit checks in
place so we don't inadvertently introduce new problems.

Jon.

On Thu, Oct 6, 2016 at 9:02 AM, Ted Yu <yuzhihong@gmail.com> wrote:

> Jon:
> Once the goals you outlined below are achieved, would user be able to use
> build artifacts compiled against hadoop 2.7.1 on a cluster deployed with
> hadoop
> 3.0.0-alpha1 ?
>
> Cheers
>
> On Thu, Oct 6, 2016 at 12:07 AM, Jonathan Hsieh <jon@cloudera.com> wrote:
>
> > I'd like to get the -Dhadoop.profile=3.0 at least to compiling and
> passing
> > licensing working for the first hbase alpha (or whatever we end up
> calling
> > it)
> >
> > I'll propose these items:
> > 1) peg to one of the recent hadoop alphas (hadoop 3.0.0-alpha1 is the
> most
> > recent). currently we are against 3.0.0-SNAPSHOT.
> > 2) add precompile checks against a hadoop 3.x (HBASE-16733)
> > 3) get 'mvn test install -Dskiptests' to succeed without licensing issues
> > (HBASE-16712)
> > 4) Have a job setup in jenkins so that we can gain insight and burn down
> > unit tests failures against hadoop3.
> >
> > These items have a good chance of landing in the next week or two.
> >
> > Other related issues that are nice to have but wouldn't block an hbase
> > alpha include:
> > 1) having no always failing unit tests against hadoop3 (HBASE-6581)
> >
> > Thoughts?
> > Jon.
> >
> > On Mon, Oct 3, 2016 at 3:27 PM, Stephen Jiang <syuanjiangdev@gmail.com>
> > wrote:
> >
> > > Hello, All,
> > >
> > > It is time to discuss about the schedule of HBase 2.0 release.  HBase
> 2.0
> > > release is a big major release.  When we release 1.0, we had 0.99 as
> dev
> > > preview/beta release.  We should do something similar for the 2.0
> > release.
> > >
> > > Matteo and I talked about this.   We think about that we need some
> > > Alpha/Beta milestones before the RC and final Release-to-Web 2.0
> release.
> > >
> > > I don't know whether there is any discussion on this community about
> the
> > > Alpha/Beta release criteria.  My proposal is that once we cut the
> > branch-2,
> > > we should only have new features that are absolutely needed for major
> > > release (festures cannot be added in minor release) and those features
> > > should be "almost ready".  For Alpha releases, we can still accept
> these
> > > kind of features; for Beta release, only bug fixes and performance
> > > improvement on existing features (should we also accept existing
> feature
> > > improvement in Beta?  Maybe Beta 1, Not in Beta 2 - that is my take).
> > >
> > > This is a big release and requires a lot of work from Release
> Manager.  I
> > > asked Matteo whether I could help to be some kind of backup /
> > hot-standby /
> > > assistant RM.  I think he is very happy to have someone to share some
> RM
> > > duties.  Thus, I will help make this 2.0 release as smooth as possible.
> > >
> > > Here is a tentative plan:
> > > - For now, we are thinking of creating branch-2 middle of this month
> and
> > > have 2.0-Alpha1 release at the end of this month or begin of Nov.  The
> > > definition of Alpha1 is that we could deploy to a cluster and it can do
> > > some simple CRUD and table DDLs; and not crash (of course, UT passing).
> > >
> > > - Then we will have 2.0-Alpha2 in 4-6 weeks after Apha1.  It would hold
> > > higher bar.  We will run some IT tests to make sure that it would
> > > functional.
> > >
> > > - At that time, we will lock down and not allow any new features, every
> > 4-6
> > > weeks, we will have a Beta release (my realistic guess is that we would
> > hit
> > > the US Christmas holiday at that time, so first Beta would take longer
> > than
> > > 6 weeks).  For Beta release, we would fix bugs and do performance
> tuning.
> > > Planning to have 2 Betas.  (Question: in the past, do we need vote to
> > have
> > > a Beta release?)
> > >
> > > - Once the code are in the stable stage, then we will have RCs and vote
> > for
> > > the final release.
> > >
> > > Please let us know whether this is a reasonable approach that will make
> > the
> > > release successful.
> > >
> > > Currently, we are aware of the following on-going new features for 2.0:
> > new
> > > Assignment Manager, backup/restore, off-heap, protobuff 3, Hybrid
> Logical
> > > Clock, and maybe AsyncRegion / C++ client).  If you have a feature that
> > > wants to be part of 2.0 release, please send discussion to dev
> community
> > > and we can make a call on accepting/rejecting.
> > >
> > > Thanks
> > > Stephen
> > >
> >
> >
> >
> > --
> > // Jonathan Hsieh (shay)
> > // HBase Tech Lead, Software Engineer, Cloudera
> > // jon@cloudera.com // @jmhsieh
> >
>



-- 
// Jonathan Hsieh (shay)
// HBase Tech Lead, Software Engineer, Cloudera
// jon@cloudera.com // @jmhsieh

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message