hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Hadoop 3.x profile working for hbase 2.0 [Re: HBASE 2.0]
Date Thu, 06 Oct 2016 17:47:45 GMT
bq. one set of hbase binaries that will work against multiple hadoops

I would be interested to know what tests are / will be performed
against 3.0.0-alpha1
(using artifacts built against 2.7.1).

Thanks


On Thu, Oct 6, 2016 at 10:41 AM, Jonathan Hsieh <jon@cloudera.com> wrote:

> Yes.
>
> The goal is to produce one set of hbase binaries that will work against
> multiple hadoops such as the 2.7 and 3.0.0-alpha1 versions, but
> preferentially tested against and likely including binaries from a stable
> hadoop version.
>
> Up until recently, compiling against the hadoop 3 profile fails because of
> compilation issues and licensing issues,   Another issue, HBASE-16711 has
> already landed which fixed compilation against hadoop2 and hadoop3. What
> remains is on the short proposed list makes sure licensing enforcers are
> satisfied correctly and getting build infrastructure precommit checks in
> place so we don't inadvertently introduce new problems.
>
> Jon.
>
> On Thu, Oct 6, 2016 at 9:02 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > Jon:
> > Once the goals you outlined below are achieved, would user be able to use
> > build artifacts compiled against hadoop 2.7.1 on a cluster deployed with
> > hadoop
> > 3.0.0-alpha1 ?
> >
> > Cheers
> >
> > On Thu, Oct 6, 2016 at 12:07 AM, Jonathan Hsieh <jon@cloudera.com>
> wrote:
> >
> > > I'd like to get the -Dhadoop.profile=3.0 at least to compiling and
> > passing
> > > licensing working for the first hbase alpha (or whatever we end up
> > calling
> > > it)
> > >
> > > I'll propose these items:
> > > 1) peg to one of the recent hadoop alphas (hadoop 3.0.0-alpha1 is the
> > most
> > > recent). currently we are against 3.0.0-SNAPSHOT.
> > > 2) add precompile checks against a hadoop 3.x (HBASE-16733)
> > > 3) get 'mvn test install -Dskiptests' to succeed without licensing
> issues
> > > (HBASE-16712)
> > > 4) Have a job setup in jenkins so that we can gain insight and burn
> down
> > > unit tests failures against hadoop3.
> > >
> > > These items have a good chance of landing in the next week or two.
> > >
> > > Other related issues that are nice to have but wouldn't block an hbase
> > > alpha include:
> > > 1) having no always failing unit tests against hadoop3 (HBASE-6581)
> > >
> > > Thoughts?
> > > Jon.
> > >
> > > On Mon, Oct 3, 2016 at 3:27 PM, Stephen Jiang <syuanjiangdev@gmail.com
> >
> > > wrote:
> > >
> > > > Hello, All,
> > > >
> > > > It is time to discuss about the schedule of HBase 2.0 release.  HBase
> > 2.0
> > > > release is a big major release.  When we release 1.0, we had 0.99 as
> > dev
> > > > preview/beta release.  We should do something similar for the 2.0
> > > release.
> > > >
> > > > Matteo and I talked about this.   We think about that we need some
> > > > Alpha/Beta milestones before the RC and final Release-to-Web 2.0
> > release.
> > > >
> > > > I don't know whether there is any discussion on this community about
> > the
> > > > Alpha/Beta release criteria.  My proposal is that once we cut the
> > > branch-2,
> > > > we should only have new features that are absolutely needed for major
> > > > release (festures cannot be added in minor release) and those
> features
> > > > should be "almost ready".  For Alpha releases, we can still accept
> > these
> > > > kind of features; for Beta release, only bug fixes and performance
> > > > improvement on existing features (should we also accept existing
> > feature
> > > > improvement in Beta?  Maybe Beta 1, Not in Beta 2 - that is my take).
> > > >
> > > > This is a big release and requires a lot of work from Release
> > Manager.  I
> > > > asked Matteo whether I could help to be some kind of backup /
> > > hot-standby /
> > > > assistant RM.  I think he is very happy to have someone to share some
> > RM
> > > > duties.  Thus, I will help make this 2.0 release as smooth as
> possible.
> > > >
> > > > Here is a tentative plan:
> > > > - For now, we are thinking of creating branch-2 middle of this month
> > and
> > > > have 2.0-Alpha1 release at the end of this month or begin of Nov.
> The
> > > > definition of Alpha1 is that we could deploy to a cluster and it can
> do
> > > > some simple CRUD and table DDLs; and not crash (of course, UT
> passing).
> > > >
> > > > - Then we will have 2.0-Alpha2 in 4-6 weeks after Apha1.  It would
> hold
> > > > higher bar.  We will run some IT tests to make sure that it would
> > > > functional.
> > > >
> > > > - At that time, we will lock down and not allow any new features,
> every
> > > 4-6
> > > > weeks, we will have a Beta release (my realistic guess is that we
> would
> > > hit
> > > > the US Christmas holiday at that time, so first Beta would take
> longer
> > > than
> > > > 6 weeks).  For Beta release, we would fix bugs and do performance
> > tuning.
> > > > Planning to have 2 Betas.  (Question: in the past, do we need vote to
> > > have
> > > > a Beta release?)
> > > >
> > > > - Once the code are in the stable stage, then we will have RCs and
> vote
> > > for
> > > > the final release.
> > > >
> > > > Please let us know whether this is a reasonable approach that will
> make
> > > the
> > > > release successful.
> > > >
> > > > Currently, we are aware of the following on-going new features for
> 2.0:
> > > new
> > > > Assignment Manager, backup/restore, off-heap, protobuff 3, Hybrid
> > Logical
> > > > Clock, and maybe AsyncRegion / C++ client).  If you have a feature
> that
> > > > wants to be part of 2.0 release, please send discussion to dev
> > community
> > > > and we can make a call on accepting/rejecting.
> > > >
> > > > Thanks
> > > > Stephen
> > > >
> > >
> > >
> > >
> > > --
> > > // Jonathan Hsieh (shay)
> > > // HBase Tech Lead, Software Engineer, Cloudera
> > > // jon@cloudera.com // @jmhsieh
> > >
> >
>
>
>
> --
> // Jonathan Hsieh (shay)
> // HBase Tech Lead, Software Engineer, Cloudera
> // jon@cloudera.com // @jmhsieh
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message