phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pedro Boado <pedro.bo...@gmail.com>
Subject Re: [DISCUSSION] Expanding Phoenix support to additional versions of CDH
Date Fri, 23 Mar 2018 22:55:00 GMT
I should've sent an email, my bad.

On Fri, 23 Mar 2018, 22:39 James Taylor, <jamestaylor@apache.org> wrote:

> Thanks, Pedro. This sounds great. Sorry about pushing to the wrong branch.
>
> On Fri, Mar 23, 2018 at 2:43 PM, Pedro Boado <pedro.boado@gmail.com>
> wrote:
>
> > Hi all,
> >
> > as discussed last week I've just pushed three more branches 4.x-cdh5.12 ,
> > 4.x-cdh5.13 and 4.x-cdh5.14. The three of them are fully functional and
> > create parcels for each one of these versions. Parcels starting from
> today
> > are now compatible across all minor versions.
> >
> > I'll keep rebasing this three branches onto 4.x-cdh5.11 to keep changes
> to
> > a minimum so please DO NOT PUSH Changes to  any of these three branches.
> >
> > I've also renamed 4.x-cdh5.11.2 as  4.x-cdh5.11 .  This will be (for now
> )
> > our base CDH branch so if you want to APPLY YOUR PATCHES please do it ON
> > 4.x-cdh5.11
> >
> > @JamesTaylor I've just noticed you've pushed  4.x-cdh5.11.2 again. I've
> > reapplied all changes in 4.x-cdh5.11 and  removed 4.x-cdh5.11.2
> >
> > Is everyone happy with the approach?
> >
> > Cheers,
> > Pedro.
> >
> >
> >
> > On 16 March 2018 at 22:13, Flavio Pompermaier <pompermaier@okkam.it>
> > wrote:
> >
> > > Thnaks Pedro for the great effort!
> > >
> > > On Mon, 12 Mar 2018, 05:19 James Taylor, <jamestaylor@apache.org>
> wrote:
> > >
> > > > This sounds great to me, Pedro.
> > > > Thanks!
> > > > James
> > > >
> > > > On Sun, Mar 11, 2018 at 3:30 PM Pedro Boado <pedro.boado@gmail.com>
> > > wrote:
> > > >
> > > > > I've just confirmed that changes required for cdh5.12,5.13 and 5.14
> > are
> > > > > minimal ( just changes in pom.xml and PhoenixRpcScheduler needing
a
> > few
> > > > > more methods to be implemented for cdh > 5.12 ) . All IT running
in
> > all
> > > > > versions. We could keep one branch for each minor version ( and
> keep
> > > > > rebasing them to keep only one in sync ) , build, test and name
> after
> > > the
> > > > > latest patch version available.
> > > > >
> > > > > My suggestion:
> > > > > - Create branch 4.x-cdh5.11.x  on branch 4.x-cdh5.11.2 - and remove
> > > this
> > > > > branch - ( and keep compiling against cdh 5.11.2  until a new patch
> > > > version
> > > > > 5.11.3 is made available)
> > > > > - Change parcels to be less restrictive, allowing to be installed
> > > > > regardless of the patch version  ( for instance parcel
> 4.x-cdh5.11.x
> > > > > could be installed in >= 5.11.0 and  < 5.12..0 ) . Parcel version
> > will
> > > > make
> > > > > clear what is the specific cdh version used to build that parcel
(
> > same
> > > > as
> > > > > now ) .
> > > > > - Create branch 4.x-cdh5.12.x  ( compiling against cdh 5.12.2
> until a
> > > new
> > > > > patch version is made available) . Keep rebasing onto cdh
> > 4.x-cdh5.11.x
> > > > > - Create branch 4.x-cdh5.13.x  ( compiling against cdh 5.13.2
> until a
> > > new
> > > > > patch version is made available) . Keep rebasing onto cdh
> > 4.x-cdh5.11.x
> > > > > - Create branch 4.x-cdh5.14.x  ( compiling against cdh 5.14.2
> until a
> > > new
> > > > > patch version is made available) . Keep rebasing onto cdh
> > 4.x-cdh5.11.x
> > > > >
> > > > > I will take care of all cdh branches if everyone is happy with it.
> > > > >
> > > > > Cheers.
> > > > >
> > > > >
> > > > > On 10 March 2018 at 09:46, Pedro Boado <pedro.boado@gmail.com>
> > wrote:
> > > > >
> > > > > > I rather prefer keeping several branches. Most of the changes
in
> > our
> > > > cdh
> > > > > > branch are related to a couple of interfaces in HBase 1.2-cdh
> that
> > > have
> > > > > > included changes from Apache HBase 1.3 (as Apache HBase 1.2
is
> > close
> > > to
> > > > > > EOM, Cloudera keeps patching it's versions by applying some
> changes
> > > > from
> > > > > > HBase 1.3) and that Phoenix implements. Shim layer would make
> thing
> > > > more
> > > > > > complicated.
> > > > > >
> > > > > > It's quite likely that the only change needed is at pom.xml
> level (
> > > cdh
> > > > > > version-specific grandparent pom and related properties in parent
> > > pom )
> > > > > >
> > > > > > This could be solved by keeping our current 4.x-cdh-5.11.2 in
> sync
> > (
> > > > > > renamed as 4.x-cdh5 ) with 4.x-HBase-1.2, and then keep rebasing
> > > > > supported
> > > > > > version branches 4.x-cdh-5.11.2, 4.x-cdh-5.13.2, etc  onto
> 4.x-cdh5
> > > > > ~HEAD.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 10 Mar 2018 05:36, "James Taylor" <jamestaylor@apache.org>
> > wrote:
> > > > > >
> > > > > >> If you're willing to sign up to keep the CDH branches in
sync
> with
> > > the
> > > > > >> 4.x-HBase-1.2 branches, then that seems fine. Should we
look
> into
> > > some
> > > > > >> kind
> > > > > >> of automation for syncing all the branches?
> > > > > >>
> > > > > >> Another approach is a shim layer, but we've been reluctant
to
> set
> > > that
> > > > > up
> > > > > >> as there's a fair amount of initial overhead to implement.
> > > > > >>
> > > > > >> On Thu, Mar 8, 2018 at 4:24 PM, Pedro Boado <pboado@apache.org>
> > > > wrote:
> > > > > >>
> > > > > >> > Hi all,
> > > > > >> >
> > > > > >> > After our first release of Phoenix for CDH 5.11.2 it
looks
> like
> > a
> > > > good
> > > > > >> idea
> > > > > >> > expanding support to other newer versions. I'd like
to discuss
> > > with
> > > > > you
> > > > > >> > guys what would be the best approach as we have several
> options:
> > > > > >> >
> > > > > >> > - Releasing one single generated parcel with wider
> > compatibility (
> > > > > >> > cloudera-labs phoenix style ) . The issue here is that
all
> > > > transitive
> > > > > >> > dependencies packaged in phoenix fatjars would be specific
to
> a
> > > cdh
> > > > > >> version
> > > > > >> > (let's say, cdh5.11.2) but would be running against
different
> > cdh
> > > > > >> version
> > > > > >> > (maybe cdh5.14.0 ) . There is a small chance of
> incompatibility
> > > > across
> > > > > >> > versions ( even when all of them are HBase 1.2 based
) . Also
> we
> > > > > >> wouldn't
> > > > > >> > be running our IT against all these cdh versions.
> > > > > >> >
> > > > > >> > - Maybe work further into packaging (and removing)
any
> > dependency
> > > > that
> > > > > >> is
> > > > > >> > already shipped in cdh. That would improve compatibility
of
> this
> > > > > unique
> > > > > >> > parcel version across cdh releases. But fat client
fatjars
> would
> > > > still
> > > > > >> be a
> > > > > >> > problem for use from out of hadoop cluster.
> > > > > >> >
> > > > > >> > - Release several parcels specific to different cdh
versions (
> > my
> > > > > >> favourite
> > > > > >> > option ) . That is the safest option for better compatibility
> as
> > > we
> > > > > >> would
> > > > > >> > be shipping the exact same libraries in the parcel
as used in
> > that
> > > > > >> version
> > > > > >> > of cdh. And we'd also be doing IT for several cdh versions.
> > > Downside
> > > > > is
> > > > > >> a
> > > > > >> > little bit more of release effort ( not a big deal
) , more
> > > branches
> > > > > in
> > > > > >> git
> > > > > >> > and  more web server space needed for keeping several
parcels
> (
> > > I'm
> > > > > not
> > > > > >> > sure if that is an issue )
> > > > > >> >
> > > > > >> > All ideas are welcome.
> > > > > >> >
> > > > > >> > Thanks!
> > > > > >> >
> > > > > >>
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Un saludo.
> > > > > Pedro Boado.
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Un saludo.
> > Pedro Boado.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message