phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Taylor <jamestay...@apache.org>
Subject Re: [DISCUSSION] Expanding Phoenix support to additional versions of CDH
Date Fri, 23 Mar 2018 22:39:49 GMT
Thanks, Pedro. This sounds great. Sorry about pushing to the wrong branch.

On Fri, Mar 23, 2018 at 2:43 PM, Pedro Boado <pedro.boado@gmail.com> wrote:

> Hi all,
>
> as discussed last week I've just pushed three more branches 4.x-cdh5.12 ,
> 4.x-cdh5.13 and 4.x-cdh5.14. The three of them are fully functional and
> create parcels for each one of these versions. Parcels starting from today
> are now compatible across all minor versions.
>
> I'll keep rebasing this three branches onto 4.x-cdh5.11 to keep changes to
> a minimum so please DO NOT PUSH Changes to  any of these three branches.
>
> I've also renamed 4.x-cdh5.11.2 as  4.x-cdh5.11 .  This will be (for now )
> our base CDH branch so if you want to APPLY YOUR PATCHES please do it ON
> 4.x-cdh5.11
>
> @JamesTaylor I've just noticed you've pushed  4.x-cdh5.11.2 again. I've
> reapplied all changes in 4.x-cdh5.11 and  removed 4.x-cdh5.11.2
>
> Is everyone happy with the approach?
>
> Cheers,
> Pedro.
>
>
>
> On 16 March 2018 at 22:13, Flavio Pompermaier <pompermaier@okkam.it>
> wrote:
>
> > Thnaks Pedro for the great effort!
> >
> > On Mon, 12 Mar 2018, 05:19 James Taylor, <jamestaylor@apache.org> wrote:
> >
> > > This sounds great to me, Pedro.
> > > Thanks!
> > > James
> > >
> > > On Sun, Mar 11, 2018 at 3:30 PM Pedro Boado <pedro.boado@gmail.com>
> > wrote:
> > >
> > > > I've just confirmed that changes required for cdh5.12,5.13 and 5.14
> are
> > > > minimal ( just changes in pom.xml and PhoenixRpcScheduler needing a
> few
> > > > more methods to be implemented for cdh > 5.12 ) . All IT running in
> all
> > > > versions. We could keep one branch for each minor version ( and keep
> > > > rebasing them to keep only one in sync ) , build, test and name after
> > the
> > > > latest patch version available.
> > > >
> > > > My suggestion:
> > > > - Create branch 4.x-cdh5.11.x  on branch 4.x-cdh5.11.2 - and remove
> > this
> > > > branch - ( and keep compiling against cdh 5.11.2  until a new patch
> > > version
> > > > 5.11.3 is made available)
> > > > - Change parcels to be less restrictive, allowing to be installed
> > > > regardless of the patch version  ( for instance parcel 4.x-cdh5.11.x
> > > > could be installed in >= 5.11.0 and  < 5.12..0 ) . Parcel version
> will
> > > make
> > > > clear what is the specific cdh version used to build that parcel (
> same
> > > as
> > > > now ) .
> > > > - Create branch 4.x-cdh5.12.x  ( compiling against cdh 5.12.2 until a
> > new
> > > > patch version is made available) . Keep rebasing onto cdh
> 4.x-cdh5.11.x
> > > > - Create branch 4.x-cdh5.13.x  ( compiling against cdh 5.13.2 until a
> > new
> > > > patch version is made available) . Keep rebasing onto cdh
> 4.x-cdh5.11.x
> > > > - Create branch 4.x-cdh5.14.x  ( compiling against cdh 5.14.2 until a
> > new
> > > > patch version is made available) . Keep rebasing onto cdh
> 4.x-cdh5.11.x
> > > >
> > > > I will take care of all cdh branches if everyone is happy with it.
> > > >
> > > > Cheers.
> > > >
> > > >
> > > > On 10 March 2018 at 09:46, Pedro Boado <pedro.boado@gmail.com>
> wrote:
> > > >
> > > > > I rather prefer keeping several branches. Most of the changes in
> our
> > > cdh
> > > > > branch are related to a couple of interfaces in HBase 1.2-cdh that
> > have
> > > > > included changes from Apache HBase 1.3 (as Apache HBase 1.2 is
> close
> > to
> > > > > EOM, Cloudera keeps patching it's versions by applying some changes
> > > from
> > > > > HBase 1.3) and that Phoenix implements. Shim layer would make thing
> > > more
> > > > > complicated.
> > > > >
> > > > > It's quite likely that the only change needed is at pom.xml level
(
> > cdh
> > > > > version-specific grandparent pom and related properties in parent
> > pom )
> > > > >
> > > > > This could be solved by keeping our current 4.x-cdh-5.11.2 in sync
> (
> > > > > renamed as 4.x-cdh5 ) with 4.x-HBase-1.2, and then keep rebasing
> > > > supported
> > > > > version branches 4.x-cdh-5.11.2, 4.x-cdh-5.13.2, etc  onto 4.x-cdh5
> > > > ~HEAD.
> > > > >
> > > > >
> > > > >
> > > > > On 10 Mar 2018 05:36, "James Taylor" <jamestaylor@apache.org>
> wrote:
> > > > >
> > > > >> If you're willing to sign up to keep the CDH branches in sync
with
> > the
> > > > >> 4.x-HBase-1.2 branches, then that seems fine. Should we look
into
> > some
> > > > >> kind
> > > > >> of automation for syncing all the branches?
> > > > >>
> > > > >> Another approach is a shim layer, but we've been reluctant to
set
> > that
> > > > up
> > > > >> as there's a fair amount of initial overhead to implement.
> > > > >>
> > > > >> On Thu, Mar 8, 2018 at 4:24 PM, Pedro Boado <pboado@apache.org>
> > > wrote:
> > > > >>
> > > > >> > Hi all,
> > > > >> >
> > > > >> > After our first release of Phoenix for CDH 5.11.2 it looks
like
> a
> > > good
> > > > >> idea
> > > > >> > expanding support to other newer versions. I'd like to discuss
> > with
> > > > you
> > > > >> > guys what would be the best approach as we have several
options:
> > > > >> >
> > > > >> > - Releasing one single generated parcel with wider
> compatibility (
> > > > >> > cloudera-labs phoenix style ) . The issue here is that all
> > > transitive
> > > > >> > dependencies packaged in phoenix fatjars would be specific
to a
> > cdh
> > > > >> version
> > > > >> > (let's say, cdh5.11.2) but would be running against different
> cdh
> > > > >> version
> > > > >> > (maybe cdh5.14.0 ) . There is a small chance of incompatibility
> > > across
> > > > >> > versions ( even when all of them are HBase 1.2 based ) .
Also we
> > > > >> wouldn't
> > > > >> > be running our IT against all these cdh versions.
> > > > >> >
> > > > >> > - Maybe work further into packaging (and removing) any
> dependency
> > > that
> > > > >> is
> > > > >> > already shipped in cdh. That would improve compatibility
of this
> > > > unique
> > > > >> > parcel version across cdh releases. But fat client fatjars
would
> > > still
> > > > >> be a
> > > > >> > problem for use from out of hadoop cluster.
> > > > >> >
> > > > >> > - Release several parcels specific to different cdh versions
(
> my
> > > > >> favourite
> > > > >> > option ) . That is the safest option for better compatibility
as
> > we
> > > > >> would
> > > > >> > be shipping the exact same libraries in the parcel as used
in
> that
> > > > >> version
> > > > >> > of cdh. And we'd also be doing IT for several cdh versions.
> > Downside
> > > > is
> > > > >> a
> > > > >> > little bit more of release effort ( not a big deal ) , more
> > branches
> > > > in
> > > > >> git
> > > > >> > and  more web server space needed for keeping several parcels
(
> > I'm
> > > > not
> > > > >> > sure if that is an issue )
> > > > >> >
> > > > >> > All ideas are welcome.
> > > > >> >
> > > > >> > Thanks!
> > > > >> >
> > > > >>
> > > > >
> > > >
> > > >
> > > > --
> > > > Un saludo.
> > > > Pedro Boado.
> > > >
> > >
> >
>
>
>
> --
> Un saludo.
> Pedro Boado.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message