hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Dimiduk <ndimi...@gmail.com>
Subject Re: [DISCUSSION] "Convenience binaries" bundling Phoenix
Date Thu, 19 Mar 2015 00:15:18 GMT
On Wed, Mar 18, 2015 at 4:32 PM, Andrew Purtell <apurtell@apache.org> wrote:

> A better analogy would be Pig deciding to ship DataFu UDFs in a
> convenience artifact.
>

A swaying argument. It still seems to me a bit like putting the cart in
front of the horse, given the dependency DAG. Does the eventual inclusion
of a Phoenix Query Server (PHOENIX-971) impact your opinion on this?

On Wed, Mar 18, 2015 at 2:45 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:
>
> > ​​
> > HBase shipping a tgz containing Phoenix sounds backwards to me. It seems
> > like Hadoop project shipping a tgz with Hive included as a convenience.
> > Such convenience packages would be great, but I don't think it's HBase
> > project's responsibility to do so any more than it would be Hadoop
> > project's responsibility in my above example. Upstreamers packaging
> > downstreamers sounds backwards to me.
> >
> > On the other hand, I think it makes a lot of sense for *someone* to ship
> a
> > "batteries included"Phoenix tgz with Hadoop and HBase. If anyone, that
> > someone should be Phoenix. I don't think it that unreasonable for Phoenix
> > to do this.
> >
> > Or have a 3rd party packaging, as I mentioned before. I don't know what
> > BigTop packages, but such a bundle seems reasonable for that project to
> > produce, as one of many "batteries included" Apache products.
> >
> > On Wednesday, March 18, 2015, lars hofhansl <larsh@apache.org> wrote:
> >
> > > Again... 'bit late chiming in.
> > >
> > > I'd be +1 on this. We'd essentially just ship HBase with an extra
> shell.I
> > > don't think Phoenix can do this, because it does not usually ship the
> > bits
> > > needed to run HBase, but just the bits to add to HBase to have
> > > Phoenix.Phoenix should not be in the business to maintain an HBase
> > > distribution.
> > > One could argue that we should not be in the business to maintain a
> > > Phoenix distribution. But we wouldn't really. We'd just add the Phoenix
> > jar
> > > to our binary distro, pre-hook the coprocessors, and add the SQLLine,
> and
> > > psql.On the HBase side I'd suggest we'd add a script to dev-support to
> > > package up the HBase+Phoenix distro, and maybe we add a new target to
> the
> > > pom to pull a Phoenix version.
> > >
> > > What exactly we'd ship should be discussed (all the Phoenix tools, or
> > just
> > > some?) Giving HBase a nice SQL shell seems cool to me, and that's were
> > some
> > > more of the more hairy issues might be found.
> > >
> > > -- Lars
> > >
> > >       From: Andrew Purtell <apurtell@apache.org <javascript:;>>
> > >  To: "dev@hbase.apache.org <javascript:;>" <dev@hbase.apache.org
> > > <javascript:;>>
> > >  Sent: Tuesday, March 17, 2015 12:44 PM
> > >  Subject: [DISCUSSION] "Convenience binaries" bundling Phoenix
> > >
> > > Consider if the HBase project starts releasing new "convenience
> > binaries",
> > > in addition to the existing ones, in which we bundle a
> > recent/vetted/stable
> > > version of Phoenix, with the site file changes for loading their
> > > coprocessors already patched in (to hbase-default.xml) For now this
> would
> > > be done for 0.98 only, since that's the only release line supported by
> an
> > > actively developed Phoenix version. We could also do this for 0.94
> > releases
> > > with Phoenix 3 if the 0.94 RM wants, but I doubt there would be any
> > demand
> > > for this, Phoenix 3 is inactive because that community has all moved to
> > 4,
> > > I'd imagine that carries over here.
> > >
> > > Advantages:
> > >
> > > - HBase would ship with a SQL access option. There's the Phoenix JDBC
> > > driver of course, and we'd also bundle the psql and sqlline exec
> wrappers
> > > from the Phoenix binary distribution. We'd have both the jruby shell
> and
> > a
> > > SQL shell, this is a powerful combination.
> > >
> > > - HBase ships with a library that assists users in making efficient
> > queries
> > > if their data is typed, but this doesn't include the server side
> > > optimizations that the Phoenix coprocessors provide, and in that case
> no
> > > hand rolling is necessary.
> > >
> > > - HBase would ship with secondary indexes. These would not cover all
> > > possible use cases and requirements, let's stipulate that now and hope
> > this
> > > doesn't kick off another circular discussion on that front.
> > Unquestionably
> > > this is a compelling Phoenix feature so some use cases obviously can
> > > benefit, and if users find the combined distribution useful enough we
> > don't
> > > have to discuss secondary indexes in HBase core again.
> > >
> > > - We will have done the necessary integration work for the combined
> > result
> > > to be easy to use. Apache software cat herders will appreciate this.
> > >
> > > - It's totally optional, simply ignore the new binary packages if you
> > don't
> > > care. This is not a Grand Unification proposal.
> > >
> > > Concerns:
> > >
> > > - More work for the RM. Unquestionably.
> > >
> > > - Concerns about the quality of the combined convenience artifact: Is
> > there
> > > an implied warranty? Could we disclaim? Should we disclaim? If not, how
> > > does HBase do QA on this. Related to the above concern about RM
> > bandwidth.
> > > Maybe Phoenix could help.
> > >
> > > - Increased coupling between the projects. Frankly, I think this
> already
> > > there, we just don't see it until we trip over issues that could have
> > been
> > > avoided with more communication between projects. Pushing on Phoenix
> for
> > > bits for a monthly HBase release cadence will surface issues faster and
> > > improve communication between the projects. This benefits Phoenix with
> > more
> > > QA bandwidth. This benefits HBase because we see Phoenix bringing in a
> > > significant number of users.
> > >
> > > - We may want to revisit again normalizing type support in HBase's
> client
> > > library and Phoenix, eventually.
> > >
> > > I could add more items to the advantage or concern lists but mainly
> want
> > to
> > > float the idea for feedback at this time.
> > >
> > > Thoughts?
> > >
> > > --
> > > Best regards,
> > >
> > >   - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> > >
> > >
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message