hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: [DISCUSSION] "Convenience binaries" bundling Phoenix
Date Wed, 18 Mar 2015 23:32:20 GMT
>
​
HBase shipping a tgz containing Phoenix sounds backwards to me. It seems like
Hadoop project shipping a tgz with Hive included as a convenience.

I think the situation is different because Phoenix is a coprocessor
application, and as a project only produces coprocessor jars that require
an HBase distribution from somewhere else. A better analogy would be Pig
deciding to ship DataFu UDFs in a convenience artifact.


On Wed, Mar 18, 2015 at 2:45 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:

> ​​
> HBase shipping a tgz containing Phoenix sounds backwards to me. It seems
> like Hadoop project shipping a tgz with Hive included as a convenience.
> Such convenience packages would be great, but I don't think it's HBase
> project's responsibility to do so any more than it would be Hadoop
> project's responsibility in my above example. Upstreamers packaging
> downstreamers sounds backwards to me.
>
> On the other hand, I think it makes a lot of sense for *someone* to ship a
> "batteries included"Phoenix tgz with Hadoop and HBase. If anyone, that
> someone should be Phoenix. I don't think it that unreasonable for Phoenix
> to do this.
>
> Or have a 3rd party packaging, as I mentioned before. I don't know what
> BigTop packages, but such a bundle seems reasonable for that project to
> produce, as one of many "batteries included" Apache products.
>
> On Wednesday, March 18, 2015, lars hofhansl <larsh@apache.org> wrote:
>
> > Again... 'bit late chiming in.
> >
> > I'd be +1 on this. We'd essentially just ship HBase with an extra shell.I
> > don't think Phoenix can do this, because it does not usually ship the
> bits
> > needed to run HBase, but just the bits to add to HBase to have
> > Phoenix.Phoenix should not be in the business to maintain an HBase
> > distribution.
> > One could argue that we should not be in the business to maintain a
> > Phoenix distribution. But we wouldn't really. We'd just add the Phoenix
> jar
> > to our binary distro, pre-hook the coprocessors, and add the SQLLine, and
> > psql.On the HBase side I'd suggest we'd add a script to dev-support to
> > package up the HBase+Phoenix distro, and maybe we add a new target to the
> > pom to pull a Phoenix version.
> >
> > What exactly we'd ship should be discussed (all the Phoenix tools, or
> just
> > some?) Giving HBase a nice SQL shell seems cool to me, and that's were
> some
> > more of the more hairy issues might be found.
> >
> > -- Lars
> >
> >       From: Andrew Purtell <apurtell@apache.org <javascript:;>>
> >  To: "dev@hbase.apache.org <javascript:;>" <dev@hbase.apache.org
> > <javascript:;>>
> >  Sent: Tuesday, March 17, 2015 12:44 PM
> >  Subject: [DISCUSSION] "Convenience binaries" bundling Phoenix
> >
> > Consider if the HBase project starts releasing new "convenience
> binaries",
> > in addition to the existing ones, in which we bundle a
> recent/vetted/stable
> > version of Phoenix, with the site file changes for loading their
> > coprocessors already patched in (to hbase-default.xml) For now this would
> > be done for 0.98 only, since that's the only release line supported by an
> > actively developed Phoenix version. We could also do this for 0.94
> releases
> > with Phoenix 3 if the 0.94 RM wants, but I doubt there would be any
> demand
> > for this, Phoenix 3 is inactive because that community has all moved to
> 4,
> > I'd imagine that carries over here.
> >
> > Advantages:
> >
> > - HBase would ship with a SQL access option. There's the Phoenix JDBC
> > driver of course, and we'd also bundle the psql and sqlline exec wrappers
> > from the Phoenix binary distribution. We'd have both the jruby shell and
> a
> > SQL shell, this is a powerful combination.
> >
> > - HBase ships with a library that assists users in making efficient
> queries
> > if their data is typed, but this doesn't include the server side
> > optimizations that the Phoenix coprocessors provide, and in that case no
> > hand rolling is necessary.
> >
> > - HBase would ship with secondary indexes. These would not cover all
> > possible use cases and requirements, let's stipulate that now and hope
> this
> > doesn't kick off another circular discussion on that front.
> Unquestionably
> > this is a compelling Phoenix feature so some use cases obviously can
> > benefit, and if users find the combined distribution useful enough we
> don't
> > have to discuss secondary indexes in HBase core again.
> >
> > - We will have done the necessary integration work for the combined
> result
> > to be easy to use. Apache software cat herders will appreciate this.
> >
> > - It's totally optional, simply ignore the new binary packages if you
> don't
> > care. This is not a Grand Unification proposal.
> >
> > Concerns:
> >
> > - More work for the RM. Unquestionably.
> >
> > - Concerns about the quality of the combined convenience artifact: Is
> there
> > an implied warranty? Could we disclaim? Should we disclaim? If not, how
> > does HBase do QA on this. Related to the above concern about RM
> bandwidth.
> > Maybe Phoenix could help.
> >
> > - Increased coupling between the projects. Frankly, I think this already
> > there, we just don't see it until we trip over issues that could have
> been
> > avoided with more communication between projects. Pushing on Phoenix for
> > bits for a monthly HBase release cadence will surface issues faster and
> > improve communication between the projects. This benefits Phoenix with
> more
> > QA bandwidth. This benefits HBase because we see Phoenix bringing in a
> > significant number of users.
> >
> > - We may want to revisit again normalizing type support in HBase's client
> > library and Phoenix, eventually.
> >
> > I could add more items to the advantage or concern lists but mainly want
> to
> > float the idea for feedback at this time.
> >
> > Thoughts?
> >
> > --
> > Best regards,
> >
> >   - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
> >
> >
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message