hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: [DISCUSSION] "Convenience binaries" bundling Phoenix
Date Thu, 19 Mar 2015 19:08:03 GMT
The idea I am proposing now is the inclusion of Phoenix bits into a
convenience artifact that results in a SQL access skin for HBase. What we
have available today for that are the coprocessors, the JDBC driver, and
sqlline. Whether or not we'd want to include a query server as another type
of gateway and connector, like REST or Thrift, could be a future topic of
discussion, but is out of scope of my proposal, certainly now because it
doesn't exist yet, and even later when it does.

On Wed, Mar 18, 2015 at 5:15 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:

> On Wed, Mar 18, 2015 at 4:32 PM, Andrew Purtell <apurtell@apache.org>
> wrote:
>
> > A better analogy would be Pig deciding to ship DataFu UDFs in a
> > convenience artifact.
> >
>
> A swaying argument. It still seems to me a bit like putting the cart in
> front of the horse, given the dependency DAG. Does the eventual inclusion
> of a Phoenix Query Server (PHOENIX-971) impact your opinion on this?
>
> On Wed, Mar 18, 2015 at 2:45 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:
> >
> > > ​​
> > > HBase shipping a tgz containing Phoenix sounds backwards to me. It
> seems
> > > like Hadoop project shipping a tgz with Hive included as a convenience.
> > > Such convenience packages would be great, but I don't think it's HBase
> > > project's responsibility to do so any more than it would be Hadoop
> > > project's responsibility in my above example. Upstreamers packaging
> > > downstreamers sounds backwards to me.
> > >
> > > On the other hand, I think it makes a lot of sense for *someone* to
> ship
> > a
> > > "batteries included"Phoenix tgz with Hadoop and HBase. If anyone, that
> > > someone should be Phoenix. I don't think it that unreasonable for
> Phoenix
> > > to do this.
> > >
> > > Or have a 3rd party packaging, as I mentioned before. I don't know what
> > > BigTop packages, but such a bundle seems reasonable for that project to
> > > produce, as one of many "batteries included" Apache products.
> > >
> > > On Wednesday, March 18, 2015, lars hofhansl <larsh@apache.org> wrote:
> > >
> > > > Again... 'bit late chiming in.
> > > >
> > > > I'd be +1 on this. We'd essentially just ship HBase with an extra
> > shell.I
> > > > don't think Phoenix can do this, because it does not usually ship the
> > > bits
> > > > needed to run HBase, but just the bits to add to HBase to have
> > > > Phoenix.Phoenix should not be in the business to maintain an HBase
> > > > distribution.
> > > > One could argue that we should not be in the business to maintain a
> > > > Phoenix distribution. But we wouldn't really. We'd just add the
> Phoenix
> > > jar
> > > > to our binary distro, pre-hook the coprocessors, and add the SQLLine,
> > and
> > > > psql.On the HBase side I'd suggest we'd add a script to dev-support
> to
> > > > package up the HBase+Phoenix distro, and maybe we add a new target to
> > the
> > > > pom to pull a Phoenix version.
> > > >
> > > > What exactly we'd ship should be discussed (all the Phoenix tools, or
> > > just
> > > > some?) Giving HBase a nice SQL shell seems cool to me, and that's
> were
> > > some
> > > > more of the more hairy issues might be found.
> > > >
> > > > -- Lars
> > > >
> > > >       From: Andrew Purtell <apurtell@apache.org <javascript:;>>
> > > >  To: "dev@hbase.apache.org <javascript:;>" <dev@hbase.apache.org
> > > > <javascript:;>>
> > > >  Sent: Tuesday, March 17, 2015 12:44 PM
> > > >  Subject: [DISCUSSION] "Convenience binaries" bundling Phoenix
> > > >
> > > > Consider if the HBase project starts releasing new "convenience
> > > binaries",
> > > > in addition to the existing ones, in which we bundle a
> > > recent/vetted/stable
> > > > version of Phoenix, with the site file changes for loading their
> > > > coprocessors already patched in (to hbase-default.xml) For now this
> > would
> > > > be done for 0.98 only, since that's the only release line supported
> by
> > an
> > > > actively developed Phoenix version. We could also do this for 0.94
> > > releases
> > > > with Phoenix 3 if the 0.94 RM wants, but I doubt there would be any
> > > demand
> > > > for this, Phoenix 3 is inactive because that community has all moved
> to
> > > 4,
> > > > I'd imagine that carries over here.
> > > >
> > > > Advantages:
> > > >
> > > > - HBase would ship with a SQL access option. There's the Phoenix JDBC
> > > > driver of course, and we'd also bundle the psql and sqlline exec
> > wrappers
> > > > from the Phoenix binary distribution. We'd have both the jruby shell
> > and
> > > a
> > > > SQL shell, this is a powerful combination.
> > > >
> > > > - HBase ships with a library that assists users in making efficient
> > > queries
> > > > if their data is typed, but this doesn't include the server side
> > > > optimizations that the Phoenix coprocessors provide, and in that case
> > no
> > > > hand rolling is necessary.
> > > >
> > > > - HBase would ship with secondary indexes. These would not cover all
> > > > possible use cases and requirements, let's stipulate that now and
> hope
> > > this
> > > > doesn't kick off another circular discussion on that front.
> > > Unquestionably
> > > > this is a compelling Phoenix feature so some use cases obviously can
> > > > benefit, and if users find the combined distribution useful enough we
> > > don't
> > > > have to discuss secondary indexes in HBase core again.
> > > >
> > > > - We will have done the necessary integration work for the combined
> > > result
> > > > to be easy to use. Apache software cat herders will appreciate this.
> > > >
> > > > - It's totally optional, simply ignore the new binary packages if you
> > > don't
> > > > care. This is not a Grand Unification proposal.
> > > >
> > > > Concerns:
> > > >
> > > > - More work for the RM. Unquestionably.
> > > >
> > > > - Concerns about the quality of the combined convenience artifact: Is
> > > there
> > > > an implied warranty? Could we disclaim? Should we disclaim? If not,
> how
> > > > does HBase do QA on this. Related to the above concern about RM
> > > bandwidth.
> > > > Maybe Phoenix could help.
> > > >
> > > > - Increased coupling between the projects. Frankly, I think this
> > already
> > > > there, we just don't see it until we trip over issues that could have
> > > been
> > > > avoided with more communication between projects. Pushing on Phoenix
> > for
> > > > bits for a monthly HBase release cadence will surface issues faster
> and
> > > > improve communication between the projects. This benefits Phoenix
> with
> > > more
> > > > QA bandwidth. This benefits HBase because we see Phoenix bringing in
> a
> > > > significant number of users.
> > > >
> > > > - We may want to revisit again normalizing type support in HBase's
> > client
> > > > library and Phoenix, eventually.
> > > >
> > > > I could add more items to the advantage or concern lists but mainly
> > want
> > > to
> > > > float the idea for feedback at this time.
> > > >
> > > > Thoughts?
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > >   - Andy
> > > >
> > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > > (via Tom White)
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message