hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Beaudreault <bbeaudrea...@hubspot.com>
Subject Re: [DISCUSSION] "Convenience binaries" bundling Phoenix
Date Thu, 19 Mar 2015 19:30:32 GMT
As a user I do think this would be pretty convenient, but I wonder if it
sets a bad precedent.  What sets Phoenix apart from other popular
coprocessors in the future? Should they be included too? If so it then adds
stress on you guys to come up with guidelines that must be met to be
considered "production ready", and then vet each of the pre-packaged
plugins against that at each release.

I'm wondering if this is something that more naturally lies in the various
distributions' hands (CDH, Horton, etc).  They already do the work to make
sure X works with Y at a particular version, backporting important
security/performance fixes, etc.  It seems reasonable for them to package a
particular version of phoenix that has been tested with a particular
version of the entire stack. I'm not sure what the process would be to
suggest this to them.

I know a bunch of you already work at the various companies that make
distributions, so the work might eventually fall to you anyway.  But it at
least sets a clear separation that more closely maps to how things have
historically seemed to work for other parts of the apache/hadoop stack.

On Thu, Mar 19, 2015 at 3:08 PM, Andrew Purtell <apurtell@apache.org> wrote:

> The idea I am proposing now is the inclusion of Phoenix bits into a
> convenience artifact that results in a SQL access skin for HBase. What we
> have available today for that are the coprocessors, the JDBC driver, and
> sqlline. Whether or not we'd want to include a query server as another type
> of gateway and connector, like REST or Thrift, could be a future topic of
> discussion, but is out of scope of my proposal, certainly now because it
> doesn't exist yet, and even later when it does.
>
> On Wed, Mar 18, 2015 at 5:15 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:
>
> > On Wed, Mar 18, 2015 at 4:32 PM, Andrew Purtell <apurtell@apache.org>
> > wrote:
> >
> > > A better analogy would be Pig deciding to ship DataFu UDFs in a
> > > convenience artifact.
> > >
> >
> > A swaying argument. It still seems to me a bit like putting the cart in
> > front of the horse, given the dependency DAG. Does the eventual inclusion
> > of a Phoenix Query Server (PHOENIX-971) impact your opinion on this?
> >
> > On Wed, Mar 18, 2015 at 2:45 PM, Nick Dimiduk <ndimiduk@gmail.com>
> wrote:
> > >
> > > > ​​
> > > > HBase shipping a tgz containing Phoenix sounds backwards to me. It
> > seems
> > > > like Hadoop project shipping a tgz with Hive included as a
> convenience.
> > > > Such convenience packages would be great, but I don't think it's
> HBase
> > > > project's responsibility to do so any more than it would be Hadoop
> > > > project's responsibility in my above example. Upstreamers packaging
> > > > downstreamers sounds backwards to me.
> > > >
> > > > On the other hand, I think it makes a lot of sense for *someone* to
> > ship
> > > a
> > > > "batteries included"Phoenix tgz with Hadoop and HBase. If anyone,
> that
> > > > someone should be Phoenix. I don't think it that unreasonable for
> > Phoenix
> > > > to do this.
> > > >
> > > > Or have a 3rd party packaging, as I mentioned before. I don't know
> what
> > > > BigTop packages, but such a bundle seems reasonable for that project
> to
> > > > produce, as one of many "batteries included" Apache products.
> > > >
> > > > On Wednesday, March 18, 2015, lars hofhansl <larsh@apache.org>
> wrote:
> > > >
> > > > > Again... 'bit late chiming in.
> > > > >
> > > > > I'd be +1 on this. We'd essentially just ship HBase with an extra
> > > shell.I
> > > > > don't think Phoenix can do this, because it does not usually ship
> the
> > > > bits
> > > > > needed to run HBase, but just the bits to add to HBase to have
> > > > > Phoenix.Phoenix should not be in the business to maintain an HBase
> > > > > distribution.
> > > > > One could argue that we should not be in the business to maintain
a
> > > > > Phoenix distribution. But we wouldn't really. We'd just add the
> > Phoenix
> > > > jar
> > > > > to our binary distro, pre-hook the coprocessors, and add the
> SQLLine,
> > > and
> > > > > psql.On the HBase side I'd suggest we'd add a script to dev-support
> > to
> > > > > package up the HBase+Phoenix distro, and maybe we add a new target
> to
> > > the
> > > > > pom to pull a Phoenix version.
> > > > >
> > > > > What exactly we'd ship should be discussed (all the Phoenix tools,
> or
> > > > just
> > > > > some?) Giving HBase a nice SQL shell seems cool to me, and that's
> > were
> > > > some
> > > > > more of the more hairy issues might be found.
> > > > >
> > > > > -- Lars
> > > > >
> > > > >       From: Andrew Purtell <apurtell@apache.org <javascript:;>>
> > > > >  To: "dev@hbase.apache.org <javascript:;>" <dev@hbase.apache.org
> > > > > <javascript:;>>
> > > > >  Sent: Tuesday, March 17, 2015 12:44 PM
> > > > >  Subject: [DISCUSSION] "Convenience binaries" bundling Phoenix
> > > > >
> > > > > Consider if the HBase project starts releasing new "convenience
> > > > binaries",
> > > > > in addition to the existing ones, in which we bundle a
> > > > recent/vetted/stable
> > > > > version of Phoenix, with the site file changes for loading their
> > > > > coprocessors already patched in (to hbase-default.xml) For now this
> > > would
> > > > > be done for 0.98 only, since that's the only release line supported
> > by
> > > an
> > > > > actively developed Phoenix version. We could also do this for 0.94
> > > > releases
> > > > > with Phoenix 3 if the 0.94 RM wants, but I doubt there would be any
> > > > demand
> > > > > for this, Phoenix 3 is inactive because that community has all
> moved
> > to
> > > > 4,
> > > > > I'd imagine that carries over here.
> > > > >
> > > > > Advantages:
> > > > >
> > > > > - HBase would ship with a SQL access option. There's the Phoenix
> JDBC
> > > > > driver of course, and we'd also bundle the psql and sqlline exec
> > > wrappers
> > > > > from the Phoenix binary distribution. We'd have both the jruby
> shell
> > > and
> > > > a
> > > > > SQL shell, this is a powerful combination.
> > > > >
> > > > > - HBase ships with a library that assists users in making efficient
> > > > queries
> > > > > if their data is typed, but this doesn't include the server side
> > > > > optimizations that the Phoenix coprocessors provide, and in that
> case
> > > no
> > > > > hand rolling is necessary.
> > > > >
> > > > > - HBase would ship with secondary indexes. These would not cover
> all
> > > > > possible use cases and requirements, let's stipulate that now and
> > hope
> > > > this
> > > > > doesn't kick off another circular discussion on that front.
> > > > Unquestionably
> > > > > this is a compelling Phoenix feature so some use cases obviously
> can
> > > > > benefit, and if users find the combined distribution useful enough
> we
> > > > don't
> > > > > have to discuss secondary indexes in HBase core again.
> > > > >
> > > > > - We will have done the necessary integration work for the combined
> > > > result
> > > > > to be easy to use. Apache software cat herders will appreciate
> this.
> > > > >
> > > > > - It's totally optional, simply ignore the new binary packages if
> you
> > > > don't
> > > > > care. This is not a Grand Unification proposal.
> > > > >
> > > > > Concerns:
> > > > >
> > > > > - More work for the RM. Unquestionably.
> > > > >
> > > > > - Concerns about the quality of the combined convenience artifact:
> Is
> > > > there
> > > > > an implied warranty? Could we disclaim? Should we disclaim? If not,
> > how
> > > > > does HBase do QA on this. Related to the above concern about RM
> > > > bandwidth.
> > > > > Maybe Phoenix could help.
> > > > >
> > > > > - Increased coupling between the projects. Frankly, I think this
> > > already
> > > > > there, we just don't see it until we trip over issues that could
> have
> > > > been
> > > > > avoided with more communication between projects. Pushing on
> Phoenix
> > > for
> > > > > bits for a monthly HBase release cadence will surface issues faster
> > and
> > > > > improve communication between the projects. This benefits Phoenix
> > with
> > > > more
> > > > > QA bandwidth. This benefits HBase because we see Phoenix bringing
> in
> > a
> > > > > significant number of users.
> > > > >
> > > > > - We may want to revisit again normalizing type support in HBase's
> > > client
> > > > > library and Phoenix, eventually.
> > > > >
> > > > > I could add more items to the advantage or concern lists but mainly
> > > want
> > > > to
> > > > > float the idea for feedback at this time.
> > > > >
> > > > > Thoughts?
> > > > >
> > > > > --
> > > > > Best regards,
> > > > >
> > > > >   - Andy
> > > > >
> > > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > > Hein
> > > > > (via Tom White)
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message