hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: Ruby shell versions for HBase 2.0
Date Wed, 13 May 2015 23:53:43 GMT
> Only shell we could
​> ​
swap in w/o annoying users would be SQL (ducks);
​[...]
 You would
​> ​
also have to manage expectations: i.e. that the SQL would be extremely
​> ​
basic and that the REPL does not come with an idling 100 node  cluster
​> ​
ready to take up any involved queries.

If we are talking about adding a SQL shell, it doesn't have to be extremely
basic, we can revisit my proposal from the other month about embedding
Phoenix.


On Wed, May 13, 2015 at 4:50 PM, Stack <stack@duboce.net> wrote:

> On Wed, May 13, 2015 at 1:38 PM, Andrew Purtell <apurtell@apache.org>
> wrote:
>
> > > That's only if you assume HBase users actually understand Ruby though,
> > no?
> >
> > Yes, and this is a fair point.
> >
> > The language REPL in which we're embedding the shell DSL doesn't have to
> be
> > Ruby. JavaScript would work, maybe vi
> ​​
> a Nashorn. Python would work, via
> > Jython. Or Scala's REPL, even.
> >
> >
> >
> JRuby had best interpreter and the others considered had extra baggage
> doing a DSL (For history, see HBASE-487). Having to put tabs in your shell
> doing loops and no jline at the time made jython a non-runner though I
> wanted it to win (Has anything changed here?) JS is probably way better
> than when I looked at it back then?
>
> Regardless, whether jython, JS, scala, or custom, IMO, we'd just be pissing
> of users if we swap one idiosyncratic for another.  Only shell we could
> swap in w/o annoying users would be SQL (ducks); and even here you'd
> probably have to implement a bunch of operators on the other side of
> sqllines bang operator (or equivalent) to do the hbase specifics. You would
> also have to manage expectations: i.e. that the SQL would be extremely
> basic and that the REPL does not come with an idling 100 node  cluster
> ready to take up any involved queries.
>
> St.Ack
>
>
>
>
> > On Wed, May 13, 2015 at 1:10 PM, Josh Elser <josh.elser@gmail.com>
> wrote:
> >
> > > Andrew Purtell wrote:
> > >
> > >> This is because the Accumulo shell is a custom built shell? If so, we
> > had
> > >> one of those once and replaced it with the IRB based one. We didn't
> > settle
> > >> on JRuby right away but, at the time, the consensus was we didn't want
> > to
> > >> be in the business of maintaining yet another REPL when specialist
> open
> > >> source communities have already done that. Why has this changed? Has
> it?
> > >>
> > >
> > > Accumulo shell is just a Java program that uses JLine. It has no
> control
> > > flow structures, variables, or anything like that.
> > >
> > >  Why is the Accumulo shell easier to script with? Does it have control
> > flow
> > >> constructs? Variable assignment? Or is it easier somehow because it
> does
> > >> not have those things? Even the system shell has those. Hmm... Are we
> > >> saying that any control flow or variables/substitution should be done
> by
> > >> calling our shell from a bash or sh script? Wouldn't that be super
> slow
> > >> given Java's startup time for every invocation of our shell? A bit
> like
> > >> trying to script the AWS command line utilities, which is
> excruciating.
> > >>
> > >
> > > This is what I was trying to get at: the lack of typical language
> > > constructs is both beneficial and irritating. I think it depends on
> what
> > > you really want to do. Does that mean trying to maintain two shells is
> > > worth it? I'm not sure, but I'm also not sure where the happy medium
> is.
> > >
> > > If you exec the shell to just run one command at a time, yes. It is
> slow.
> > > You can redirect input or provide a file of commands to run at one time
> > > which don't suffer from the JVM startup problem.
> > >
> > >  I can see why a custom shell built with Java would easier to test
> where
> > >> the
> > >> rest of the project is using Surefire/JUnit. That's a developer
> > >> convenience
> > >> concern though, not a user convenience one.
> > >>
> > >
> > > That's only if you assume HBase users actually understand Ruby though,
> > no?
> > > I would think that something which acts like your normal unix shell
> would
> > > be familiar to users w/o the need to understand some other language.
> > >
> > >
> > >
> > >> On Wed, May 13, 2015 at 10:41 AM, Sean Busbey<busbey@cloudera.com>
> > >> wrote:
> > >>
> > >>  Pros:
> > >>>
> > >>> * It's easier to test and maintain
> > >>> * it's easier to script with
> > >>> * its interactive mode "feels" mode focused on the task of
> interacting
> > >>> with
> > >>> a cluster to me (maybe this is just acting like a psql or mysql
> shell)
> > >>>
> > >>> Cons
> > >>>
> > >>> * adding custom commands requires knowing java
> > >>>
> > >>> --
> > >>> Sean
> > >>> On May 13, 2015 12:31 PM, "Andrew Purtell"<apurtell@apache.org>
> > wrote:
> > >>>
> > >>>  Why is the Accumulo shell superior?
> > >>>>
> > >>>> Is it scriptable?
> > >>>>
> > >>>>
> > >>>> On Wed, May 13, 2015 at 10:28 AM, Sean Busbey<busbey@cloudera.com>
> > >>>>
> > >>> wrote:
> > >>>
> > >>>> I would love to rip out the JRuby shell entirely and make something
> > >>>>>
> > >>>> closer
> > >>>>
> > >>>>> to the Accumulo shell, but I expect that will
> > >>>>>
> > >>>>> 1) be way more work
> > >>>>>
> > >>>>> 2) be even less compatible for those that rely on customizations.
> > >>>>>
> > >>>>> I figured given time we could get a preview "user shell" (rather
> than
> > >>>>> "power shell" via irb) together in 2.0 and aim for default
in 3.0.
> > >>>>>
> > >>>>> --
> > >>>>> Sean
> > >>>>> On May 13, 2015 12:19 PM, "Stack"<stack@duboce.net> 
wrote:
> > >>>>>
> > >>>>>  Nice writeup Sean.
> > >>>>>>
> > >>>>>> Yeah, +1 to new jruby in hbase 2.0. We'd need to be careful
> license
> > >>>>>>
> > >>>>> is
> > >>>
> > >>>> still amenable and hopefully jruby 9k will be slimmer than jruby
> > >>>>>>
> > >>>>> 1.7+.
> > >>>
> > >>>> But if we are going to do a significant shell refactor for hbase
> 2.0,
> > >>>>>> should we consider doing something more radical; e.g. a
new shell?
> > If
> > >>>>>> interest, could start up a new thread so don't distract
from this
> > >>>>>>
> > >>>>> one.
> > >>>
> > >>>> St.Ack
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> On Wed, May 13, 2015 at 9:22 AM, Sean Busbey<busbey@cloudera.com>
> > >>>>>>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Hi folks!
> > >>>>>>>
> > >>>>>>> If you weren't aware, our current shell relies on Ruby,
> > >>>>>>>
> > >>>>>> specifically
> > >>>
> > >>>> the
> > >>>>>
> > >>>>>> REPL program IRB from JRuby. When we launched 1.0 we were
on JRuby
> > >>>>>>>
> > >>>>>> 1.6
> > >>>>
> > >>>>> with
> > >>>>>>
> > >>>>>>> defaults, which means we're stuck on Ruby 1.8.
> > >>>>>>>
> > >>>>>>> For those that don't already know, Ruby 1.8 is super
old and has
> > >>>>>>>
> > >>>>>> been
> > >>>
> > >>>> walking off into the sunset for a few years now. Most (but not
> > >>>>>>>
> > >>>>>> all!)
> > >>>
> > >>>> formal
> > >>>>>>
> > >>>>>>> support systems for running Ruby have EOLed 1.8 and
there are
> > >>>>>>>
> > >>>>>> numerous
> > >>>>
> > >>>>> known issues with it.
> > >>>>>>>
> > >>>>>>> Right now there's an open ticket to get us on JRuby
1.7 so that
> our
> > >>>>>>>
> > >>>>>> shell
> > >>>>>
> > >>>>>> can work on PPC systems[1]. That version of JRuby defaults
to Ruby
> > >>>>>>>
> > >>>>>> 1.9
> > >>>>
> > >>>>> but
> > >>>>>>
> > >>>>>>> can be run in Ruby 1.8 mode. There are some implementation
> details
> > >>>>>>> outstanding, but I'm hoping that ticket can work out
such that it
> > >>>>>>>
> > >>>>>> can
> > >>>
> > >>>> land
> > >>>>>>
> > >>>>>>> in branch-1.
> > >>>>>>>
> > >>>>>>> For HBase 2.0, I'd like us to plan for a little farther
out in
> the
> > >>>>>>>
> > >>>>>> future
> > >>>>>
> > >>>>>> than just updating to Ruby 1.9 (though that would be a
fine
> > >>>>>>>
> > >>>>>> incremental
> > >>>>
> > >>>>> step with some non-trivial work attached). The "current" version
of
> > >>>>>>>
> > >>>>>> Ruby
> > >>>>>
> > >>>>>> is
> > >>>>>>
> > >>>>>>> 2.2. Much like the move from 1.8 ->  1.9 it is not
backwards
> > >>>>>>>
> > >>>>>> compatible.
> > >>>>
> > >>>>> JRuby's next major maintenance release line is "version 9000"[2]
> > >>>>>>>
> > >>>>>> and
> > >>>
> > >>>> it
> > >>>>
> > >>>>> will start out *only* supporting Ruby 2.2. Right now JRuby
9000 is
> > >>>>>>>
> > >>>>>> in
> > >>>
> > >>>> its
> > >>>>>
> > >>>>>> second "preview" release. They still have a few feature
complete
> > >>>>>>>
> > >>>>>> items
> > >>>>
> > >>>>> to
> > >>>>>
> > >>>>>> address before they hit their first GA release.
> > >>>>>>>
> > >>>>>>> I'd like us to move to Ruby 2.2 via JRuby 9000 for
HBase 2.0.
> This
> > >>>>>>>
> > >>>>>> will
> > >>>>>
> > >>>>>> cause operator pain to folks with advanced scripts based
on Ruby
> > >>>>>>>
> > >>>>>> 1.8,
> > >>>
> > >>>> but
> > >>>>>
> > >>>>>> it should allow us to update versions to avoid e.g. perf,
> > >>>>>>>
> > >>>>>> correctness,
> > >>>>
> > >>>>> and
> > >>>>>>
> > >>>>>>> security issues much more easily over the lifetime
of 2.0.
> > >>>>>>>
> > >>>>>>> What do folks think? Would JRuby 9000 need to hit a
GA release
> > >>>>>>>
> > >>>>>> prior
> > >>>
> > >>>> to
> > >>>>
> > >>>>> HBase 2.0 getting released for us to adopt it? Or would it
only
> > >>>>>>>
> > >>>>>> need
> > >>>
> > >>>> enough
> > >>>>>>
> > >>>>>>> of Ruby 2.2 to run our current shell?
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> [1]: https://issues.apache.org/jira/browse/HBASE-13338
> > >>>>>>> [2]:
> > >>>>>>>
> > >>>>>> http://www.slideshare.net/CharlesNutter/over-9000-jruby-in-2015
> > >>>
> > >>>> --
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message