hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: Ruby shell versions for HBase 2.0
Date Wed, 13 May 2015 20:38:25 GMT
> That's only if you assume HBase users actually understand Ruby though, no?

Yes, and this is a fair point.

The language REPL in which we're embedding the shell DSL doesn't have to be
Ruby. JavaScript would work, maybe via Nashorn. Python would work, via
Jython. Or Scala's REPL, even.


On Wed, May 13, 2015 at 1:10 PM, Josh Elser <josh.elser@gmail.com> wrote:

> Andrew Purtell wrote:
>
>> This is because the Accumulo shell is a custom built shell? If so, we had
>> one of those once and replaced it with the IRB based one. We didn't settle
>> on JRuby right away but, at the time, the consensus was we didn't want to
>> be in the business of maintaining yet another REPL when specialist open
>> source communities have already done that. Why has this changed? Has it?
>>
>
> Accumulo shell is just a Java program that uses JLine. It has no control
> flow structures, variables, or anything like that.
>
>  Why is the Accumulo shell easier to script with? Does it have control flow
>> constructs? Variable assignment? Or is it easier somehow because it does
>> not have those things? Even the system shell has those. Hmm... Are we
>> saying that any control flow or variables/substitution should be done by
>> calling our shell from a bash or sh script? Wouldn't that be super slow
>> given Java's startup time for every invocation of our shell? A bit like
>> trying to script the AWS command line utilities, which is excruciating.
>>
>
> This is what I was trying to get at: the lack of typical language
> constructs is both beneficial and irritating. I think it depends on what
> you really want to do. Does that mean trying to maintain two shells is
> worth it? I'm not sure, but I'm also not sure where the happy medium is.
>
> If you exec the shell to just run one command at a time, yes. It is slow.
> You can redirect input or provide a file of commands to run at one time
> which don't suffer from the JVM startup problem.
>
>  I can see why a custom shell built with Java would easier to test where
>> the
>> rest of the project is using Surefire/JUnit. That's a developer
>> convenience
>> concern though, not a user convenience one.
>>
>
> That's only if you assume HBase users actually understand Ruby though, no?
> I would think that something which acts like your normal unix shell would
> be familiar to users w/o the need to understand some other language.
>
>
>
>> On Wed, May 13, 2015 at 10:41 AM, Sean Busbey<busbey@cloudera.com>
>> wrote:
>>
>>  Pros:
>>>
>>> * It's easier to test and maintain
>>> * it's easier to script with
>>> * its interactive mode "feels" mode focused on the task of interacting
>>> with
>>> a cluster to me (maybe this is just acting like a psql or mysql shell)
>>>
>>> Cons
>>>
>>> * adding custom commands requires knowing java
>>>
>>> --
>>> Sean
>>> On May 13, 2015 12:31 PM, "Andrew Purtell"<apurtell@apache.org>  wrote:
>>>
>>>  Why is the Accumulo shell superior?
>>>>
>>>> Is it scriptable?
>>>>
>>>>
>>>> On Wed, May 13, 2015 at 10:28 AM, Sean Busbey<busbey@cloudera.com>
>>>>
>>> wrote:
>>>
>>>> I would love to rip out the JRuby shell entirely and make something
>>>>>
>>>> closer
>>>>
>>>>> to the Accumulo shell, but I expect that will
>>>>>
>>>>> 1) be way more work
>>>>>
>>>>> 2) be even less compatible for those that rely on customizations.
>>>>>
>>>>> I figured given time we could get a preview "user shell" (rather than
>>>>> "power shell" via irb) together in 2.0 and aim for default in 3.0.
>>>>>
>>>>> --
>>>>> Sean
>>>>> On May 13, 2015 12:19 PM, "Stack"<stack@duboce.net>  wrote:
>>>>>
>>>>>  Nice writeup Sean.
>>>>>>
>>>>>> Yeah, +1 to new jruby in hbase 2.0. We'd need to be careful license
>>>>>>
>>>>> is
>>>
>>>> still amenable and hopefully jruby 9k will be slimmer than jruby
>>>>>>
>>>>> 1.7+.
>>>
>>>> But if we are going to do a significant shell refactor for hbase 2.0,
>>>>>> should we consider doing something more radical; e.g. a new shell?
If
>>>>>> interest, could start up a new thread so don't distract from this
>>>>>>
>>>>> one.
>>>
>>>> St.Ack
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, May 13, 2015 at 9:22 AM, Sean Busbey<busbey@cloudera.com>
>>>>>>
>>>>> wrote:
>>>>>
>>>>>> Hi folks!
>>>>>>>
>>>>>>> If you weren't aware, our current shell relies on Ruby,
>>>>>>>
>>>>>> specifically
>>>
>>>> the
>>>>>
>>>>>> REPL program IRB from JRuby. When we launched 1.0 we were on JRuby
>>>>>>>
>>>>>> 1.6
>>>>
>>>>> with
>>>>>>
>>>>>>> defaults, which means we're stuck on Ruby 1.8.
>>>>>>>
>>>>>>> For those that don't already know, Ruby 1.8 is super old and
has
>>>>>>>
>>>>>> been
>>>
>>>> walking off into the sunset for a few years now. Most (but not
>>>>>>>
>>>>>> all!)
>>>
>>>> formal
>>>>>>
>>>>>>> support systems for running Ruby have EOLed 1.8 and there are
>>>>>>>
>>>>>> numerous
>>>>
>>>>> known issues with it.
>>>>>>>
>>>>>>> Right now there's an open ticket to get us on JRuby 1.7 so that
our
>>>>>>>
>>>>>> shell
>>>>>
>>>>>> can work on PPC systems[1]. That version of JRuby defaults to Ruby
>>>>>>>
>>>>>> 1.9
>>>>
>>>>> but
>>>>>>
>>>>>>> can be run in Ruby 1.8 mode. There are some implementation details
>>>>>>> outstanding, but I'm hoping that ticket can work out such that
it
>>>>>>>
>>>>>> can
>>>
>>>> land
>>>>>>
>>>>>>> in branch-1.
>>>>>>>
>>>>>>> For HBase 2.0, I'd like us to plan for a little farther out in
the
>>>>>>>
>>>>>> future
>>>>>
>>>>>> than just updating to Ruby 1.9 (though that would be a fine
>>>>>>>
>>>>>> incremental
>>>>
>>>>> step with some non-trivial work attached). The "current" version of
>>>>>>>
>>>>>> Ruby
>>>>>
>>>>>> is
>>>>>>
>>>>>>> 2.2. Much like the move from 1.8 ->  1.9 it is not backwards
>>>>>>>
>>>>>> compatible.
>>>>
>>>>> JRuby's next major maintenance release line is "version 9000"[2]
>>>>>>>
>>>>>> and
>>>
>>>> it
>>>>
>>>>> will start out *only* supporting Ruby 2.2. Right now JRuby 9000 is
>>>>>>>
>>>>>> in
>>>
>>>> its
>>>>>
>>>>>> second "preview" release. They still have a few feature complete
>>>>>>>
>>>>>> items
>>>>
>>>>> to
>>>>>
>>>>>> address before they hit their first GA release.
>>>>>>>
>>>>>>> I'd like us to move to Ruby 2.2 via JRuby 9000 for HBase 2.0.
 This
>>>>>>>
>>>>>> will
>>>>>
>>>>>> cause operator pain to folks with advanced scripts based on Ruby
>>>>>>>
>>>>>> 1.8,
>>>
>>>> but
>>>>>
>>>>>> it should allow us to update versions to avoid e.g. perf,
>>>>>>>
>>>>>> correctness,
>>>>
>>>>> and
>>>>>>
>>>>>>> security issues much more easily over the lifetime of 2.0.
>>>>>>>
>>>>>>> What do folks think? Would JRuby 9000 need to hit a GA release
>>>>>>>
>>>>>> prior
>>>
>>>> to
>>>>
>>>>> HBase 2.0 getting released for us to adopt it? Or would it only
>>>>>>>
>>>>>> need
>>>
>>>> enough
>>>>>>
>>>>>>> of Ruby 2.2 to run our current shell?
>>>>>>>
>>>>>>>
>>>>>>> [1]: https://issues.apache.org/jira/browse/HBASE-13338
>>>>>>> [2]:
>>>>>>>
>>>>>> http://www.slideshare.net/CharlesNutter/over-9000-jruby-in-2015
>>>
>>>> --
>>>>>>> Sean
>>>>>>>
>>>>>>>
>>>>
>>>> --
>>>> Best regards,
>>>>
>>>>     - Andy
>>>>
>>>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>>>> (via Tom White)
>>>>
>>>>
>>
>>
>>


-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message