hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jesse Yates <jesse.k.ya...@gmail.com>
Subject Re: updating shell functionality
Date Wed, 23 Nov 2011 21:02:56 GMT
Yeah, the jvm repls are getting pretty sweet. However, that is going to be
a bunch of work to just get back the same functionality we have now. I
don't think there is a really compelling reason to make the switch - we
don't get a ton more functionality and it isn't significantly easier to
make changes add functionality (at least, after initially thinking about
the design).

The unit tests actually are pretty nice as it stands now - they are well
encapsulated and look really clean in the ruby syntax.

There are some issues where it feels like people made a kludge of certain
parts, without realizing what was happening because it was just one more
thing. There has been a lot of added functionality recently making the
previous shell that much more awkward. I would be open to doing a rewrite
if we really get that much more out of it - do you have any compelling
reasons besides it makes the translation a little easier (which isn't that
bad right now)? Yeah its time for a bit of redesign, but lets not throw the
baby out with the bath water.

Another idea that has come up is that often times you want to work in the
context of a table, rather than entering it for every command. Accumulo
does this where you pick the table context to work in. Sometimes its really
simple and nice and other times its a pain when you forget to make the
switch.

I was thinking we could have the option to drop into a table context
(switching into a TabledShell?), but still keep around a lot of the current
functionality. This has implications for the current shell paradigm of
"verb, table, row, etc." (as stack put it), which for some of the current
functionality (eg. coprocessors, attributes, possibly some of the security)
feels a bit forced and awkward.

-Jesse
On Tue, Nov 22, 2011 at 11:15 AM, Elliott Clark <eclark@ngmoco.com> wrote:

> So a little more on the crazy idea side.
>
> Java type repls are starting to get more love and are a little more
> developed than they were when jruby was first used.  Is is time to start
> considering a more java repl (scala, clojure, groovy, et al) ?  All of
> those would have less of a translation of java -> repl command, would be
> more easily testable with maven/junit and a Ci server.  Shouldn't be too
> hard to get either scala or groovy shell's to work hbase env and load the
> right classes.  Then auto complete should do the rest for making the
> transition pretty easy. In addition it would be nice for encoding issues to
> go away.
>
> http://www.scala-lang.org/node/2097
> http://groovy.codehaus.org/Groovy+Shell
>
>
> I agree that moving away from using strings for args seems like the way to
> go.  So any love to the shell would be appreciated.
>
> On Tue, Nov 22, 2011 at 10:48 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > Interesting idea.
> >
> > See the following for reference:
> >
> >
> http://asqyzeron.wordpress.com/2007/01/14/toying-with-the-dynamic-features-of-ruby-adding-methods-at-run-time/
> >
> >
> http://www.vitarara.org/cms/ruby_metaprogamming_declaratively_adding_methods_to_a_class
> >
> > On Tue, Nov 22, 2011 at 10:34 AM, Jesse Yates <jesse.k.yates@gmail.com
> > >wrote:
> >
> > > Hey all,
> > >
> > > I'm trying to take some time to figure out the right way to do support
> to
> > > future 'pluggable' features/extensions to modifying tables. Specially,
> > I'm
> > > thinking about the 'alter' command, but in the future, this model could
> > be
> > > applied to other facets of the shell.
> > >
> > >  In the easiest, and hackiest) way, we could just take in the full
> class
> > > name to call and then lookup the java class, but I like Gary's comment
> > > from HBASE-4605
> > > (Constraints) <https://issues.apache.org/jira/browse/HBASE-4605>:
> > >
> > > "3. provide a extensions dir for the shell:
> > >
> > >   - extensions drop a simple jruby scriptlet in a file in the dir (say
> > >   lib/ruby/hbase/ext)
> > >   - the scriptlet does some simple registration of the available
> > >   methods/commands
> > >   - the shell code loads all files"
> > >
> > > In doing this a 'clean' way, we could also do a lot in making the shell
> > > easier to use. (Stick with me, its gets a little verbose here).
> > >
> > > What I'm proposing is that instead of altering tables like this(for
> > setting
> > > arbitrary keys, but this could be things like MAX_FILESIZE, etc):
> > >
> > >  alter 'table', METHOD=>'table_attr', "key1" => "v1"
> > >
> > > We could actually have the use make explicit method calls (which should
> > > give auto-complete in irb) on an object you get from alter. Then you
> can
> > do
> > > something like this:
> > >
> > >  hbase> alter('table').table_attr "key1" => "v1"
> > >
> > > in the simple case. And we allow a lot of the basic functionality you
> > > already get with Java HTD.
> > >
> > > However, this becomes interesting when we want to add more features.
> For
> > > instance, with constraints, it would look something like:
> > >
> > >  hbase> alter('table').addConstraint('com.my.Constraint')
> > >
> > > What's even cooler is, since we can reopen ruby classes and add
> > > functionality - for instance when we load all the files in the
> > > lib/ruby/hbase/ext - we are actually open up the object returned by
> alter
> > > (maybe something like AlterTable class?) and add methods. This means we
> > > also get auto-complete when doing an alteration on a table.
> > >
> > > This is then a really extensible way to add new functionality and
> cleanup
> > > older. The ext scripts just open up the actual classes they need to
> > modify
> > > to add the method functionality, without all the mucking around with
> > > strings that we are doing now.
> > >
> > > For instance, adding a coprocessor can go from:
> > >
> > >  hbase> alter 't1', METHOD =>
> > >
> > >
> >
> 'table_att''coprocessor'=>'hdfs:///foo.jar|com.foo.FooRegionObserver|1001|arg1=1,arg2=2
> > > to now something like:
> > >
> > >  hbase> alter('t1').addCoprocessor 'com.foo.FooRegionObserver',
> > > 'hdfs:///foo.jar', '1001' {'arg1'=>'1','arg2'=>'2'}
> > >
> > >
> > > Or slightly more verbosely:
> > >
> > >  hbase> alter('t1').addCoprocessor('com.foo.FooRegionObserver',
> > > 'hdfs:///foo.jar', '1001' {'arg1'=>'1','arg2'=>'2'})
> > >
> > >
> > > This approach might also be something we can extend to much of the
> > current
> > > shell functionality while still retaining much of the current code. I
> > think
> > > moving this this kind of approach applies anywhere we would need to do
> > > chaining of invocations. Clearly, it depends on the situation, but
> > > leveraging OOP, rather than switching on string, makes a lot of sense
> (at
> > > least to me :)
> > >
> > > What does everyone think of this approach? Am I missing anything
> > > technically (haven't really played with JRuby before) or historically?
> Or
> > > maybe people have other preferences?
> > >
> > > Thanks!
> > >
> > > -- Jesse
> > > -------------------
> > > Jesse Yates
> > > 240-888-2200
> > > @jesse_yates
> > >
> >
>



-- 
-------------------
Jesse Yates
240-888-2200
@jesse_yates

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message