accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Launching Accumulo Servers Programatically
Date Mon, 11 May 2015 21:21:59 GMT
Very similar to the problems I ran into on ACCUMULO-2739 (trying to run 
MAC in-process).

One thing I really like from HBase is how their RPC servers are 
implemented. They build the server using protocol-buffers and then they 
have code which receives RPCs and dispatches them to that server "stub". 
The cool part is that internally, the server just looks like a normal 
object and can make "RPCs" locally.

If you look at the public static void main(String[]) method on each 
"server" process, we have the same implementation in nearly all of them. 
It would be nice to consolidate this all (and would likely scratch your 
itch too).

Maybe collecting a list as sub-tasks on a JIRA issue would be good if 
you want to work on this. Otherwise, brainstorming more here is also 
swell :)

Jim Klucar wrote:
> Devs,
>
> I've been working on the mesos-accumulo framework. For those not familiar
> with Mesos, it is a cluster resource manager that allows you to launch
> tasks across the cluster. To do that, you write a mesos framework.
>
> We have proven this is possible with accumulo by writing a quick Scala
> framework implementation that uses the accumulo script to launch servers
> from a command line. This worked fine but we would like more insight into
> the servers once they're launched, outside of does the pid still exist on
> the machine. What I would like to do is something like:
>
> org.apache.accumulo.server.Master master = new o.a.a.server.Master();
>
> from inside the mesos executor code. Then setup some config, or pass it in
> the constructor and call master.run() or whatever. The point is at the end
> of some procedure the server is running, and I can call methods like
> master.shutdown() or master.ping() or whatever else is exposed.  A nice
> standard server interface across server types would be great.
>
> I realize this may be quite a refactoring of all the server classes so I
> thought I'd gather some opinions on the idea here instead of just writing
> JIRA tickets.
>
> Jim
>

Mime
View raw message