incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron McCurry <>
Subject Re: Blur Console communications
Date Wed, 29 May 2013 01:08:01 GMT
I have been giving this some thought.  I think that the overall goals of
the console and agent should be to provide an excellent UI experience
(which I think that it does that already) while minimizing initial setup
time and effort.  I have added some thoughts on below.

On Wed, May 22, 2013 at 10:09 PM, Chris Rohr <> wrote:

> Hi all,
> I wanted to get your thoughts on how the console is currently architected
> and how we might make it a little more drop in friendly.  Currently the
> console is made up of two parts, the GUI (a Ruby on Rails app) and the
> Agent (a java app).

I think that we should add the agent as a project under the normal build

> The agent is responsible for collecting information on the zookeeper
> instances, controllers, shards, tables and queries.  It also has a bit of
> code to get some metrics from HDFS.  The agent basically sits on a timer
> asking zookeeper (java api), blur (thrift) and HDFS (java api) for its
> current information and writes it to a database (currently set up for mysql
> but I don't believe the sql is mysql specific).

I think that we should be as db agnostic as possible, and perhaps follow
Hive's initial setup by using derby db.

> The gui is responsible for displaying the information from the database
> into status pages and metrics pages.  It does have a thrift interface so
> that it can run searches and talk to HDFS.  Again I don't believe that
> there is mysql specific sql going on here, but I can check.  The Rails
> application does not talk to the agent directly, all information sharing is
> done through the database currently.

One inital thought here was how hard would it be to use jruby instead of a
standard ruby install?  Because if we could package it as up as a nice war
(I think that you can do this with jruby + some libs) then it would be
easier for people to drop it into their existing java environments.  We
could also run it as an embedding web app inside of jetty.

> As for "installing" the console it requires unpacking the distribution and
> going through a series of steps to configure the database for both sides
> and to configure the agent for how to talk to blur.  Other than the admin
> creating symlinks to the configuration files, this currently has to be done
> for each update of the code.  We have also had the issue of not being able
> to package mysql with the application, so this becomes the responsibility
> of the person setting up the install to go get it and configure it.

I think as simple as possible is the way to go here.  Then give people
instructions on how to modify the setup to scale or use another DB, etc.


> I have always wanted this app to have very simple setup or maybe even a
> full install process, but I just didn't get it fully there and wanted to
> get the community's thoughts on how to best do this for the future.
> Thanks,
> Chris

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message