incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Rohr <rohr.ch...@gmail.com>
Subject Re: Blur Console v2 Design Proposal
Date Mon, 30 Sep 2013 16:39:54 GMT
Michael,

Thank you for your question.  Hopefully this answers the bill:

Of the features that I listed not being available the following were done
by the Java application (back-end) piece in v1:

- Zookeeper ensemble status
    This feature utilized the Java zookeeper libraries to connect and check
the status.  I have not found a Javascript Library that can do this
(outside of using NodeJS).  There are only 2 ways we can do this going
forward if this feature is indeed needed, 1. Add the zookeeper status to
the Blur API or 2. Add on demand checking in the Jetty server (not sure
what the load would be due to this)

- Ability to monitor more than one Blur instance in the same tool
    This feature was possible because the Java application had its own
configuration and allowed many instances to be setup all at once.  Since
this will be started (hopefully) with the start-all.sh and it will be using
the blur-site.config for its configuration, it will not make sense to have
this version contain many instances

- HDFS information
    The Java application did all polling HDFS node and capacity status as
well as metric gathering.  The Rails application did have a thrift
interface that allowed file browsing, but this I believe can all be used in
other tools.

- Keeping track of more queries than the Blur cache of 2 minutes
    The Java application would keep 30 minutes of queries so that we could
go back and see if something bad had happened.  If we are using Blur
exclusively and not storing in a DB now, than we can't maintain the queries
for a longer period of time, with the exception of allowing the browser to
keep track until page reload.  So basically you would get 2 minutes on
reload and then it would add in new ones as you stayed on the page.

Hope this helps,
Chris


On Mon, Sep 30, 2013 at 10:34 AM, Michael Blumberg <
michael.blumberg@bti360.com> wrote:

> Hi Chris,
>
> You mentioned that certain features would not be in v2 for a variety of
> reasons one of which was a lack of a server polling component. Which
> feature(s) in particular is/are affected by this and is the lack of support
> on the server or client side?
>
> -Michael
>
>
> On Sun, Sep 29, 2013 at 8:33 PM, Chris Rohr <rohr.chris@gmail.com> wrote:
>
> > All,
> >
> > As I have begun to start the next version of the console I wanted to give
> > everyone my proposal for what should be in it and get any feedback that
> you
> > may have.  I have created a new branch (new-blur-console) and will be
> > pushing it up as soon as I can.  Please let me know if you have any
> > questions/opinions/suggestions about this proposal.  If I don't hear from
> > anyone, I will just assume it looks good and I will move forward.
> >
> > Thanks in advance,
> > Chris
> >
> > *Blur Console v2 Design Proposal*
> >
> > *Purpose*
> >
> > This writeup will help to determine what direction the console portion of
> > Blur should go.  There has always been a need for a push button, easy
> setup
> > solution so that dev-ops could get up and running with the console
> faster.
> > *NOTE: *This is purely a suggestion and all opinions and recommendations
> > are welcome.
> >
> >
> > *Server*
> >
> > Java program that runs an embedded Jetty server.  The purpose of the
> Jetty
> > server would be to serve up any of the html pages and resources needed
> for
> > the page as well as potentially any JSON based calls.
> >
> >
> > *Interaction with Blur*
> >
> > The console would utilize the Javascript generated Thrift API of Blur to
> > communicate and retrieve information from Blur.
> >
> >
> > *Main Site Framework*
> >
> > We are looking to use AngularJS to provide an MVC framework in the
> browser
> > so that we will not have to rely on a front-end component and a back-end
> > component to this system.  This should allow this tool to be much more
> > lightweight.
> >
> >
> > *Features*
> >
> > The main page of the site would be broken into tabs for major pieces of
> > functionality.  The list below will outline what each tab would provide:
> >
> >
> >
> >    - Dashboard
> >       - Controller Node Status (chart)
> >       - Shard Node Status by Cluster (chart)
> >       - Query Load by Cluster (chart)
> >       - Query Performance by Cluster (chart)
> >       - Number of Tables (chart)
> >       - Long Running Query Warnings
> >    - Tables
> >       - Listing of Tables by Cluster and by Status
> >       - For each table
> >          - Shard Layout
> >          - Schema
> >          - Term Lookup
> >       - Actions
> >          - Disable Table
> >          - Enable Table
> >          - Delete Table (with or without deleting underlying index in
> HDFS)
> >        - Queries
> >       - List of Running Queries (last 2 minutes)
> >       - For each Query
> >          - Display base info about the query (query string, user, status)
> >          - Extra info about query (selector information)
> >        - Search (live search of Blur)
> >
> >
> > In addition we can have some base user preferences that will be stored in
> > local storage or cookies in the browser.
> >
> >
> > *Things missing from v1*
> >
> > The following items will not be in v2 either because they are not in the
> > Blur API yet or because of the lack of a server polling component.  These
> > are all things that were nice to have but aren’t as critical.  If any of
> > these become critical we can figure out how to provide this information
> to
> > the console.
> >
> >
> >
> >    - Zookeeper ensemble status
> >    - Ability to monitor more than one Blur instance in the same tool
> (each
> >    instance will have its own Console now)
> >    - All HDFS pieces (deferring to other purpose built tools like Hue)
> >    - User auditing of actions like disabling or deleting a table (we
> could
> >    do this with the server component)
> >    - Keeping track of more queries than the Blur cache of 2 minutes (we
> >    could do something that until a page refresh we could keep track of
> > more).
> >
> >
> > *Other Items to consider*
> >
> > One other big item to consider is how to configure the console.  I’m
> > thinking that we could just figure out how to tie into the blur-site
> > config.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message