incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Rohr <rohr.ch...@gmail.com>
Subject Re: Blur Console v2 Design Proposal
Date Mon, 30 Sep 2013 16:29:02 GMT
Vikrant,

Thank you for your thoughts.  I will try to answer your questions here:

   >>> What is this serving up? Is this a web service which would be
consumed by the AngularJS front end. Since you mentioned that it would be a
Jetty server(HTTP server + servlet container) a little clarification would
be great.

In my initial thoughts, all the Jetty server would be used for is to
provide an HTTP server and serve up the static html pages.  All of the
dynamic nature of the console would be done through Javascript using
AngluarJS and the JS Blur API.  However, in the future if there is a need
for functionality that can't be done from the Browser directly (i.e. User
logins, zookeeper status, etc) then we could setup some basic JSON web
service calls that would be served up by Jetty.

   >>> Are we going to pull the information from within the
Browser(something like AJAX) or the Jetty server would do a query to the
Thrift service and pass it back to the Browser.

We will be trying to use the JS Blur API to access Blur directly, however,
we may run into Cross site scripting issues depending on how thrift works
in JS and if that is the case then we would need to build proxies through
the Jetty server.

   >>> Since we will be using AngularJS and a Jetty server please clarify
about not relying on a front-end and back-end component.  A typical
scenario involving all the above 3 components would be helpful. This would
also help us in figuring out the options we have while selecting the
technology.

Currently the console contains 2 pieces of software, a Ruby on Rails
application (front-end) and a Java application (back-end).  The Java
application handles all of the information collection in Blur (i.e. polling
for statues, and tables, and queries) and stores everything in a database.
The Ruby on Rails application presents all of the data in GUI form.  The
Ruby on Rails application does have some direct access to Blur, but does
not have direct communication to the Java application.  This proposal does
away with the 2 separate pieces of code to 1. help simplify the logic and
2. help simply setup and startup of the console.

I hope this answers your questions, please let me know if you think of any
more.

Chris


On Mon, Sep 30, 2013 at 2:38 AM, Vikrant Navalgund <
vikrant.navalgund@gmail.com> wrote:

> Hello Chris,
> Thanks for the writeup.
> As a developer interested in contributing to the new BLUR console I have a
> few questions.
> Please ignore my lack of understanding of the architecture.
> Treat me as a newbie to BLUR. My questions are inline.
>
>
> On Mon, Sep 30, 2013 at 10:33 AM, Chris Rohr <rohr.chris@gmail.com> wrote:
>
> > All,
> >
> > As I have begun to start the next version of the console I wanted to give
> > everyone my proposal for what should be in it and get any feedback that
> you
> > may have.  I have created a new branch (new-blur-console) and will be
> > pushing it up as soon as I can.  Please let me know if you have any
> > questions/opinions/suggestions about this proposal.  If I don't hear from
> > anyone, I will just assume it looks good and I will move forward.
> >
> > Thanks in advance,
> > Chris
> >
> > *Blur Console v2 Design Proposal*
> >
> > *Purpose*
> >
> > This writeup will help to determine what direction the console portion of
> > Blur should go.  There has always been a need for a push button, easy
> setup
> > solution so that dev-ops could get up and running with the console
> faster.
> > *NOTE: *This is purely a suggestion and all opinions and recommendations
> > are welcome.
> >
> >
> > *Server*
> >
> > Java program that runs an embedded Jetty server.  The purpose of the
> Jetty
> > server would be to serve up any of the html pages and resources needed
> for
> > the page as well as potentially any JSON based calls.
> >
>
>    >>> What is this serving up? Is this a web service which would be
> consumed by the
>    >>> AngularJS front end. Since you mentioned that it would be a Jetty
> server(HTTP server +
>    >>> servlet container) a little clarification would be great.
>
> >
> >
> > *Interaction with Blur*
> >
> > The console would utilize the Javascript generated Thrift API of Blur to
> > communicate and retrieve information from Blur.
> >
>
>    >>> Are we going to pull the information from within the
> Browser(something like AJAX) or the Jetty server
>    >>> would do a query to the Thrift service and pass it back to the
> Browser.
>
> >
> >
> > *Main Site Framework*
> >
> > We are looking to use AngularJS to provide an MVC framework in the
> browser
> > so that we will not have to rely on a front-end component and a back-end
> > component to this system.  This should allow this tool to be much more
> > lightweight.
> >
>
>    >>> Since we will be using AngularJS and a Jetty server please clarify
> about not
>    >>> relying on a front-end and back-end component.
>
> >>> A typical scenario involving all the above 3 components would be
> > helpful.
> >
>    >>> This would also help us in figuring out the options we have while
> selecting
>    >>> the technology.
>
> >
> > *Features*
> >
> > The main page of the site would be broken into tabs for major pieces of
> > functionality.  The list below will outline what each tab would provide:
> >
> >
> >
> >    - Dashboard
> >       - Controller Node Status (chart)
> >       - Shard Node Status by Cluster (chart)
> >       - Query Load by Cluster (chart)
> >       - Query Performance by Cluster (chart)
> >       - Number of Tables (chart)
> >       - Long Running Query Warnings
> >    - Tables
> >       - Listing of Tables by Cluster and by Status
> >       - For each table
> >          - Shard Layout
> >          - Schema
> >          - Term Lookup
> >       - Actions
> >          - Disable Table
> >          - Enable Table
> >          - Delete Table (with or without deleting underlying index in
> HDFS)
> >        - Queries
> >       - List of Running Queries (last 2 minutes)
> >       - For each Query
> >          - Display base info about the query (query string, user, status)
> >          - Extra info about query (selector information)
> >        - Search (live search of Blur)
> >
> >
> > In addition we can have some base user preferences that will be stored in
> > local storage or cookies in the browser.
> >
>
>
> > >>> The feature list looks pretty good as of now.
> >
> > *Things missing from v1*
> >
> > The following items will not be in v2 either because they are not in the
> > Blur API yet or because of the lack of a server polling component.  These
> > are all things that were nice to have but aren’t as critical.  If any of
> > these become critical we can figure out how to provide this information
> to
> > the console.
> >
> >
> >
> >    - Zookeeper ensemble status
> >    - Ability to monitor more than one Blur instance in the same tool
> (each
> >    instance will have its own Console now)
> >    - All HDFS pieces (deferring to other purpose built tools like Hue)
> >    - User auditing of actions like disabling or deleting a table (we
> could
> >    do this with the server component)
> >    - Keeping track of more queries than the Blur cache of 2 minutes (we
> >    could do something that until a page refresh we could keep track of
> > more).
> >
> >
> > *Other Items to consider*
> >
> > One other big item to consider is how to configure the console.  I’m
> > thinking that we could just figure out how to tie into the blur-site
> > config.
> >
>
>
>
> --
> *Regards*,
> *Vikrant Navalgund*
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message