crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chao Shi <stepi...@live.com>
Subject Re: About status web page
Date Wed, 27 Feb 2013 06:56:31 GMT
It seems luigi uses a swig wrapper of graphviz. There is also a swig
wrapper for Java.

I also found Oozie has a nice GWT based
UI<https://issues.apache.org/jira/secure/attachment/12539680/Whole-Screen.png>.
But I think it too heavy. I'm open to other options.

On Wed, Feb 27, 2013 at 12:02 AM, Josh Wills <josh.wills@gmail.com> wrote:

> Hey Chao,
>
> I am woefully inept when it comes to data visualization technologies. What
> do the guys from Spotify do to visualize their workflows?
>
> https://github.com/spotify/luigi
>
> J
>
>
> On Mon, Feb 25, 2013 at 10:13 PM, Chao Shi <stepinto@live.com> wrote:
>
> > Hi Crunch Devs,
> >
> > I'm interested in adding a web status page to crunch. I'm working on a
> > prototype first, which simply runs a jetty server and renders the dot
> file
> > produced by DotFileWriter at browser. The dot rendering work is done by
> > viz.js <https://github.com/mdaines/viz.js>. It can successfully render
> the
> > plan into SVG.
> >
> > I think there are 2 issues I hit with viz.js:
> >
> > 1. The license of viz.js is unclear. It is compiled from GraphViz source
> > code with emscripten. GraphViz is Eclipse Public License 1.0.
> >
> > 2. viz.js is big and slow. It is a 1.4MB compressed JS. It takes 1 or 2
> > seconds on my laptop to render my pipeline (30+ MRs). I think it good to
> > have the graph refresh frequently and show the running status of the
> > pipeline (i.e. whether MRs are done or not). Thus the rendering time
> would
> > be too slow.
> >
> > Another approach is to call graphviz command at server side, if viz.js is
> > not possible. I can't find any pure Java implementation of graphviz.
> >
> > Looking forward to your advices.
> >
> > Thanks,
> > Chao
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message