crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <josh.wi...@gmail.com>
Subject Re: About status web page
Date Tue, 26 Feb 2013 16:02:15 GMT
Hey Chao,

I am woefully inept when it comes to data visualization technologies. What
do the guys from Spotify do to visualize their workflows?

https://github.com/spotify/luigi

J


On Mon, Feb 25, 2013 at 10:13 PM, Chao Shi <stepinto@live.com> wrote:

> Hi Crunch Devs,
>
> I'm interested in adding a web status page to crunch. I'm working on a
> prototype first, which simply runs a jetty server and renders the dot file
> produced by DotFileWriter at browser. The dot rendering work is done by
> viz.js <https://github.com/mdaines/viz.js>. It can successfully render the
> plan into SVG.
>
> I think there are 2 issues I hit with viz.js:
>
> 1. The license of viz.js is unclear. It is compiled from GraphViz source
> code with emscripten. GraphViz is Eclipse Public License 1.0.
>
> 2. viz.js is big and slow. It is a 1.4MB compressed JS. It takes 1 or 2
> seconds on my laptop to render my pipeline (30+ MRs). I think it good to
> have the graph refresh frequently and show the running status of the
> pipeline (i.e. whether MRs are done or not). Thus the rendering time would
> be too slow.
>
> Another approach is to call graphviz command at server side, if viz.js is
> not possible. I can't find any pure Java implementation of graphviz.
>
> Looking forward to your advices.
>
> Thanks,
> Chao
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message