incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabriel Reid <>
Subject Generating DOT files for Crunch job plans
Date Sat, 27 Oct 2012 09:16:52 GMT

In the few times that I've debugged issues in the planner in Crunch, it always takes me a
bit of time to figure out (again) how things work there. I've been thinking/planning of writing
some more inline docs and doing a bit of refactoring in the code to help myself (and others)
with doing this in the future, but something else that I was thinking of was the generation
of DOT[1] files for pipelines so that it's easier to visualize what's going on.

I'm sure that functionality like this can be useful (at least to me, as I was just using it
in a somewhat ad-hoc way to debug CRUNCH-102), but I'm not sure if this is something we want
to expose easily, or keep pretty hidden to just use for debugging. I believe Pig provides
this same functionality with the "explain" command.

Any thoughts on adding this, particularly around how we could/should expose it in the API?

- Gabriel

View raw message