crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allan Shoup (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-418) Auto-logging Plan
Date Thu, 12 Jun 2014 20:05:06 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14029719#comment-14029719
] 

Allan Shoup commented on CRUNCH-418:
------------------------------------

I was mainly looking for something that makes it easier to get the plan. The best way I know
of to get the plan currently is to run the pipeline asynchronously, get the plan, do something
with the plan, block on the asynchronous call to wait for it to finish. It seems like there
should be a simpler method. Something that's there automatically without having to do anything
would be great. I don't disagree that using the log file for this is awkward. The reason I
suggested the log file was that there is already tooling that collects and exposes the log
files. The disadvantage of writing to a separate plan file is if you're not sitting at the
command line (for example when running with Oozie), collection of non-standard files is more
difficult. Perhaps if the framework provided this file automatically or at least established
the convention, tooling would be more apt to take advantage of the feature.

> Auto-logging Plan
> -----------------
>
>                 Key: CRUNCH-418
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-418
>             Project: Crunch
>          Issue Type: Wish
>            Reporter: Allan Shoup
>            Priority: Minor
>
> It would be handy if Crunch was capable of logging out the plan DOT diagram automatically
whenever the pipeline was run.
> By default I would log the plan at a level below INFO - probably at DEBUG.
> Additionally, configuration could be provided to specify the level that the plan should
be logged at (for example if you always wanted it logged at INFO level, or if you wanted to
disable logging the plan altogether).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message