crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabriel Reid (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-418) Auto-logging Plan
Date Sun, 15 Jun 2014 12:24:01 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031863#comment-14031863
] 

Gabriel Reid commented on CRUNCH-418:
-------------------------------------

Another idea would be to introduce a job plan output directory config setting. This could
be a local or HDFS URI (or I suppose any supported filesystem URI), and if it's populated
then the planner could write the job plan dot file there, with the name of the file being
the job name plus the timestamp. 

I think that this would make it easy to automate writing the job plan files somewhere standardized
in an opt-in manner, would make it easy to deal with the dot files (without copying/pasting),
and wouldn't pollute the log file with this content.

What do you think?

> Auto-logging Plan
> -----------------
>
>                 Key: CRUNCH-418
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-418
>             Project: Crunch
>          Issue Type: Wish
>            Reporter: Allan Shoup
>            Priority: Minor
>
> It would be handy if Crunch was capable of logging out the plan DOT diagram automatically
whenever the pipeline was run.
> By default I would log the plan at a level below INFO - probably at DEBUG.
> Additionally, configuration could be provided to specify the level that the plan should
be logged at (for example if you always wanted it logged at INFO level, or if you wanted to
disable logging the plan altogether).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message