crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabriel Reid (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CRUNCH-283) Add additional job dependency info to the job's dotfile
Date Thu, 17 Oct 2013 07:12:42 GMT

     [ https://issues.apache.org/jira/browse/CRUNCH-283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gabriel Reid updated CRUNCH-283:
--------------------------------

    Attachment: CRUNCH-283.patch

+1, that always bugged me that the side-inputs (i.e. MapsideJoin) wasn't shown properly.

Here's a very slightly changed patch that uses a dotted line for data flow that comes via
ParallelDoOptions to help distinguish it from the "normal" MR data flow -- what do you think?

> Add additional job dependency info to the job's dotfile
> -------------------------------------------------------
>
>                 Key: CRUNCH-283
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-283
>             Project: Crunch
>          Issue Type: Improvement
>            Reporter: Josh Wills
>         Attachments: CRUNCH-283.patch, CRUNCH-283.patch
>
>
> Came up with a couple of improvements to the dotfile to help with debugging:
> 1) Add the target dependencies that are implied by ParallelDoOptions to the directed
graphs (target -> PCollectionImpl)
> 2) Add a label to each of the clustered subgraphs that includes the Crunch JobID, to
make it easier to map from running jobs to the dotfile for diagnosis.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message