crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christian Tzolov (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CRUNCH-438) Visualizations of some important internal/intermediate pipeline planning states
Date Thu, 04 Dec 2014 10:09:15 GMT

     [ https://issues.apache.org/jira/browse/CRUNCH-438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Christian Tzolov updated CRUNCH-438:
------------------------------------
    Attachment: CRUNCH-438.5.patch

[~jwills] the attached CRUNCH-438.5.patch keeps the getPlanDotFile() method for backward compatibility
( Internally it delegates to the namedDotFiles map).

Considering that this patch has no functional value i don't like how intrusive this it is.
Particularly the MSCRPlanner#plan() method. To compute the additional diagrams we have to
hook dotfile code inside. I've been thinking of refactoring the method lifecycle exposing
some callback hooks. But this would bring even more complexity and is not justifiable for
a single use case. 
Are the there any other use cases that may need MSCRPlanner#plan() refactoring?

> Visualizations of some important internal/intermediate pipeline planning states
> -------------------------------------------------------------------------------
>
>                 Key: CRUNCH-438
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-438
>             Project: Crunch
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 0.10.0, 0.8.3
>            Reporter: Christian Tzolov
>            Assignee: Christian Tzolov
>         Attachments: CRUNCH-438.2.patch, CRUNCH-438.3.patch, CRUNCH-438.4.patch, CRUNCH-438.5.patch,
CRUNCH-438.patch
>
>
> To improve the understability of the pipeline planning stages it would help to visualize
some intermediate planning states like:
> - PCollection lineage. (visualizing the output-pcollection-targets structure) 
> - MSCRPlanner's planning Graphs before and after the split up of dependent GBK nodes
> - RTNode hierarchy along with the Input and Output configurations as persistent in the
Configuration before the execution of the pipeline. 
> Most of the information can be intercepted in the MSCRPlanner#plan()  method.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message