crunch-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject svn commit: r1650928 - /crunch/site/trunk/content/user-guide.mdtext
Date Sun, 11 Jan 2015 17:27:41 GMT
Author: tzolov
Date: Sun Jan 11 17:27:41 2015
New Revision: 1650928

edit the pipeline visualization section


Modified: crunch/site/trunk/content/user-guide.mdtext
--- crunch/site/trunk/content/user-guide.mdtext (original)
+++ crunch/site/trunk/content/user-guide.mdtext Sun Jan 11 17:27:41 2015
@@ -1605,16 +1605,20 @@ a distributed data set using either the
 ### Pipeline execution plan visualizations
-Crunch provides tools to visualize the pipeline execution plans. The [PipelineExecution](apidocs/0.10.0/org/apache/crunch/PipelineExecution.html)
 `String getPlanDotFile()` method returns an execution plan visualization in DOT format. When
the output folder property is set, Crunch produces a DOT file after each pipeline run. 
+Crunch provides tools to visualize the pipeline execution plan. The [PipelineExecution](apidocs/0.10.0/org/apache/crunch/PipelineExecution.html)
 `String getPlanDotFile()` method returns a DOT format visualization of the exaction plan.
Furthermore if the output folder is set then Crunch will save the dotfile diagram on each
pipeline execution: 
-Additional aspects of the execution plans are provided when the DOT file debug mode is enabled.
Then Crunch provides 4 additional DOT diagrams visualizing different internal stages of the
execution plan. Such plans include PCollection lineage, Base graph plan, Split graph plans,
Run-time nodes. 
-Note: To enable the debug mode you should set an out put folder first. The following snapped
switches the DOT file debug mode.  As a result 5 DOT diagrams are generated in the output
folder after each Pipeline execution:
-	    Configuration conf = ...
-	    String dotfileDir = ...
+	    Configuration conf =...;     
+	    String dotfileDir =...;
+	    // Set DOT files out put folder path
 	    DotfileUtills.setPipelineDotfileOutputDir(conf, dotfileDir);
-	    DotfileUtills.enableDebugDotfiles(conf);
+Additional details of the Crunch execution plan can be exposed by enabling the dotfile debug
mode like this:
+	    // Requires the output folder to be set.
+	    DotfileUtills.enableDebugDotfiles(conf);
+This will produce (and save) 4 additional diagrams that visualize the internal stages of
the pipeline execution plan. Such diagrams are the PCollection pineage, the pipeline base
and split graphs and the run-time node (RTNode) representation. 
+(Note: The debug mode requires the output folder to be set. )
\ No newline at end of file

View raw message