beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xu Mingmin (JIRA)" <>
Subject [jira] [Created] (BEAM-1705) add method in Pipeline to visualize the DAG
Date Mon, 13 Mar 2017 20:27:41 GMT
Xu Mingmin created BEAM-1705:

             Summary: add method in Pipeline to visualize the DAG
                 Key: BEAM-1705
             Project: Beam
          Issue Type: New Feature
          Components: sdk-java-core
            Reporter: Xu Mingmin
            Assignee: Xu Mingmin

add a method to print out Pipeline with a easy-to-read format.
Take WindowedWordCount.Java for example, the output could looks like:
1. TextIO.Read/Read.out [PCollection]}]
  2. ParDo(AddTimestamp).out [PCollection]}]
    3. Window.Into()/Window.Assign.out [PCollection]}]	
      4. WordCount.CountWords/ParDo(ExtractWords).out [PCollection]}]
        5. WordCount.CountWords/Count.PerElement/Init/Map.out [PCollection]}]
          6. WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey.out [PCollection]}]
            7. WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues/ParDo(Anonymous).out
              8. ParDo(Anonymous).out [PCollection]}]
                9. GroupByKey.out [PCollection]}]
                  10. ParDo(WriteWindowedFiles).out [PCollection]}]

Add the serial numbers here so it's readable for a complex pipeline with multiple-in and multiple-out.

This message was sent by Atlassian JIRA

View raw message