Mailing-List: contact issues-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
Date: Thu, 23 Feb 2017 21:08:44 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@drill.apache.org
Message-ID: <JIRA.13042470.1486961468000.47645.1487884124833@Atlassian.JIRA>
In-Reply-To: <JIRA.13042470.1486961468000@Atlassian.JIRA>
References: <JIRA.13042470.1486961468000@Atlassian.JIRA> <JIRA.13042470.1486961468914@jira-lw-us.apache.org>
Subject: [jira] [Commented] (DRILL-5260) Refinements to new "Cluster
 Fixture" test framework
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 23 Feb 2017 21:08:57 -0000


    [ https://issues.apache.org/jira/browse/DRILL-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881266#comment-15881266 ] 

ASF GitHub Bot commented on DRILL-5260:
---------------------------------------

Github user sohami commented on a diff in the pull request:

    https://github.com/apache/drill/pull/753#discussion_r102603020
  
    --- Diff: exec/java-exec/src/test/java/org/apache/drill/test/ProfileParser.java ---
    @@ -42,44 +48,207 @@
     public class ProfileParser {
     
       JsonObject profile;
    +  String query;
       List<String> plans;
    +  List<OpDefInfo> operations;
    +  Map<Integer,FragInfo> fragments = new HashMap<>();
    +  private List<OpDefInfo> topoOrder;
     
       public ProfileParser( File file ) throws IOException {
         try (FileReader fileReader = new FileReader(file);
              JsonReader reader = Json.createReader(fileReader)) {
           profile = (JsonObject) reader.read();
         }
    +
    +    parse();
    +  }
    +
    +  private void parse() {
    +    parseQuery();
    +    parsePlans();
    +    buildFrags();
    +    parseFragProfiles();
    +    mapOpProfiles();
    +    aggregateOpers();
    +    buildTree();
    +  }
    +
    +  private void parseQuery() {
    +    query = profile.getString("query");
    +    query = query.replace("//n", "\n");
    +  }
    +
    +  /**
    +   * Parse a text version of the plan as it appears in the JSON
    +   * query profile.
    +   */
    +
    +  private static class PlanParser {
    +
    +    List<String> plans = new ArrayList<>();
    +    List<OpDefInfo> operations = new ArrayList<>();
    +    List<OpDefInfo> sorted = new ArrayList<>();
    +
    +    public void parsePlans(String plan) {
    +      plans = new ArrayList<>( );
    +      String parts[] = plan.split("\n");
    +      for (String part : parts) {
    +        plans.add(part);
    +        OpDefInfo opDef = new OpDefInfo( part );
    +        operations.add(opDef);
    +      }
    +      sortList();
    +    }
    +
    +    public void sortList() {
    +      List<OpDefInfo> raw = new ArrayList<>( );
    +      raw.addAll( operations );
    +      Collections.sort( raw, new Comparator<OpDefInfo>() {
    +        @Override
    +        public int compare(OpDefInfo o1, OpDefInfo o2) {
    +          int result = Integer.compare(o1.majorId, o2.majorId);
    +          if ( result == 0 ) {
    +            result = Integer.compare(o1.stepId, o2.stepId);
    +          }
    +          return result;
    +        }
    +      });
    +      int currentFrag = 0;
    +      int currentStep = 0;
    +      for ( OpDefInfo opDef : raw ) {
    +        if ( currentFrag < opDef.majorId ) {
    +          currentFrag++;
    +          OpDefInfo sender = new OpDefInfo( currentFrag, 0 );
    +          sender.isInferred = true;
    +          sender.name = "Sender";
    +          sorted.add(sender);
    +          currentStep = 1;
    +          opDef.inferredParent = sender;
    +          sender.children.add( opDef );
    +        }
    +        if ( opDef.stepId > currentStep ) {
    +          OpDefInfo unknown = new OpDefInfo( currentFrag, currentStep );
    +          unknown.isInferred = true;
    +          unknown.name = "Unknown";
    +          sorted.add(unknown);
    +          opDef.inferredParent = unknown;
    +          unknown.children.add( opDef );
    +        }
    +        sorted.add( opDef );
    +        currentStep = opDef.stepId + 1;
    +      }
    +    }
    +  }
    +
    +  /**
    +   * Parse the plan portion of the query profile.
    +   */
    +
    +  private void parsePlans() {
    +    PlanParser parser = new PlanParser();
    +    String plan = getPlan( );
    +    parser.parsePlans(plan);
    +    plans = parser.plans;
    +    topoOrder = parser.operations;
    +    operations = parser.sorted;
    +  }
    --- End diff --
    
    wrong assignment ?` topoOrder = parser.sorted` and `operations = parser.operations` ?


> Refinements to new "Cluster Fixture" test framework
> ---------------------------------------------------
>
>                 Key: DRILL-5260
>                 URL: https://issues.apache.org/jira/browse/DRILL-5260
>             Project: Apache Drill
>          Issue Type: Improvement
>    Affects Versions: 1.10
>            Reporter: Paul Rogers
>            Assignee: Paul Rogers
>            Priority: Minor
>             Fix For: 1.10
>
>
> Roll-up of a number of enhancements to the cluster fixture framework.
> * Config option to suppress printing of CSV and other output. (Allows printing for single tests, not printing when running from Maven.)
> * Parsing of query profiles to extract plan and run time information.
> * Fix bug in log fixture when enabling logging for a package.
> * Improved ZK support.
> * Set up the new CTTAS default temporary workspace for tests.
> * Revise TestDrillbitResiliance to use the new framework.
> * Revise TestWindowFrame to to use the new framework.
> * Revise TestMergeJoinWithSchemaChanges to use the new framework.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)