drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From paul-rogers <...@git.apache.org>
Subject [GitHub] drill pull request #753: DRILL-5260: Extend "Cluster Fixture" test framework
Date Sat, 25 Feb 2017 00:21:30 GMT
Github user paul-rogers commented on a diff in the pull request:

    https://github.com/apache/drill/pull/753#discussion_r103060090
  
    --- Diff: exec/java-exec/src/test/java/org/apache/drill/test/ProfileParser.java ---
    @@ -138,9 +414,208 @@ public long getMetric(int id) {
         }
       }
     
    -  public Map<Integer,OpInfo> getOpInfo( ) {
    +  /**
    +   * Information about an operator definition: the plan-time information
    +   * that appears in the plan portion of the profile. Also holds the
    +   * "actuals" from the minor fragment portion of the profile.
    +   * Allows integrating the "planned" vs. "actual" performance of the
    +   * query.
    +   */
    +
    +  public static class OpDefInfo {
    +    public String opName;
    +    public boolean isInferred;
    +    public int majorId;
    +    public int stepId;
    +    public String args;
    +    public List<FieldDef> columns;
    +    public int globalLevel;
    +    public int localLevel;
    +    public int id;
    +    public int branchId;
    +    public boolean isBranchRoot;
    +    public double estMemoryCost;
    +    public double estNetCost;
    +    public double estIOCost;
    +    public double estCpuCost;
    +    public double estRowCost;
    +    public double estRows;
    +    public String name;
    +    public long actualMemory;
    +    public int actualBatches;
    +    public long actualRows;
    +    public OpDefInfo inferredParent;
    +    public List<OperatorProfile> opExecs = new ArrayList<>( );
    +    public List<OpDefInfo> children = new ArrayList<>( );
    +
    +    // 00-00    Screen : rowType = RecordType(VARCHAR(10) Year, VARCHAR(65536) Month,
VARCHAR(100) Devices, VARCHAR(100) Tier, VARCHAR(100) LOB, CHAR(10) Gateway, BIGINT Day, BIGINT
Hour, INTEGER Week, VARCHAR(100) Week_end_date, BIGINT Usage_Cnt): \
    +    // rowcount = 100.0, cumulative cost = {7.42124276972414E9 rows, 7.663067406383167E10
cpu, 0.0 io, 2.24645048816E10 network, 2.692766612982188E8 memory}, id = 129302
    +    //
    +    // 00-01      Project(Year=[$0], Month=[$1], Devices=[$2], Tier=[$3], LOB=[$4], Gateway=[$5],
Day=[$6], Hour=[$7], Week=[$8], Week_end_date=[$9], Usage_Cnt=[$10]) :
    +    // rowType = RecordType(VARCHAR(10) Year, VARCHAR(65536) Month, VARCHAR(100) Devices,
VARCHAR(100) Tier, VARCHAR(100) LOB, CHAR(10) Gateway, BIGINT Day, BIGINT Hour, INTEGER Week,
VARCHAR(100) Week_end_date, BIGINT Usage_Cnt): rowcount = 100.0, cumulative cost = {7.42124275972414E9
rows, 7.663067405383167E10 cpu, 0.0 io, 2.24645048816E10 network, 2.692766612982188E8 memory},
id = 129301
    +
    +    public OpDefInfo(String plan) {
    +      Pattern p = Pattern.compile( "^(\\d+)-(\\d+)(\\s+)(\\w+)(?:\\((.*)\\))?\\s*:\\s*(.*)$"
);
    +      Matcher m = p.matcher(plan);
    +      if (!m.matches()) {
    +        throw new IllegalStateException( "Could not parse plan: " + plan );
    +      }
    +      majorId = Integer.parseInt(m.group(1));
    +      stepId = Integer.parseInt(m.group(2));
    +      name = m.group(4);
    +      args = m.group(5);
    +      String tail = m.group(6);
    +      String indent = m.group(3);
    +      globalLevel = (indent.length() - 4) / 2;
    +
    +      p = Pattern.compile("rowType = RecordType\\((.*)\\): (rowcount .*)");
    +      m = p.matcher(tail);
    --- End diff --
    
    As it turns out, different scan operators use a different syntax for this info. This syntax
works for the scan operators used in tests thus far. We'll need to extend this for others.
    
    Actually, we'll want to combine this functionality with the full-blown parser used earlier.
This was a quick & dirty on that predated the fancier one. And, as noted, the real solution
is to include the info in the profile in JSON to avoid the need to fiddle with parsing the
text format.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message