drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From paul-rogers <...@git.apache.org>
Subject [GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...
Date Wed, 23 Aug 2017 01:09:12 GMT
Github user paul-rogers commented on a diff in the pull request:

    https://github.com/apache/drill/pull/906#discussion_r134636771
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java
---
    @@ -768,4 +765,73 @@ else if (exprHasPrefix && refHasPrefix) {
           }
         }
       }
    +
    +  /**
    +   * handle FAST NONE specially when Project for query output. This happens when input
returns a
    +   * FAST NONE directly ( input does not return any batch with schema/data).
    +   *
    +   * Project operator has to return a batch with schema derived using the following 3
rules:
    +   *  Case 1:  *  ==>  expand into an empty list of columns.
    +   *  Case 2:  regular column reference ==> treat as nullable-int column
    +   *  Case 3:  expressions => Call ExpressionTreeMaterialization over an empty vector
contain.
    +   *           Once the expression is materialized without error, use the output type
of materialized
    +   *           expression.
    +   * The batch is constructed with the above rules, and recordCount = 0.
    +   * Returned with OK_NEW_SCHEMA to down-stream operator.
    +   */
    +  @Override
    +  protected IterOutcome handleFastNone() {
    +    if (! popConfig.isOutputProj()) {
    +      return super.handleFastNone();
    +    }
    +
    +    allocationVectors = new ArrayList<>();
    +    final List<NamedExpression> exprs = getExpressionList();
    +    final ErrorCollector collector = new ErrorCollectorImpl();
    +    VectorContainer fakedIncomingVC = new VectorContainer();
    +
    +    for (NamedExpression namedExpression : exprs) {
    +      if (namedExpression.getExpr() instanceof SchemaPath) {
    +        final NameSegment expr = ((SchemaPath) namedExpression.getExpr()).getRootSegment();
    +        if (expr.getPath().contains(StarColumnHelper.STAR_COLUMN)) {
    +          continue; // * would expand into an empty list.
    +        } else {
    +          final TypeProtos.MajorType majorType = TypeProtos.MajorType.newBuilder()
    +              .setMinorType(MinorType.INT)
    +              .setMode(TypeProtos.DataMode.OPTIONAL)
    +              .build();
    +
    +          MaterializedField outputField = MaterializedField.create(namedExpression.getRef().getRootSegment().getPath(),
majorType);
    +          final ValueVector vv = container.addOrGet(outputField, callBack);
    +          allocationVectors.add(vv);
    +        }
    +        continue;
    +      }
    +
    +      final LogicalExpression materializedExpr = ExpressionTreeMaterializer.materialize(namedExpression.getExpr(),
    +          fakedIncomingVC,
    +          collector,
    +          context.getFunctionRegistry(),
    +          true,
    +          unionTypeEnabled);
    +
    +      if (collector.hasErrors()) {
    +        throw new IllegalArgumentException(String.format("Failure while trying to materialize
expressions : %s.  Errors:\n %s.",
    +            namedExpression.getExpr(),
    +            collector.toErrorString()));
    +      }
    +
    +      final MaterializedField outputField = MaterializedField.create(namedExpression.getRef().getRootSegment().getPath(),
    +          materializedExpr.getMajorType());
    +      final ValueVector vv = container.addOrGet(outputField, callBack);
    +      allocationVectors.add(vv);
    +
    +    }
    +
    +    doAlloc(0);
    +    container.buildSchema(SelectionVectorMode.NONE);
    +    wasNone = true;
    +    return IterOutcome.OK_NEW_SCHEMA;
    +  }
    --- End diff --
    
    The above issues aside, a "mental execution" of the "fast none" code path suggests it
should work.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message