drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5546) Schema change problems caused by empty batch
Date Wed, 30 Aug 2017 05:46:01 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16146661#comment-16146661
] 

ASF GitHub Bot commented on DRILL-5546:
---------------------------------------

Github user jinfengni commented on a diff in the pull request:

    https://github.com/apache/drill/pull/906#discussion_r135974928
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java
---
    @@ -768,4 +765,73 @@ else if (exprHasPrefix && refHasPrefix) {
           }
         }
       }
    +
    +  /**
    +   * handle FAST NONE specially when Project for query output. This happens when input
returns a
    +   * FAST NONE directly ( input does not return any batch with schema/data).
    +   *
    +   * Project operator has to return a batch with schema derived using the following 3
rules:
    +   *  Case 1:  *  ==>  expand into an empty list of columns.
    +   *  Case 2:  regular column reference ==> treat as nullable-int column
    +   *  Case 3:  expressions => Call ExpressionTreeMaterialization over an empty vector
contain.
    +   *           Once the expression is materialized without error, use the output type
of materialized
    +   *           expression.
    +   * The batch is constructed with the above rules, and recordCount = 0.
    +   * Returned with OK_NEW_SCHEMA to down-stream operator.
    +   */
    +  @Override
    +  protected IterOutcome handleFastNone() {
    +    if (! popConfig.isOutputProj()) {
    +      return super.handleFastNone();
    +    }
    +
    +    allocationVectors = new ArrayList<>();
    +    final List<NamedExpression> exprs = getExpressionList();
    +    final ErrorCollector collector = new ErrorCollectorImpl();
    +    VectorContainer fakedIncomingVC = new VectorContainer();
    +
    +    for (NamedExpression namedExpression : exprs) {
    +      if (namedExpression.getExpr() instanceof SchemaPath) {
    +        final NameSegment expr = ((SchemaPath) namedExpression.getExpr()).getRootSegment();
    +        if (expr.getPath().contains(StarColumnHelper.STAR_COLUMN)) {
    +          continue; // * would expand into an empty list.
    +        } else {
    +          final TypeProtos.MajorType majorType = TypeProtos.MajorType.newBuilder()
    +              .setMinorType(MinorType.INT)
    +              .setMode(TypeProtos.DataMode.OPTIONAL)
    +              .build();
    +
    +          MaterializedField outputField = MaterializedField.create(namedExpression.getRef().getRootSegment().getPath(),
majorType);
    +          final ValueVector vv = container.addOrGet(outputField, callBack);
    +          allocationVectors.add(vv);
    +        }
    +        continue;
    +      }
    +
    +      final LogicalExpression materializedExpr = ExpressionTreeMaterializer.materialize(namedExpression.getExpr(),
    +          fakedIncomingVC,
    +          collector,
    +          context.getFunctionRegistry(),
    +          true,
    +          unionTypeEnabled);
    +
    +      if (collector.hasErrors()) {
    +        throw new IllegalArgumentException(String.format("Failure while trying to materialize
expressions : %s.  Errors:\n %s.",
    +            namedExpression.getExpr(),
    +            collector.toErrorString()));
    +      }
    --- End diff --
    
    This code has been removed in revised patch.
    



> Schema change problems caused by empty batch
> --------------------------------------------
>
>                 Key: DRILL-5546
>                 URL: https://issues.apache.org/jira/browse/DRILL-5546
>             Project: Apache Drill
>          Issue Type: Bug
>            Reporter: Jinfeng Ni
>            Assignee: Jinfeng Ni
>
> There have been a few JIRAs opened related to schema change failure caused by empty batch.
This JIRA is opened as an umbrella for all those related JIRAS ( such as DRILL-4686, DRILL-4734,
DRILL4476, DRILL-4255, etc).
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message