drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4081) Handle schema changes in ExternalSort
Date Mon, 16 Nov 2015 01:44:10 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15006129#comment-15006129

ASF GitHub Bot commented on DRILL-4081:

Github user jacques-n commented on a diff in the pull request:

    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/UnionFunctions.java
    @@ -32,16 +33,72 @@
     import org.apache.drill.exec.expr.holders.UnionHolder;
     import org.apache.drill.exec.expr.holders.IntHolder;
     import org.apache.drill.exec.expr.holders.VarCharHolder;
    +import org.apache.drill.exec.resolver.TypeCastRules;
     import org.apache.drill.exec.vector.complex.impl.UnionReader;
     import org.apache.drill.exec.vector.complex.reader.FieldReader;
     import javax.inject.Inject;
    +import java.util.Set;
      * The class contains additional functions for union types in addition to those in GUnionFunctions
     public class UnionFunctions {
    +  /**
    +   * Returns zero if the inputs have equivalent types. Two numeric types are considered
equivalent, as are a combination
    +   * of date/timestamp. If not equivalent, returns a value determined by the numeric
value of the MinorType enum
    --- End diff --
    let's add an additional ordering ordinal to the minortype enum so that we match Mongo's
ordering among types

> Handle schema changes in ExternalSort
> -------------------------------------
>                 Key: DRILL-4081
>                 URL: https://issues.apache.org/jira/browse/DRILL-4081
>             Project: Apache Drill
>          Issue Type: Improvement
>            Reporter: Steven Phillips
>            Assignee: Steven Phillips
> This improvement will make use of the Union vector to handle schema changes. When a new
schema appears, the schema will be "merged" with the previous schema. The result will be a
new schema that uses Union type to store the columns where this is a type conflict. All of
the batches (including the batches that have already arrived) will be coerced into this new
> A new comparison function will be included to handle the comparison of Union type. Comparison
of union type will work as follows:
> 1. All numeric types can be mutually compared, and will be compared using Drill implicit
cast rules.
> 2. All other types will not be compared against other types, but only among values of
the same type.
> 3. There will be an overall precedence of types with regards to ordering. This precedence
is not yet defined, but will be as part of the work on this issue.

This message was sent by Atlassian JIRA

View raw message