drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4081) Handle schema changes in ExternalSort
Date Mon, 16 Nov 2015 01:44:10 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15006129#comment-15006129
] 

ASF GitHub Bot commented on DRILL-4081:
---------------------------------------

Github user jacques-n commented on a diff in the pull request:

    https://github.com/apache/drill/pull/257#discussion_r44882906
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/UnionFunctions.java
---
    @@ -32,16 +33,72 @@
     import org.apache.drill.exec.expr.holders.UnionHolder;
     import org.apache.drill.exec.expr.holders.IntHolder;
     import org.apache.drill.exec.expr.holders.VarCharHolder;
    +import org.apache.drill.exec.resolver.TypeCastRules;
     import org.apache.drill.exec.vector.complex.impl.UnionReader;
     import org.apache.drill.exec.vector.complex.reader.FieldReader;
     
     import javax.inject.Inject;
    +import java.util.Set;
     
     /**
      * The class contains additional functions for union types in addition to those in GUnionFunctions
      */
     public class UnionFunctions {
     
    +  /**
    +   * Returns zero if the inputs have equivalent types. Two numeric types are considered
equivalent, as are a combination
    +   * of date/timestamp. If not equivalent, returns a value determined by the numeric
value of the MinorType enum
    --- End diff --
    
    let's add an additional ordering ordinal to the minortype enum so that we match Mongo's
ordering among types


> Handle schema changes in ExternalSort
> -------------------------------------
>
>                 Key: DRILL-4081
>                 URL: https://issues.apache.org/jira/browse/DRILL-4081
>             Project: Apache Drill
>          Issue Type: Improvement
>            Reporter: Steven Phillips
>            Assignee: Steven Phillips
>
> This improvement will make use of the Union vector to handle schema changes. When a new
schema appears, the schema will be "merged" with the previous schema. The result will be a
new schema that uses Union type to store the columns where this is a type conflict. All of
the batches (including the batches that have already arrived) will be coerced into this new
schema.
> A new comparison function will be included to handle the comparison of Union type. Comparison
of union type will work as follows:
> 1. All numeric types can be mutually compared, and will be compared using Drill implicit
cast rules.
> 2. All other types will not be compared against other types, but only among values of
the same type.
> 3. There will be an overall precedence of types with regards to ordering. This precedence
is not yet defined, but will be as part of the work on this issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message