drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steven Phillips (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-4081) Handle schema changes in ExternalSort
Date Thu, 12 Nov 2015 21:50:11 GMT
Steven Phillips created DRILL-4081:

             Summary: Handle schema changes in ExternalSort
                 Key: DRILL-4081
                 URL: https://issues.apache.org/jira/browse/DRILL-4081
             Project: Apache Drill
          Issue Type: Improvement
            Reporter: Steven Phillips
            Assignee: Steven Phillips

This improvement will make use of the Union vector to handle schema changes. When a new schema
appears, the schema will be "merged" with the previous schema. The result will be a new schema
that uses Union type to store the columns where this is a type conflict. All of the batches
(including the batches that have already arrived) will be coerced into this new schema.

A new comparison function will be included to handle the comparison of Union type. Comparison
of union type will work as follows:

1. All numeric types can be mutually compared, and will be compared using Drill implicit cast

2. All other types will not be compared against other types, but only among values of the
same type.

3. There will be an overall precedence of types with regards to ordering. This precedence
is not yet defined, but will be as part of the work on this issue.

This message was sent by Atlassian JIRA

View raw message