drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-6129) Query fails on nested data type schema change
Date Thu, 01 Feb 2018 20:22:02 GMT

    [ https://issues.apache.org/jira/browse/DRILL-6129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349196#comment-16349196
] 

ASF GitHub Bot commented on DRILL-6129:
---------------------------------------

Github user amansinha100 commented on the issue:

    https://github.com/apache/drill/pull/1106
  
    Seems ok to fix the RecordBatchLoader.isSameSchema() since it is missing the checks for
nested columns.  In order to do the consolidation that Paul suggested,  you might want to
open an enhancement JIRA.  Since MaterializedField and RecordBatchLoader have separate class
hierarchies, there's not a direct way to have a single method do the comparison.   One option
is to create a utility 'SchemaComparator' class that incorporates various static utility methods
and Javadoc the rules for the comparisons. 
    
    So, I am +1  on this change. 


> Query fails on nested data type schema change
> ---------------------------------------------
>
>                 Key: DRILL-6129
>                 URL: https://issues.apache.org/jira/browse/DRILL-6129
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Client - CLI
>    Affects Versions: 1.10.0
>            Reporter: salim achouche
>            Assignee: salim achouche
>            Priority: Minor
>             Fix For: 1.13.0
>
>
> Use-Case -
>  * Assume two parquet files with similar schemas except for a nested column
>  * Schema file1
>  ** int64 field1
>  ** optional group field2
>  *** optional group field2.1 (LIST)
>  **** repeated group list
>  ***** optional group element
>  ****** optional int64 child_field
>  * Schema file2
>  ** int64 field1
>  ** optional group field2
>  *** optional group field2.1 (LIST)
>  **** repeated group list
>  ***** optional group element
>  ****** optional group child_field
>  ******* optional int64 child_field_f1
>  ******* optional int64 child_field_f1
>  * Essentially child_field changed from an int64 to a group of fields
>  
> Observed Query Failure
> select * from <file1 and file2>;
> Error: Unexpected RuntimeException: java.lang.IllegalArgumentException: The field $bits$(UINT1:REQUIRED)
doesn't match the provided metadata major_type {
>   minor_type: MAP
>   mode: REQUIRED
> Note that selecting one file at a time succeeds which seems to indicate the issue has
to do with the schema change logic. 
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message