drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aman Sinha (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-2222) For Parquet data check for schema change when we read footers during planning
Date Wed, 25 Feb 2015 01:02:06 GMT

     [ https://issues.apache.org/jira/browse/DRILL-2222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Aman Sinha updated DRILL-2222:
------------------------------
    Fix Version/s: 1.0.0

> For Parquet data check for schema change when we read footers during planning
> -----------------------------------------------------------------------------
>
>                 Key: DRILL-2222
>                 URL: https://issues.apache.org/jira/browse/DRILL-2222
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Query Planning & Optimization
>    Affects Versions: 0.7.0
>            Reporter: Aman Sinha
>            Assignee: Jinfeng Ni
>             Fix For: 1.0.0
>
>
> For parquet data, we are already reading the footers for row count information during
the planning phase.  It would make sense to also check for schema change as we read each file's
footer and throw an error early, otherwise we could fail in unexpected ways later during execution.
 For example, here's a run-time error:  
> {code}
>  "java.lang.IllegalStateException: Failure while reading vector.  Expected vector class
of org.apache.drill.exec.vector.NullableDecimal18Vector but was holding vector class org.apache.drill.exec.vector.NullableDecimal28SparseVector."
> {code}
> The reason for this error is that some data files have the column defined as decimal(15,
6) while others have decimal(19, 6).  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message