hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mustafa Iman (Jira)" <j...@apache.org>
Subject [jira] [Assigned] (HIVE-23455) Improve error message for external orc table
Date Tue, 12 May 2020 21:59:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-23455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mustafa Iman reassigned HIVE-23455:
-----------------------------------


> Improve error message for external orc table
> --------------------------------------------
>
>                 Key: HIVE-23455
>                 URL: https://issues.apache.org/jira/browse/HIVE-23455
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Mustafa Iman
>            Assignee: Mustafa Iman
>            Priority: Major
>
> Since there is no schema validation for external tables, users may face various errors
if their orc data and external table schema does not match. If orc schema has fewer columns
than projection OrcEncodedDataConsumer may receive an incomplete TypeDescription array which
will manifest itself as NullPointerException later.
> We can at least verify that OrcEncodedDataConsumer gets enough TypeDescriptions. If assertion
fails, user sees there is something wrong with the schema and hopefully resolves the problem
quickly. If there are enough columns in the file but the schema of the query does not match,
user generally sees a ClassCastException. If there are enough columns and types accidentally
match, there is nothing we can do as this is an external table.
> We have seen this when trying to use a managed table as external table location. Although
user facing schemas are the same, managed table has acid related metadata. I am adding a q
file demonstrating NullPointerException with TestMiniLlapLocalCliDriver and the output after
the fix. I haven't added this to precommit tests as it is hard to assert the exception message
from mini driver framework and effectively it is just changing the error.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message