drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venki Korukanti (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (DRILL-3893) Issue with Drill after Hive Alters the Table
Date Tue, 01 Dec 2015 22:05:11 GMT

     [ https://issues.apache.org/jira/browse/DRILL-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Venki Korukanti resolved DRILL-3893.
------------------------------------
       Resolution: Fixed
         Assignee: Venki Korukanti
    Fix Version/s: 1.4.0

>  Issue with Drill after Hive Alters the Table
> ---------------------------------------------
>
>                 Key: DRILL-3893
>                 URL: https://issues.apache.org/jira/browse/DRILL-3893
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Functions - Hive, Storage - Hive
>    Affects Versions: 1.0.0, 1.1.0
>         Environment: DEV
>            Reporter: arnab chatterjee
>            Assignee: Venki Korukanti
>             Fix For: 1.4.0
>
>
> I reproduced this again on another partitioned table with existing data.
> Providing some more details. I have enabled the version mode for errors. Drill is unable
to fetch the new column name that was introduced.This most likely to me seems  to me that
it’s still picking up the stale metadata of hive.
> if (!tableColumns.contains(columnName)) {
>             if (partitionNames.contains(columnName)) {
>               selectedPartitionNames.add(columnName);
>             } else {
>               throw new ExecutionSetupException(String.format("Column %s does not exist",
columnName));
>             }
>           }
> select testdata from testtable;
> Error: SYSTEM ERROR: ExecutionSetupException: Column testdata does not exist
> Fragment 0:0
> [Error Id: be5cccba-97f6-4cc4-94e8-c11a4c53c8f4 on x.x.com:9999]
>   (org.apache.drill.common.exceptions.ExecutionSetupException) Failure while initializing
HiveRecordReader: Column testdata does not exist
>     org.apache.drill.exec.store.hive.HiveRecordReader.init():241
>     org.apache.drill.exec.store.hive.HiveRecordReader.<init>():138
>     org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch():58
>     org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch():34
>     org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():150
>     org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173
>     org.apache.drill.exec.physical.impl.ImplCreator.getRootExec():106
>     org.apache.drill.exec.physical.impl.ImplCreator.getExec():81
>     org.apache.drill.exec.work.fragment.FragmentExecutor.run():235
>     org.apache.drill.common.SelfCleaningRunnable.run():38
>     java.util.concurrent.ThreadPoolExecutor.runWorker():1142
>     java.util.concurrent.ThreadPoolExecutor$Worker.run():617
>     java.lang.Thread.run():745
>   Caused By (org.apache.drill.common.exceptions.ExecutionSetupException) Column testdata
does not exist
>     org.apache.drill.exec.store.hive.HiveRecordReader.init():206
>     org.apache.drill.exec.store.hive.HiveRecordReader.<init>():138
>     org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch():58
>     org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch():34
>     org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():150
>     org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173
>     org.apache.drill.exec.physical.impl.ImplCreator.getRootExec():106
>     org.apache.drill.exec.physical.impl.ImplCreator.getExec():81
>     org.apache.drill.exec.work.fragment.FragmentExecutor.run():235
>     org.apache.drill.common.SelfCleaningRunnable.run():38
>     java.util.concurrent.ThreadPoolExecutor.runWorker():1142
>     java.util.concurrent.ThreadPoolExecutor$Worker.run():617
>     java.lang.Thread.run():745 (state=,code=0)
> #########################################################
> Please note that this is a partitioned table with existing data.
> Does Drill Cache the Meta somewhere and hence it’s not getting reflected immediately
?
> DRILL CLI
> > select x from xx;
> Error: SYSTEM ERROR: ExecutionSetupException: Column x does not exist
> Fragment 0:0
> [Error Id: 62086e22-1341-459e-87ce-430a24cc5119 on x.x.com:999] (state=,code=0)
> HIVE CLI
> hive> describe formatted x;
> OK
> # col_name              data_type               comment
> <columns>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message