drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Parth Chandra (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-3579) Drill on Hive query fails if partition table has __HIVE_DEFAULT_PARTITION__
Date Fri, 31 Jul 2015 22:52:06 GMT

     [ https://issues.apache.org/jira/browse/DRILL-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Parth Chandra updated DRILL-3579:
---------------------------------
    Priority: Critical  (was: Major)

> Drill on Hive query fails if partition table has __HIVE_DEFAULT_PARTITION__
> ---------------------------------------------------------------------------
>
>                 Key: DRILL-3579
>                 URL: https://issues.apache.org/jira/browse/DRILL-3579
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Functions - Hive
>    Affects Versions: 1.1.0
>         Environment: Drill 1.1 on Hive 1.0
>            Reporter: Hao Zhu
>            Assignee: Venki Korukanti
>            Priority: Critical
>             Fix For: 1.2.0
>
>
> If Hive's partition table has __HIVE_DEFAULT_PARTITION__ in the case of null values in
the partition column, Drill on Hive query will fail.
> Minimum reproduce:
> 1.Hive:
> {code}
> CREATE TABLE h1_testpart2(id INT) PARTITIONED BY(id2 int);
> set hive.exec.dynamic.partition.mode=nonstrict;
> INSERT OVERWRITE TABLE h1_testpart2 PARTITION(id2) SELECT 1 as id1 , 20150101 as id2
from h1_passwords limit 1;
> INSERT OVERWRITE TABLE h1_testpart2 PARTITION(id2) SELECT 1 as id1 , null as id2 from
h1_passwords limit 1;
> {code}
> 2. Filesystem looks like:
> {code}
> h1 h1_testpart2]# ls -altr
> total 2
> drwxrwxrwx 89 mapr mapr 87 Jul 30 00:04 ..
> drwxr-xr-x  2 mapr mapr  1 Jul 30 00:05 id2=20150101
> drwxr-xr-x  2 mapr mapr  1 Jul 30 00:05 id2=__HIVE_DEFAULT_PARTITION__
> drwxr-xr-x  4 mapr mapr  2 Jul 30 00:05 .
> {code}
> 3.Drill will fail:
> {code}
> select * from h1_testpart2;
> Error: SYSTEM ERROR: NumberFormatException: For input string: "__HIVE_DEFAULT_PARTITION__"
> Fragment 0:0
> [Error Id: 509eb392-db9a-42f3-96ea-fb597425f49f on h1.poc.com:31010]
>   (java.lang.reflect.UndeclaredThrowableException) null
>     org.apache.hadoop.security.UserGroupInformation.doAs():1581
>     org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():136
>     org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173
>     org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():131
>     org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173
>     org.apache.drill.exec.physical.impl.ImplCreator.getRootExec():106
>     org.apache.drill.exec.physical.impl.ImplCreator.getExec():81
>     org.apache.drill.exec.work.fragment.FragmentExecutor.run():235
>     org.apache.drill.common.SelfCleaningRunnable.run():38
>     java.util.concurrent.ThreadPoolExecutor.runWorker():1142
>     java.util.concurrent.ThreadPoolExecutor$Worker.run():617
>     java.lang.Thread.run():745
>   Caused By (org.apache.drill.common.exceptions.ExecutionSetupException) Failure while
initializing HiveRecordReader: For input string: "__HIVE_DEFAULT_PARTITION__"
>     org.apache.drill.exec.store.hive.HiveRecordReader.init():241
>     org.apache.drill.exec.store.hive.HiveRecordReader.<init>():138
>     org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch():58
>     org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch():34
>     org.apache.drill.exec.physical.impl.ImplCreator$2.run():138
>     org.apache.drill.exec.physical.impl.ImplCreator$2.run():136
>     java.security.AccessController.doPrivileged():-2
>     javax.security.auth.Subject.doAs():422
>     org.apache.hadoop.security.UserGroupInformation.doAs():1566
>     org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():136
>     org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173
>     org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():131
>     org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173
>     org.apache.drill.exec.physical.impl.ImplCreator.getRootExec():106
>     org.apache.drill.exec.physical.impl.ImplCreator.getExec():81
>     org.apache.drill.exec.work.fragment.FragmentExecutor.run():235
>     org.apache.drill.common.SelfCleaningRunnable.run():38
>     java.util.concurrent.ThreadPoolExecutor.runWorker():1142
>     java.util.concurrent.ThreadPoolExecutor$Worker.run():617
>     java.lang.Thread.run():745
>   Caused By (java.lang.NumberFormatException) For input string: "__HIVE_DEFAULT_PARTITION__"
>     java.lang.NumberFormatException.forInputString():65
>     java.lang.Integer.parseInt():580
>     java.lang.Integer.parseInt():615
>     org.apache.drill.exec.store.hive.HiveRecordReader.convertPartitionType():605
>     org.apache.drill.exec.store.hive.HiveRecordReader.init():236
>     org.apache.drill.exec.store.hive.HiveRecordReader.<init>():138
>     org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch():58
>     org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch():34
>     org.apache.drill.exec.physical.impl.ImplCreator$2.run():138
>     org.apache.drill.exec.physical.impl.ImplCreator$2.run():136
>     java.security.AccessController.doPrivileged():-2
>     javax.security.auth.Subject.doAs():422
>     org.apache.hadoop.security.UserGroupInformation.doAs():1566
>     org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():136
>     org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173
>     org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():131
>     org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173
>     org.apache.drill.exec.physical.impl.ImplCreator.getRootExec():106
>     org.apache.drill.exec.physical.impl.ImplCreator.getExec():81
>     org.apache.drill.exec.work.fragment.FragmentExecutor.run():235
>     org.apache.drill.common.SelfCleaningRunnable.run():38
>     java.util.concurrent.ThreadPoolExecutor.runWorker():1142
>     java.util.concurrent.ThreadPoolExecutor$Worker.run():617
>     java.lang.Thread.run():745 (state=,code=0)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message