drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mehant Baid (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-1092) 'IN' operator causing problems when used with order by on a hive table backed by avro data
Date Thu, 10 Jul 2014 20:52:04 GMT

    [ https://issues.apache.org/jira/browse/DRILL-1092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14057934#comment-14057934
] 

Mehant Baid commented on DRILL-1092:
------------------------------------

With the avro data source mentioned here, I am not able to query using Hive and hit the below
exception. Can we make sure we are able to complete queries using hive successfully? 
	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
	at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.doNextWithExceptionHandler(HadoopShimsSecure.java:304)
	at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.next(HadoopShimsSecure.java:220)
	at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:228)
	at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:213)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:418)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
	at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.io.IOException: org.apache.avro.AvroRuntimeException: java.io.EOFException
	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
	at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:288)
	at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:101)
	at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:41)
	at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:109)
	at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.doNextWithExceptionHandler(HadoopShimsSecure.java:302)
	... 11 more
Caused by: org.apache.avro.AvroRuntimeException: java.io.EOFException
	at org.apache.avro.file.DataFileStream.next(DataFileStream.java:222)
	at org.apache.hadoop.hive.ql.io.avro.AvroGenericRecordReader.next(AvroGenericRecordReader.java:149)
	at org.apache.hadoop.hive.ql.io.avro.AvroGenericRecordReader.next(AvroGenericRecordReader.java:52)
	at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:286)
	... 15 more
Caused by: java.io.EOFException
	at org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:462)
	at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:120)
	at org.apache.avro.io.ValidatingDecoder.readInt(ValidatingDecoder.java:83)
	at org.apache.avro.generic.GenericDatumReader.readInt(GenericDatumReader.java:364)
	at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
	at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:173)
	at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144)
	at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:135)
	at org.apache.avro.file.DataFileStream.next(DataFileStream.java:233)
	at org.apache.avro.file.DataFileStream.next(DataFileStream.java:220)


> 'IN' operator causing problems when used with order by on a hive table backed by avro
data
> ------------------------------------------------------------------------------------------
>
>                 Key: DRILL-1092
>                 URL: https://issues.apache.org/jira/browse/DRILL-1092
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Operators, Storage - Hive
>            Reporter: Rahul Challapalli
>            Assignee: Mehant Baid
>            Priority: Blocker
>         Attachments: error.log, plan_failure.txt, plan_success.txt, trade (3).ddl
>
>
> git.commit.id.abbrev=a1a6144
> build# 26266
> the below query fails to execute and the details of the error are present in the attached
log file. The hive table is backed by avro data. The hive ddl used is also attached. 
> select seqno from hive.trade where symbol in ('TZA', 'INDL') order by seqno;
> Tested the same query on a different data set using parquet and it has no issues
> select adRevenue from uservisits where countryCode in ('NLD', 'FRA') order by adRevenue;



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message