drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Westin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-2767) Fragment error on TPCH Scale Factor 30 on a query that completed successfully previously
Date Mon, 13 Apr 2015 18:50:12 GMT

    [ https://issues.apache.org/jira/browse/DRILL-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14492835#comment-14492835
] 

Chris Westin commented on DRILL-2767:
-------------------------------------

The first exception in drillbit1.log:

2015-04-11 01:43:51,155 [2ad783a9-2bfd-793b-e9ec-b1ef9d2f5afd:frag:1:3] DEBUG o.a.d.exec.physical.impl.ScanBatch
- Failed to read the batch. Stopping...
org.apache.drill.common.exceptions.DrillRuntimeException: java.io.IOException: invalid distance
code
	at org.apache.drill.exec.store.hive.HiveRecordReader.next(HiveRecordReader.java:319) ~[drill-storage-hive-core-0.8.0.jar:0.8.0]
	at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:165) ~[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
	at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
	at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:68) [drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
	at org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:99)
[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
	at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:58) [drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
	at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:163) [drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
	at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) [drill-common-0.8.0-rebuffed.jar:0.8.0]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71]
	at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
Caused by: java.io.IOException: invalid distance code
	at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.inflateBytesDirect(Native Method)
~[hadoop-common-2.4.1.jar:na]
	at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.decompress(ZlibDecompressor.java:228)
~[hadoop-common-2.4.1.jar:na]
	at org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:91)
~[hadoop-common-2.4.1.jar:na]
	at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:85) ~[hadoop-common-2.4.1.jar:na]
	at java.io.InputStream.read(InputStream.java:101) ~[na:1.7.0_71]
	at org.apache.hadoop.util.LineReader.fillBuffer(LineReader.java:180) ~[hadoop-common-2.4.1.jar:na]
	at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216) ~[hadoop-common-2.4.1.jar:na]
	at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174) ~[hadoop-common-2.4.1.jar:na]
	at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:209) ~[hadoop-mapreduce-client-core-2.4.1.jar:na]
	at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:47) ~[hadoop-mapreduce-client-core-2.4.1.jar:na]
	at org.apache.drill.exec.store.hive.HiveRecordReader.next(HiveRecordReader.java:305) ~[drill-storage-hive-core-0.8.0.jar:0.8.0]
	... 10 common frames omitted
2015-04-11 01:43:51,156 [2ad783a9-2bfd-793b-e9ec-b1ef9d2f5afd:frag:1:3] ERROR o.a.drill.exec.ops.FragmentContext
- Fragment Context received failure.

First exception in drillbit2.log:
2015-04-11 01:43:51,130 [2ad783a9-2bfd-793b-e9ec-b1ef9d2f5afd:frag:1:8] DEBUG o.a.d.exec.physical.impl.ScanBatch
- Failed to read the batch. Stopping...
org.apache.drill.common.exceptions.DrillRuntimeException: java.io.IOException: invalid code
lengths set
	at org.apache.drill.exec.store.hive.HiveRecordReader.next(HiveRecordReader.java:319) ~[drill-storage-hive-core-0.8.0.jar:0.8.0]
	at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:165) ~[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
	at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
	at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:68) [drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
	at org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:99)
[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
	at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:58) [drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
	at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:163) [drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
	at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) [drill-common-0.8.0-rebuffed.jar:0.8.0]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71]
	at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
Caused by: java.io.IOException: invalid code lengths set
	at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.inflateBytesDirect(Native Method)
~[hadoop-common-2.4.1.jar:na]
	at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.decompress(ZlibDecompressor.java:228)
~[hadoop-common-2.4.1.jar:na]
	at org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:91)
~[hadoop-common-2.4.1.jar:na]
	at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:85) ~[hadoop-common-2.4.1.jar:na]
	at java.io.InputStream.read(InputStream.java:101) ~[na:1.7.0_71]
	at org.apache.hadoop.util.LineReader.fillBuffer(LineReader.java:180) ~[hadoop-common-2.4.1.jar:na]
	at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216) ~[hadoop-common-2.4.1.jar:na]
	at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174) ~[hadoop-common-2.4.1.jar:na]
	at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:209) ~[hadoop-mapreduce-client-core-2.4.1.jar:na]
	at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:47) ~[hadoop-mapreduce-client-core-2.4.1.jar:na]
	at org.apache.drill.exec.store.hive.HiveRecordReader.next(HiveRecordReader.java:305) ~[drill-storage-hive-core-0.8.0.jar:0.8.0]
	... 10 common frames omitted
2015-04-11 01:43:51,131 [2ad783a9-2bfd-793b-e9ec-b1ef9d2f5afd:frag:1:8] ERROR o.a.drill.exec.ops.FragmentContext
- Fragment Context received failure.
org.apache.drill.common.exceptions.DrillRuntimeException: java.io.IOException: invalid code
lengths set

Is there a problem reading compressed stuff from hive?


> Fragment error on TPCH Scale Factor 30 on a query that completed successfully previously
> ----------------------------------------------------------------------------------------
>
>                 Key: DRILL-2767
>                 URL: https://issues.apache.org/jira/browse/DRILL-2767
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - Hive
>         Environment: AWS EMR cluster of three m1.xlarge nodes
>            Reporter: Alexander Zarei
>         Attachments: drillbitcore1.log, drillbitcore1.out, drillbitcore2.log, drillbitcore2.out,
drillbitmaster.out
>
>
> The following sequence led to the error:
> Executed the query 
> bq. SELECT * FROM `realhive`.`tpch_text_30`.`lineitem`
> and it took about 43 minutes to execute successfully. 
> After ward I ran the query 
> bq. SELECT * FROM `realhive`.`tpch_text_2`.`lineitem`
> for 6 times to find an optimization value for the ODBC driver. 
> Afterward, I submitted the first query again
> bq. SELECT * FROM `realhive`.`tpch_text_30`.`lineitem`
>  
> and the Drill Cluster returned a fragment error.
> Log files with debug level for the Drillbits on the master node as well as the core nodes
of the cluster are attached.
> Also the connection through the ODBC driver on Linux 32 bit was "Direct" to the drillbit
on the master node of the Hadoop cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message