drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abhishek Girish (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-1944) JsonReader fails to read arrays of size 2000
Date Tue, 24 Mar 2015 18:01:54 GMT

    [ https://issues.apache.org/jira/browse/DRILL-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378273#comment-14378273
] 

Abhishek Girish commented on DRILL-1944:
----------------------------------------

I'm hitting this issue on the latest master (Git.commit.ID: f1b59ed):
{code:sql}
> select * from `queries.json` limit 1;
Query failed: Query stopped., Record was too large to copy into vector. [ 98d26812-4a6a-4a66-add3-23a97177793c
on abhi5.qa.lab:31010 ]
Error: exception while executing query: Failure while executing query. (state=,code=0)
{code}

Error snippet from logs:
{code}
2015-03-24 10:46:23,737 [2aee5d11-b728-7659-fb2e-8bd65fae5f19:frag:0:0] ERROR o.a.d.e.physical.impl.ScreenCreator
- Error 98d26812-4a6a-4a66-add3-23a97177793c: Query stopped.
org.apache.drill.common.exceptions.DrillRuntimeException: Error reading JSON. - Parser was
at record: 1
        at org.apache.drill.exec.store.easy.json.JSONRecordReader.handleAndRaise(JSONRecordReader.java:104)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.store.easy.json.JSONRecordReader.next(JSONRecordReader.java:153)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:165) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:113)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:96)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:68) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:96)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:58) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:145)
[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_65]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_65]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
Caused by: java.io.IOException: Record was too large to copy into vector.
        at org.apache.drill.exec.store.easy.json.JSONRecordReader.next(JSONRecordReader.java:145)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        ... 22 common frames omitted
{code}

> JsonReader fails to read arrays of size 2000
> --------------------------------------------
>
>                 Key: DRILL-1944
>                 URL: https://issues.apache.org/jira/browse/DRILL-1944
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - JSON
>            Reporter: Rahul Challapalli
>            Assignee: Jason Altekruse
>         Attachments: error.log, scalar-array-2000.json
>
>
> git.commit.id.abbrev=b491cdb
> I tried a select * on a json file which contained a single array of strings. Drill fails
to read the file if the array has 2000 elements. However it works for an array size of 1500.
So I am not sure if it is a known limitation or a bug.
> Query :
> {code}
> jdbc:drill:schema=dfs.drillTestDir> select * from `data-shapes/wide-records/single/wide-record1.json`;
> Query failed: Query stopped., Record was too large to copy into vector. [ 5b320c2f-fff3-4de5-8095-8043c30510fd
on qa-node191.qa.lab:31010 ]
> {code}
> I attached the data file and the error log. Let me know if you have any questions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message