drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vitalii Diravka (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-5544) Out of heap running CTAS against text delimited
Date Fri, 26 May 2017 11:40:04 GMT
Vitalii Diravka created DRILL-5544:

             Summary: Out of heap running CTAS against text delimited
                 Key: DRILL-5544
                 URL: https://issues.apache.org/jira/browse/DRILL-5544
             Project: Apache Drill
          Issue Type: Bug
          Components: Storage - Parquet
    Affects Versions: 1.10.0
         Environment: - 4 nodes cluster
- 8G of Java heap and 64G of direct memory
- planner.width.max_per_node = 40

To generate lineitem.tbl file unzip dbgen.tgz archive and run:
{code}dbgen -TL -s 500{code}
            Reporter: Vitalii Diravka
            Assignee: Vitalii Diravka
             Fix For: 1.11.0

This query causes the drillbit to hang:
create table xyz as
cast(columns[0] as bigint) l_orderkey,
cast(columns[1] as integer) l_poartkey,
cast(columns[2] as integer) l_suppkey,
cast(columns[3] as integer) l_linenumber,
cast(columns[4] as double) l_quantity,
cast(columns[5] as double) l_extendedprice,
cast(columns[6] as double) l_discount,
cast(columns[7] as double) l_tax,
cast(columns[8] as char(1)) l_returnflag,
cast(columns[9] as char(1)) l_linestatus,
cast(columns[10] as date) l_shipdate,
cast(columns[11] as date) l_commitdate,
cast(columns[12] as date) l_receiptdate,
cast(columns[13] as char(25)) l_shipinstruct,
cast(columns[14] as char(10)) l_shipmode,
cast(columns[15] as varchar(44)) l_comment

OOM "Java heap space" from the drillbit.log:
2017-02-07 22:38:11,031 [2765b496-0b5b-a3df-c252-a8bb9cd2e52f:frag:1:53] DEBUG o.a.d.e.s.p.ParquetDirectByteBufferAllocator
- ParquetDirectByteBufferAllocator: Allocated 209715 bytes. Allocated ByteBuffer id: 1563631814
2017-02-07 22:38:16,478 [2765b496-0b5b-a3df-c252-a8bb9cd2e52f:frag:1:1] ERROR o.a.d.exec.server.BootStrapContext
- org.apache.drill.exec.work.WorkManager$WorkerBee$1.run() leaked an exception.
java.lang.OutOfMemoryError: Java heap space
2017-02-07 22:38:17,391 [2765b496-0b5b-a3df-c252-a8bb9cd2e52f:frag:1:13] ERROR o.a.drill.common.CatastrophicFailure
- Catastrophic Failure Occurred, exiting. Information message: Unable to handle out of memory
condition in FragmentExecutor.

This message was sent by Atlassian JIRA

View raw message