drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Westin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-3070) Memory Leak when we run out of memory
Date Thu, 14 May 2015 23:01:01 GMT

     [ https://issues.apache.org/jira/browse/DRILL-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Westin updated DRILL-3070:
--------------------------------
    Fix Version/s: 1.0.0

> Memory Leak when we run out of memory
> -------------------------------------
>
>                 Key: DRILL-3070
>                 URL: https://issues.apache.org/jira/browse/DRILL-3070
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Flow
>            Reporter: Rahul Challapalli
>            Assignee: Jason Altekruse
>             Fix For: 1.0.0
>
>         Attachments: node1.log, node2.log
>
>
> git.commit.id.abbrev=d10769f
> No of Drillbits : 2
> DRILL_MAX_DIRECT_MEMORY="1G"
> I ran the below query on tpsh SF 100
> {code}
> select
>   l.l_shipmode,
>   sum(case
>     when o.o_orderpriority = '1-URGENT'
>       or o.o_orderpriority = '2-HIGH'
>       then 1
>     else 0
>   end) as high_line_count,
>   sum(case
>     when o.o_orderpriority <> '1-URGENT'
>       and o.o_orderpriority <> '2-HIGH'
>       then 1
>     else 0
>   end) as low_line_count
> from
>   orders o,
>   lineitem l
> where
>   o.o_orderkey = l.l_orderkey
>   and l.l_shipmode in ('TRUCK', 'REG AIR')
>   and l.l_commitdate < l.l_receiptdate
>   and l.l_shipdate < l.l_commitdate
>   and l.l_receiptdate >= date '1994-01-01'
>   and l.l_receiptdate < date '1994-01-01' + interval '1' year
> group by
>   l.l_shipmode
> order by
>   l.l_shipmode;
> +------------+-----------------+----------------+
> | l_shipmode | high_line_count | low_line_count |
> +------------+-----------------+----------------+
> java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR: org.apache.drill.exec.memory.OutOfMemoryRuntimeException:
Failure while reallocating buffer to 262152 bytes
> Fragment 7:30
> [Error Id: 65a09d7d-75f2-4bbb-b8f9-cfb0bdfc2655 on qa-node191.qa.lab:31010]
> 	at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> 	at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> 	at sqlline.SqlLine.print(SqlLine.java:1809)
> 	at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> 	at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> 	at sqlline.SqlLine.dispatch(SqlLine.java:889)
> 	at sqlline.SqlLine.begin(SqlLine.java:763)
> 	at sqlline.SqlLine.start(SqlLine.java:498)
> 	at sqlline.SqlLine.main(SqlLine.java:460)
> {code}
> Before memory :
> {code}
> 0: jdbc:drill:schema=dfs_eea> select * from sys.memory;
> +------------+------------+--------------+------------+----------------+--------------------+------------+
> |  hostname  | user_port  | heap_current |  heap_max  | direct_current | jvm_direct_current
| direct_max |
> +------------+------------+--------------+------------+----------------+--------------------+------------+
> | qa-node190.qa.lab | 31010      | 126726408    | 4151836672 | 11799087       | 100676940
         | 1073741824 |
> | qa-node191.qa.lab | 31010      | 299617984    | 4151836672 | 7750365        | 50345292
          | 1073741824 |
> +------------+------------+--------------+------------+----------------+--------------------+------------+
> {code}
> After Memory :
> {code}
> 0: jdbc:drill:schema=dfs_eea> select * from sys.memory;
> +------------+------------+--------------+------------+----------------+--------------------+------------+
> |  hostname  | user_port  | heap_current |  heap_max  | direct_current | jvm_direct_current
| direct_max |
> +------------+------------+--------------+------------+----------------+--------------------+------------+
> | qa-node190.qa.lab | 31010      | 446765888    | 4151836672 | 161309493      | 436221380
         | 1073741824 |
> | qa-node191.qa.lab | 31010      | 357580248    | 4151836672 | 199260917      | 603993536
         | 1073741824 |
> +------------+------------+--------------+------------+----------------+--------------------+------------+
> {code}
> While I expected this OOM error (as direct mem is 1GB), there seems to be a bump in directory_memory
on qa-node191.qa.lab. Not sure if this is normal
> - Rahul



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message