drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-6079) Memory leak caused by ParquetRowGroupScan
Date Mon, 22 Jan 2018 21:34:00 GMT

    [ https://issues.apache.org/jira/browse/DRILL-6079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16334961#comment-16334961
] 

ASF GitHub Bot commented on DRILL-6079:
---------------------------------------

Github user parthchandra commented on the issue:

    https://github.com/apache/drill/pull/1087
  
    +1
    Thanks Salim. 


> Memory leak caused by ParquetRowGroupScan
> -----------------------------------------
>
>                 Key: DRILL-6079
>                 URL: https://issues.apache.org/jira/browse/DRILL-6079
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - Parquet
>            Reporter: salim achouche
>            Assignee: salim achouche
>            Priority: Minor
>             Fix For: 1.13.0
>
>
> Concurrency tests with assertion enabled indicate a memory leak in the Parquet scanner
code:
> 2017-10-25 17:28:52,149 [260ed3bc-dbdf-8b4a-66d6-b2bd804e8c74:frag:11:63] ERROR o.a.d.e.w.fragment.FragmentExecutor
- SYSTEM ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: (2097152)
> Allocator(op:11:63:6:ParquetRowGroupScan) 1000000/2097152/7348224/10000000000 (res/actual/peak/limit)
> Fragment 11:63
> [Error Id: 5927becb-f000-43db-95b5-93b33064a6fd on mperf113.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: IllegalStateException:
Memory was leaked by query. Memory leaked: (2097152)
> Allocator(op:11:63:6:ParquetRowGroupScan) 1000000/2097152/7348224/10000000000 (res/actual/peak/limit)
> Fragment 11:63
> [Error Id: 5927becb-f000-43db-95b5-93b33064a6fd on mperf113.qa.lab:31010]
>         at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:586)
~[drill-common-1.11.0-mapr.jar:1.11.0-mapr]
>         at org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:301)
[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
>         at org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
>         at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:267)
[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
>         at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.11.0-mapr.jar:1.11.0-mapr]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_121]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_121]
>         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> Caused by: java.lang.IllegalStateException: Memory was leaked by query. Memory leaked:
(2097152)
> Allocator(op:11:63:6:ParquetRowGroupScan) 1000000/2097152/7348224/10000000000 (res/actual/peak/limit)
>         at org.apache.drill.exec.memory.BaseAllocator.close(BaseAllocator.java:520) ~[drill-memory-base-1.11.0-mapr.jar:1.11.0-mapr]
>         at org.apache.drill.exec.ops.AbstractOperatorExecContext.close(AbstractOperatorExecContext.java:86)
~[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
>         at org.apache.drill.exec.ops.OperatorContextImpl.close(OperatorContextImpl.java:108)
~[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
>         at org.apache.drill.exec.ops.FragmentContext.suppressingClose(FragmentContext.java:435)
~[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
>         at org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:424)
~[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
>         at org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:324)
[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
>         at org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:155)
[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
>         ... 5 common frames omitted
> 2017-10-25 17:28:52,149 [260ed3bc-dbdf-8b4a-66d6-b2bd804e8c74:frag:6:0] ERROR o.a.d.e.w.fragment.FragmentExecutor
- SYSTEM ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: (2097152)
> Allocator(op:6:0:5:ParquetRowGroupScan) 1000000/2097152/19947520/10000000000 (res/actual/peak/limit)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message