drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steven Phillips (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-3085) In ExternalSortBatch, Memory Leak in Runtime Generation Code
Date Fri, 15 May 2015 06:58:00 GMT

     [ https://issues.apache.org/jira/browse/DRILL-3085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Steven Phillips updated DRILL-3085:
-----------------------------------
    Fix Version/s: 1.0.0

> In ExternalSortBatch, Memory Leak in Runtime Generation Code
> ------------------------------------------------------------
>
>                 Key: DRILL-3085
>                 URL: https://issues.apache.org/jira/browse/DRILL-3085
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Codegen
>            Reporter: Sean Hsuan-Yi Chu
>            Assignee: Sean Hsuan-Yi Chu
>            Priority: Critical
>             Fix For: 1.0.0
>
>
> This case is related to DRILL-3065.
> In ExternalSortBatch, we have an MSorter to do sorting thing. In this class, there are
two SelectionVector4, vector4 and aux. If we fail at the time just after either gets new memory,
the close() method would fail to clean their allocated memory properly.
> To reproduce this problem, inject an exception at the last step of MSortTemplate.setup()
> Detailed Information:
> 1. Query: 
> select n_name from cp.`tpch/nation.parquet` order by n_name
> *. Using this query alone cannot help reproduce the issue. We still need to inject the
exception at the right place.
> 2. Data:
> cp.`tpch/nation.parquet`
> 3. Log:
> java.lang.IllegalStateException: Failure while closing accountor.  Expected private and
shared pools to be set to initial values.  However, one or more were not.  Stats are
> 	zone	init	allocated	delta 
> 	private	0	0	0 
> 	shared	3221225472	3195686243	25539229.
> 	at org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200)
> 	at org.apache.drill.exec.memory.Accountor.close(Accountor.java:386)
> 	at org.apache.drill.exec.memory.TopLevelAllocator.close(TopLevelAllocator.java:175)
> 	at org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:75)
> 	at com.google.common.io.Closeables.close(Closeables.java:77)
> 	at com.google.common.io.Closeables.closeQuietly(Closeables.java:108)
> 	at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:292)
> 	at org.apache.drill.BaseTestQuery.closeClient(BaseTestQuery.java:238)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> 	at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
> 	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
> 	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:211)
> 	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:67)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
> Exception in thread "Drillbit-ShutdownHook#0" java.lang.RuntimeException: Caught exception
closing Drillbit started from
> org.apache.drill.common.StackTrace.<init>:36
> org.apache.drill.exec.server.Drillbit.run:250
> org.apache.drill.BaseTestQuery.openClient:180
> org.apache.drill.BaseTestQuery.setupDefaultTestCluster:116
> sun.reflect.NativeMethodAccessorImpl.invoke0:-2
> sun.reflect.NativeMethodAccessorImpl.invoke:57
> sun.reflect.DelegatingMethodAccessorImpl.invoke:43
> java.lang.reflect.Method.invoke:606
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall:47
> org.junit.internal.runners.model.ReflectiveCallable.run:12
> org.junit.runners.model.FrameworkMethod.invokeExplosively:44
> org.junit.internal.runners.statements.RunBefores.evaluate:24
> org.junit.internal.runners.statements.RunAfters.evaluate:27
> org.junit.runners.ParentRunner.run:309
> org.junit.runner.JUnitCore.run:160
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs:74
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart:211
> com.intellij.rt.execution.junit.JUnitStarter.main:67
> sun.reflect.NativeMethodAccessorImpl.invoke0:-2
> sun.reflect.NativeMethodAccessorImpl.invoke:57
> sun.reflect.DelegatingMethodAccessorImpl.invoke:43
> java.lang.reflect.Method.invoke:606
> 	at org.apache.drill.exec.server.Drillbit$ShutdownThread.run(Drillbit.java:333)
> Caused by: java.lang.IllegalStateException: Failure while closing accountor.  Expected
private and shared pools to be set to initial values.  However, one or more were not.  Stats
are
> 	zone	init	allocated	delta 
> 	private	0	0	0 
> 	shared	3221225472	3195686243	25539229.
> 	at org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200)
> 	at org.apache.drill.exec.memory.Accountor.close(Accountor.java:386)
> 	at org.apache.drill.exec.memory.TopLevelAllocator.close(TopLevelAllocator.java:175)
> 	at org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:75)
> 	at com.google.common.io.Closeables.close(Closeables.java:77)
> 	at com.google.common.io.Closeables.closeQuietly(Closeables.java:108)
> 	at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:292)
> 	at org.apache.drill.exec.server.Drillbit$ShutdownThread.run(Drillbit.java:330)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message