drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Deneche A. Hakim (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-4542) if external sort fails to spill to disk, memory is leaked and wrong error message is displayed
Date Sat, 26 Mar 2016 09:53:25 GMT
Deneche A. Hakim created DRILL-4542:
---------------------------------------

             Summary: if external sort fails to spill to disk, memory is leaked and wrong
error message is displayed
                 Key: DRILL-4542
                 URL: https://issues.apache.org/jira/browse/DRILL-4542
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Relational Operators
    Affects Versions: 1.6.0
         Environment: When external sort fails spilling to disk, for example because there
is no space left on disk, the query fails with an error like this:
{noformat}
SYSTEM ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: (376832)
Allocator(op:1:0:6:ExternalSort) 20000000/376832/408432256/429496729 (res/actual/peak/limit)
{noformat}

2 problems here:

1. we are leaking memory

2. the "real" problem is not shown in the error message

Looking at the logs we can find the real issue, generally right after the previous error (edited
for the sake of clarity):
{noformat}
2016-03-26 02:22:28,576 [drill-executor-119] ERROR o.a.d.exec.server.BootStrapContext - org.apache.drill.exec.work.WorkManager$WorkerBee$2.run()
leaked an exception.
org.apache.hadoop.fs.FSError: java.io.IOException: No space left on device
        at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:249)
~[hadoop-common-2.7.0-mapr-1602.jar:na]
		[...]
        at org.apache.drill.exec.cache.VectorAccessibleSerializable.writeToStream(VectorAccessibleSerializable.java:172)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.xsort.BatchGroup.addBatch(BatchGroup.java:96)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.mergeAndSpill(ExternalSortBatch.java:562)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.innerNext(ExternalSortBatch.java:394)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
		[...]
{noformat}
            Reporter: Deneche A. Hakim






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message