poi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject [Bug 62081] New: java.lang.OutOfMemoryError: Map failed occurs when processing a bunch of msg files
Date Tue, 06 Feb 2018 15:36:52 GMT
https://bz.apache.org/bugzilla/show_bug.cgi?id=62081

            Bug ID: 62081
           Summary: java.lang.OutOfMemoryError: Map failed occurs when
                    processing a bunch of msg files
           Product: POI
           Version: 3.16-FINAL
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: critical
          Priority: P2
         Component: POI Overall
          Assignee: dev@poi.apache.org
          Reporter: satv73@gmail.com
  Target Milestone: ---

I am trying to read and extract contents of msg file using POI 3.16.  While
trying to process a bunch of msg files (not large, in the <500kb) and the
following error occurs.  
==========================================================================
org.apache.poi.EmptyFileException: The supplied file was empty (zero bytes
long)
        at
org.apache.poi.poifs.filesystem.NPOIFSFileSystem.<init>(NPOIFSFileSystem.java:218)
        at
org.apache.poi.poifs.filesystem.NPOIFSFileSystem.<init>(NPOIFSFileSystem.java:168)
        at
com.testcode.MSGWriter.MapiMessage.addNamedPropertiesToMessage(MapiMessage.java:236)
        at
com.testcode.MSGReader.MessageReader.parseAttachment(MessageReader.java:1352)
        at
com.testcode.MSGReader.MessageReader.parseDirectory(MessageReader.java:520)
        at
com.testcode.MSGReader.MessageReader.parseMessage(MessageReader.java:405)
        at
com.testcode.MSGReader.MessageReader.readMessageFile(MessageReader.java:314)
        at txtGeneration.txtGeneration.Task.run(MTMsgTotxtConvertor.java:83)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
java.lang.RuntimeException: java.io.IOException: Map failed
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.next(NPOIFSStream.java:173)
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.next(NPOIFSStream.java:142)
        at
org.apache.poi.poifs.filesystem.NPOIFSMiniStore.getBlockAt(NPOIFSMiniStore.java:69)
        at
org.apache.poi.poifs.filesystem.NPOIFSMiniStore.createBlockIfNeeded(NPOIFSMiniStore.java:97)
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBuffer.createBlockIfNeeded(NPOIFSStream.java:226)
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBuffer.write(NPOIFSStream.java:246)
        at
org.apache.poi.poifs.filesystem.NPOIFSDocument.store(NPOIFSDocument.java:143)
        at
org.apache.poi.poifs.filesystem.NPOIFSDocument.<init>(NPOIFSDocument.java:84)
        at
org.apache.poi.poifs.filesystem.DirectoryNode.createDocument(DirectoryNode.java:422)
        at
com.testcode.MSGWriter.MessageWriterHelper.writeNamedPropertyDocuments(MessageWriterHelper.java:938)
        at
com.testcode.MSGWriter.MapiMessage.addNamedPropertiesToMessage(MapiMessage.java:242)
        at
com.testcode.MSGReader.MessageReader.parseAttachment(MessageReader.java:1352)
        at
com.testcode.MSGReader.MessageReader.parseDirectory(MessageReader.java:520)
        at
com.testcode.MSGReader.MessageReader.parseMessage(MessageReader.java:405)
        at
com.testcode.MSGReader.MessageReader.readMessageFile(MessageReader.java:314)
        at txtGeneration.txtGeneration.Task.run(MTMsgTotxtConvertor.java:83)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Map failed
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:940)
        at
org.apache.poi.poifs.nio.FileBackedDataSource.read(FileBackedDataSource.java:99)
        at
org.apache.poi.poifs.filesystem.NPOIFSFileSystem.getBlockAt(NPOIFSFileSystem.java:484)
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.next(NPOIFSStream.java:169)
        ... 18 more
Caused by: java.lang.OutOfMemoryError: Map failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method)
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:937)
        ... 21 more
java.lang.RuntimeException: java.io.IOException: Map failed
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.next(NPOIFSStream.java:173)
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.next(NPOIFSStream.java:142)
        at
org.apache.poi.poifs.filesystem.NPOIFSMiniStore.getBlockAt(NPOIFSMiniStore.java:69)
        at
org.apache.poi.poifs.filesystem.NPOIFSMiniStore.createBlockIfNeeded(NPOIFSMiniStore.java:97)
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBuffer.createBlockIfNeeded(NPOIFSStream.java:226)
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBuffer.write(NPOIFSStream.java:246)
        at
org.apache.poi.poifs.filesystem.NPOIFSDocument.store(NPOIFSDocument.java:143)
        at
org.apache.poi.poifs.filesystem.NPOIFSDocument.<init>(NPOIFSDocument.java:84)
        at
org.apache.poi.poifs.filesystem.DirectoryNode.createDocument(DirectoryNode.java:422)
        at
com.testcode.MSGWriter.MessageWriterHelper.writeNamedPropertyDocuments(MessageWriterHelper.java:938)
        at
com.testcode.MSGWriter.MapiMessage.addNamedPropertiesToMessage(MapiMessage.java:242)
        at
com.testcode.MSGReader.MessageReader.parseAttachment(MessageReader.java:1352)
        at
com.testcode.MSGReader.MessageReader.parseDirectory(MessageReader.java:520)
        at
com.testcode.MSGReader.MessageReader.parseMessage(MessageReader.java:405)
        at
com.testcode.MSGReader.MessageReader.readMessageFile(MessageReader.java:314)
        at txtGeneration.txtGeneration.Task.run(MTMsgTotxtConvertor.java:83)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Map failed
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:940)
        at
org.apache.poi.poifs.nio.FileBackedDataSource.read(FileBackedDataSource.java:99)
        at
org.apache.poi.poifs.filesystem.NPOIFSFileSystem.getBlockAt(NPOIFSFileSystem.java:484)
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.next(NPOIFSStream.java:169)
        ... 18 more
Caused by: java.lang.OutOfMemoryError: Map failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method)
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:937)
        ... 21 more
Java HotSpot(TM) 64-Bit Server VM warning: INFO:
os::commit_memory(0x00007fd7c8a1a000, 262144, 0) failed; error='Cannot allocate
memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 262144 bytes for committing
reserved memory.
# An error report file with more information is saved as:
# /home/testuser/CVLibTest/hs_err_pid6684.log
[thread 140564401960704 also had an error]
Java HotSpot(TM) 64-Bit Server VM warning: INFO:
os::commit_memory(0x00007fd7c93f0000, 65536, 1) failed; error='Cannot allocate
memory' (errno=12)
[thread 140564395820800 also had an error]
==========================================================================
I see the following bug has been addressed in 3.15 already, but still seeing
the issue.  for now sysctl -w vm.max_map_count to a larger value seems to be
working, but can this be fixed?

Bug 60140 - OOM caused by Memory Leak in FileBackedDataSource

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org


Mime
View raw message