From dev-return-39481-archive-asf-public=cust-asf.ponee.io@poi.apache.org Tue Feb 6 16:37:00 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 0E7D1180657 for ; Tue, 6 Feb 2018 16:37:00 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id F25C5160C3A; Tue, 6 Feb 2018 15:36:59 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 200A8160C34 for ; Tue, 6 Feb 2018 16:36:58 +0100 (CET) Received: (qmail 96700 invoked by uid 500); 6 Feb 2018 15:36:58 -0000 Mailing-List: contact dev-help@poi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "POI Developers List" Delivered-To: mailing list dev@poi.apache.org Received: (qmail 96689 invoked by uid 99); 6 Feb 2018 15:36:58 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Feb 2018 15:36:58 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id B31E81A07F5 for ; Tue, 6 Feb 2018 15:36:57 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -1.511 X-Spam-Level: X-Spam-Status: No, score=-1.511 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id YnLfumLljMQG for ; Tue, 6 Feb 2018 15:36:56 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id B24B75F232 for ; Tue, 6 Feb 2018 15:36:55 +0000 (UTC) Received: from asf-bz1-us-mid.priv.apache.org (nat1-us-mid.apache.org [23.253.172.122]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTPS id 221EAE00CA for ; Tue, 6 Feb 2018 15:36:55 +0000 (UTC) Received: by asf-bz1-us-mid.priv.apache.org (ASF Mail Server at asf-bz1-us-mid.priv.apache.org, from userid 33) id CB7E960205; Tue, 6 Feb 2018 15:36:53 +0000 (UTC) From: bugzilla@apache.org To: dev@poi.apache.org Subject: [Bug 62081] New: java.lang.OutOfMemoryError: Map failed occurs when processing a bunch of msg files Date: Tue, 06 Feb 2018 15:36:52 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: POI X-Bugzilla-Component: POI Overall X-Bugzilla-Version: 3.16-FINAL X-Bugzilla-Keywords: X-Bugzilla-Severity: critical X-Bugzilla-Who: satv73@gmail.com X-Bugzilla-Status: NEW X-Bugzilla-Resolution: X-Bugzilla-Priority: P2 X-Bugzilla-Assigned-To: dev@poi.apache.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter target_milestone Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bz.apache.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 https://bz.apache.org/bugzilla/show_bug.cgi?id=3D62081 Bug ID: 62081 Summary: java.lang.OutOfMemoryError: Map failed occurs when processing a bunch of msg files Product: POI Version: 3.16-FINAL Hardware: PC OS: Linux Status: NEW Severity: critical Priority: P2 Component: POI Overall Assignee: dev@poi.apache.org Reporter: satv73@gmail.com Target Milestone: --- I am trying to read and extract contents of msg file using POI 3.16. While trying to process a bunch of msg files (not large, in the <500kb) and the following error occurs.=20=20 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D org.apache.poi.EmptyFileException: The supplied file was empty (zero bytes long) at org.apache.poi.poifs.filesystem.NPOIFSFileSystem.(NPOIFSFileSystem.ja= va:218) at org.apache.poi.poifs.filesystem.NPOIFSFileSystem.(NPOIFSFileSystem.ja= va:168) at com.testcode.MSGWriter.MapiMessage.addNamedPropertiesToMessage(MapiMessage.= java:236) at com.testcode.MSGReader.MessageReader.parseAttachment(MessageReader.java:135= 2) at com.testcode.MSGReader.MessageReader.parseDirectory(MessageReader.java:520) at com.testcode.MSGReader.MessageReader.parseMessage(MessageReader.java:405) at com.testcode.MSGReader.MessageReader.readMessageFile(MessageReader.java:314) at txtGeneration.txtGeneration.Task.run(MTMsgTotxtConvertor.java:83) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1= 149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:= 624) at java.lang.Thread.run(Thread.java:748) java.lang.RuntimeException: java.io.IOException: Map failed at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.= next(NPOIFSStream.java:173) at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.= next(NPOIFSStream.java:142) at org.apache.poi.poifs.filesystem.NPOIFSMiniStore.getBlockAt(NPOIFSMiniStore.= java:69) at org.apache.poi.poifs.filesystem.NPOIFSMiniStore.createBlockIfNeeded(NPOIFSM= iniStore.java:97) at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBuffer.createBl= ockIfNeeded(NPOIFSStream.java:226) at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBuffer.write(NP= OIFSStream.java:246) at org.apache.poi.poifs.filesystem.NPOIFSDocument.store(NPOIFSDocument.java:14= 3) at org.apache.poi.poifs.filesystem.NPOIFSDocument.(NPOIFSDocument.java:8= 4) at org.apache.poi.poifs.filesystem.DirectoryNode.createDocument(DirectoryNode.= java:422) at com.testcode.MSGWriter.MessageWriterHelper.writeNamedPropertyDocuments(Mess= ageWriterHelper.java:938) at com.testcode.MSGWriter.MapiMessage.addNamedPropertiesToMessage(MapiMessage.= java:242) at com.testcode.MSGReader.MessageReader.parseAttachment(MessageReader.java:135= 2) at com.testcode.MSGReader.MessageReader.parseDirectory(MessageReader.java:520) at com.testcode.MSGReader.MessageReader.parseMessage(MessageReader.java:405) at com.testcode.MSGReader.MessageReader.readMessageFile(MessageReader.java:314) at txtGeneration.txtGeneration.Task.run(MTMsgTotxtConvertor.java:83) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1= 149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:= 624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Map failed at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:940) at org.apache.poi.poifs.nio.FileBackedDataSource.read(FileBackedDataSource.jav= a:99) at org.apache.poi.poifs.filesystem.NPOIFSFileSystem.getBlockAt(NPOIFSFileSyste= m.java:484) at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.= next(NPOIFSStream.java:169) ... 18 more Caused by: java.lang.OutOfMemoryError: Map failed at sun.nio.ch.FileChannelImpl.map0(Native Method) at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:937) ... 21 more java.lang.RuntimeException: java.io.IOException: Map failed at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.= next(NPOIFSStream.java:173) at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.= next(NPOIFSStream.java:142) at org.apache.poi.poifs.filesystem.NPOIFSMiniStore.getBlockAt(NPOIFSMiniStore.= java:69) at org.apache.poi.poifs.filesystem.NPOIFSMiniStore.createBlockIfNeeded(NPOIFSM= iniStore.java:97) at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBuffer.createBl= ockIfNeeded(NPOIFSStream.java:226) at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBuffer.write(NP= OIFSStream.java:246) at org.apache.poi.poifs.filesystem.NPOIFSDocument.store(NPOIFSDocument.java:14= 3) at org.apache.poi.poifs.filesystem.NPOIFSDocument.(NPOIFSDocument.java:8= 4) at org.apache.poi.poifs.filesystem.DirectoryNode.createDocument(DirectoryNode.= java:422) at com.testcode.MSGWriter.MessageWriterHelper.writeNamedPropertyDocuments(Mess= ageWriterHelper.java:938) at com.testcode.MSGWriter.MapiMessage.addNamedPropertiesToMessage(MapiMessage.= java:242) at com.testcode.MSGReader.MessageReader.parseAttachment(MessageReader.java:135= 2) at com.testcode.MSGReader.MessageReader.parseDirectory(MessageReader.java:520) at com.testcode.MSGReader.MessageReader.parseMessage(MessageReader.java:405) at com.testcode.MSGReader.MessageReader.readMessageFile(MessageReader.java:314) at txtGeneration.txtGeneration.Task.run(MTMsgTotxtConvertor.java:83) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1= 149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:= 624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Map failed at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:940) at org.apache.poi.poifs.nio.FileBackedDataSource.read(FileBackedDataSource.jav= a:99) at org.apache.poi.poifs.filesystem.NPOIFSFileSystem.getBlockAt(NPOIFSFileSyste= m.java:484) at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.= next(NPOIFSStream.java:169) ... 18 more Caused by: java.lang.OutOfMemoryError: Map failed at sun.nio.ch.FileChannelImpl.map0(Native Method) at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:937) ... 21 more Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00007fd7c8a1a000, 262144, 0) failed; error=3D'Cannot al= locate memory' (errno=3D12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (mmap) failed to map 262144 bytes for committing reserved memory. # An error report file with more information is saved as: # /home/testuser/CVLibTest/hs_err_pid6684.log [thread 140564401960704 also had an error] Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00007fd7c93f0000, 65536, 1) failed; error=3D'Cannot all= ocate memory' (errno=3D12) [thread 140564395820800 also had an error] =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D I see the following bug has been addressed in 3.15 already, but still seeing the issue. for now sysctl -w vm.max_map_count to a larger value seems to be working, but can this be fixed? Bug 60140 - OOM caused by Memory Leak in FileBackedDataSource --=20 You are receiving this mail because: You are the assignee for the bug.= --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org For additional commands, e-mail: dev-help@poi.apache.org