Mailing-List: contact issues-help@commons.apache.org; run by ezmlm
Precedence: bulk
Reply-To: issues@commons.apache.org
Date: Sat, 22 Apr 2017 14:13:04 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@commons.apache.org
Message-ID: <JIRA.13040268.1486161491000.26092.1492870384443@Atlassian.JIRA>
In-Reply-To: <JIRA.13040268.1486161491000@Atlassian.JIRA>
References: <JIRA.13040268.1486161491000@Atlassian.JIRA> <JIRA.13040268.1486161491332@jira-lw-us.apache.org>
Subject: [jira] [Commented] (COMPRESS-382) OutOfMemoryError from
 CompressorStreamFactory
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Sat, 22 Apr 2017 14:13:09 -0000


    [ https://issues.apache.org/jira/browse/COMPRESS-382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15979945#comment-15979945 ] 

ASF GitHub Bot commented on COMPRESS-382:
-----------------------------------------

Github user bodewig commented on a diff in the pull request:

    https://github.com/apache/commons-compress/pull/20#discussion_r112810573
  
    --- Diff: src/test/java/org/apache/commons/compress/compressors/DetectCompressorTestCase.java ---
    @@ -167,6 +171,26 @@ private String detect(String testFileName) throws IOException, CompressorExcepti
         }
     
         @Test
    +    public void testMemoryLimit() throws Exception {
    +        testMemoryLimit("COMPRESS-382");
    +        testMemoryLimit("COMPRESS-386");
    +    }
    +
    +    private void testMemoryLimit(String fileName) throws IOException, CompressorException {
    +        CompressorStreamFactory fac = new CompressorStreamFactory(true,
    +                100);
    +        try (InputStream is = new BufferedInputStream(
    +                new FileInputStream(getFile(fileName)))) {
    +            InputStream compressorInputStream = fac.createCompressorInputStream(is);
    +            fail("Should have thrown CompressorMemoryLimitException");
    --- End diff --
    
    +1 and make it two tests :-)


> OutOfMemoryError from CompressorStreamFactory
> ---------------------------------------------
>
>                 Key: COMPRESS-382
>                 URL: https://issues.apache.org/jira/browse/COMPRESS-382
>             Project: Commons Compress
>          Issue Type: Bug
>          Components: Compressors
>    Affects Versions: 1.10, 1.11, 1.12
>         Environment: Windows7, jre1.8.0_101 x64
>            Reporter: Luis Filipe Nassif
>         Attachments: data.mui
>
>
> While using Tika-1.14 to detect file types, the attached 1KB file triggered an OOME with 1GB heap. Tika calls CompressorStreamFactory.createCompressorInputStream(in) to detect if the file is a compressor stream, but CompressorStreamFactory erroneously detects it as a LZMACompressorInputStream and when the LZMACompressorInputStream is instanciated the OOME is thrown. This error does not happen with commons-compress versions prior to 1.10, when auto detecting LZMA streams was added. OOME stacktrace below:
> {code}
> Caused by: java.lang.OutOfMemoryError: Java heap space
> 	at org.tukaani.xz.lz.LZDecoder.<init>(Unknown Source) ~[xz-1.5.jar:1.5]
> 	at org.tukaani.xz.LZMAInputStream.initialize(Unknown Source) ~[xz-1.5.jar:1.5]
> 	at org.tukaani.xz.LZMAInputStream.initialize(Unknown Source) ~[xz-1.5.jar:1.5]
> 	at org.tukaani.xz.LZMAInputStream.<init>(Unknown Source) ~[xz-1.5.jar:1.5]
> 	at org.tukaani.xz.LZMAInputStream.<init>(Unknown Source) ~[xz-1.5.jar:1.5]
> 	at org.apache.commons.compress.compressors.lzma.LZMACompressorInputStream.<init>(LZMACompressorInputStream.java:48) ~[commons-compress-1.10.jar:1.10]
> 	at org.apache.commons.compress.compressors.CompressorStreamFactory.createCompressorInputStream(CompressorStreamFactory.java:251) ~[commons-compress-1.10.jar:1.10]
> 	at org.apache.tika.parser.pkg.ZipContainerDetector.detectCompressorFormat(ZipContainerDetector.java:109) ~[tika-parsers-1.14.jar:1.14]
> 	at org.apache.tika.parser.pkg.ZipContainerDetector.detect(ZipContainerDetector.java:95) ~[tika-parsers-1.14.jar:1.14]
> 	at org.apache.tika.detect.CompositeDetector.detect(CompositeDetector.java:77) ~[tika-core-1.14.jar:1.14]
> 	at dpf.sp.gpinf.indexer.process.task.SignatureTask.process(SignatureTask.java:50) ~[iped.jar:?]
> 	at dpf.sp.gpinf.indexer.process.task.AbstractTask.processMonitorTimeout(AbstractTask.java:203) ~[iped.jar:?]
> 	at dpf.sp.gpinf.indexer.process.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:152) ~[iped.jar:?]
> 	at dpf.sp.gpinf.indexer.process.task.AbstractTask.sendToNextTask(AbstractTask.java:190) ~[iped.jar:?]
> 	at dpf.sp.gpinf.indexer.process.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:160) ~[iped.jar:?]
> 	at dpf.sp.gpinf.indexer.process.task.AbstractTask.sendToNextTask(AbstractTask.java:190) ~[iped.jar:?]
> 	at dpf.sp.gpinf.indexer.process.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:160) ~[iped.jar:?]
> 	at dpf.sp.gpinf.indexer.process.task.AbstractTask.sendToNextTask(AbstractTask.java:190) ~[iped.jar:?]
> 	at dpf.sp.gpinf.indexer.process.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:160) ~[iped.jar:?]
> 	at dpf.sp.gpinf.indexer.process.Worker.process(Worker.java:174) ~[iped.jar:?]
> 	... 1 more
> {code}


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)