commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefan Bodewig (JIRA)" <>
Subject [jira] [Commented] (COMPRESS-358) Offset is larger than block size in IWA dialect of FramedSnappyCompressorInputStream
Date Tue, 10 Jan 2017 06:18:58 GMT


Stefan Bodewig commented on COMPRESS-358:

We use an internal lock size of 32k and this one is only configurable for the raw stream,
It seems the IWA files use offsets larger than that.
uses 128k which is way bigger than the size the "spec" talks about

Note that the data portion of the chunk, i.e., the compressed contents,
can be at most 16777211 bytes (2^24 - 1, minus the checksum).
However, we place an additional restriction that the uncompressed data
in a chunk must be no longer than 65536 bytes. This allows consumers to
easily use small fixed-size buffers.

Yet another deviation, I guess.

I'll make the block size configurable for the framed streams as well, probably won't manage
to do so today, though.

> Offset is larger than block size in IWA dialect of FramedSnappyCompressorInputStream
> ------------------------------------------------------------------------------------
>                 Key: COMPRESS-358
>                 URL:
>             Project: Commons Compress
>          Issue Type: Bug
>          Components: Compressors
>    Affects Versions: 1.12
>            Reporter: Tim Allison
>            Priority: Trivial
>         Attachments: DocumentStyleSheet_commons_unsnapped.iwa, DocumentStyleSheet_fromZipStream.iwa,
DocumentStylesheet.iwa, DocumentStylesheet_iwana_unsnapped.iwa, DocumentStylesheet_uncompressed.iwa,
> I finally was able to run the FramedSnappyCompressorInputStreamon a larger number of
.iwa files.  I got the following on two files:
> {noformat}
> Offset is larger than block size
> 	at org.apache.commons.compress.compressors.snappy.SnappyCompressorInputStream.expandCopy(
> 	at org.apache.commons.compress.compressors.snappy.SnappyCompressorInputStream.fill(
> 	at
> 	at org.apache.commons.compress.compressors.snappy.FramedSnappyCompressorInputStream.readOnce(
> 	at
> 	at
> 	at java.nio.file.Files.copy(
> 	at java.nio.file.Files.copy(
> {noformat}
> No good deed goes unpunished... :)  Thank you, again!

This message was sent by Atlassian JIRA

View raw message