commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Bodewig <bode...@apache.org>
Subject Re: [compress] uncompress a file containing lz4 stripes
Date Sat, 16 Sep 2017 16:38:50 GMT
On 2017-09-15, Simo Chiegang, Boris Arthur wrote:

> So, we develop a special kind of image file with the following structure:

> -          Header file [offset 0 to 4096 bytes]

> -          Stripe 1 [4096 - 20000 bytes] (LZ4 block compressed)

> -          Empty bloc [20000 - 22000 bytes]

> -          Stripe 2 [22000 - 35000] (LZ4 block compressed)

> -          ...

> Something like that. So now I want to read this file with the class
> BlockLZ4CompressorInputStream. It is possible?

In general yes, but probbaly not with a single instance of it (and you
may need to adapt something).

BlockLZ4CompressorInputStream should be able to read a single block, but
it wont be able to detect the end of the LZ4 block unless the underlying
stream is exhausted. There is no way to identify the last sequence other
than via EOF.

If your header contains information about the compressed size of your
block(s) then you can wrap the current input stream into a
BoundedInputStream that returns EOF once the block has been read
completely. You then create a BlockLZ4CompressorInputStream wrapped
around the BoundedInputStream - and do this for each stripe.

This is pretty much is what FramedLZ4CompressorInputStream does as well.

Things get a bit more complex if match offsets of stripe 2 are allowed
to point into the contents of stripe 1. FramedLZ4CompressorInputStream
handles this in the case where "block dependency" is expected.

Stefan

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org


Mime
View raw message