Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 90A10200C11 for ; Sat, 4 Feb 2017 17:19:43 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 8F642160B63; Sat, 4 Feb 2017 16:19:43 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 51899160B6B for ; Sat, 4 Feb 2017 17:19:41 +0100 (CET) Received: (qmail 29198 invoked by uid 500); 4 Feb 2017 16:19:40 -0000 Mailing-List: contact notifications-help@commons.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@commons.apache.org Delivered-To: mailing list notifications@commons.apache.org Received: (qmail 29178 invoked by uid 99); 4 Feb 2017 16:19:40 -0000 Received: from Unknown (HELO svn01-us-west.apache.org) (209.188.14.144) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 04 Feb 2017 16:19:40 +0000 Received: from svn01-us-west.apache.org (localhost [127.0.0.1]) by svn01-us-west.apache.org (ASF Mail Server at svn01-us-west.apache.org) with ESMTP id 306383A3C96 for ; Sat, 4 Feb 2017 16:19:39 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1006212 [6/16] - in /websites/production/commons/content/proper/commons-compress: ./ apidocs/org/apache/commons/compress/compressors/lz4/ apidocs/org/apache/commons/compress/compressors/snappy/ apidocs/src-html/org/apache/commons/compress/... Date: Sat, 04 Feb 2017 16:19:38 -0000 To: notifications@commons.apache.org From: bodewig@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20170204161939.306383A3C96@svn01-us-west.apache.org> archived-at: Sat, 04 Feb 2017 16:19:43 -0000 Modified: websites/production/commons/content/proper/commons-compress/apidocs/src-html/org/apache/commons/compress/compressors/snappy/SnappyCompressorInputStream.html ============================================================================== --- websites/production/commons/content/proper/commons-compress/apidocs/src-html/org/apache/commons/compress/compressors/snappy/SnappyCompressorInputStream.html (original) +++ websites/production/commons/content/proper/commons-compress/apidocs/src-html/org/apache/commons/compress/compressors/snappy/SnappyCompressorInputStream.html Sat Feb 4 16:19:37 2017 @@ -31,268 +31,267 @@ 023 024import org.apache.commons.compress.compressors.lz77support.AbstractLZ77CompressorInputStream; 025import org.apache.commons.compress.utils.ByteUtils; -026import org.apache.commons.compress.utils.IOUtils; -027 -028/** -029 * CompressorInputStream for the raw Snappy format. -030 * -031 * <p>This implementation uses an internal buffer in order to handle -032 * the back-references that are at the heart of the LZ77 algorithm. -033 * The size of the buffer must be at least as big as the biggest -034 * offset used in the compressed stream. The current version of the -035 * Snappy algorithm as defined by Google works on 32k blocks and -036 * doesn't contain offsets bigger than 32k which is the default block -037 * size used by this class.</p> -038 * -039 * @see <a href="https://github.com/google/snappy/blob/master/format_description.txt">Snappy compressed format description</a> -040 * @since 1.7 -041 */ -042public class SnappyCompressorInputStream extends AbstractLZ77CompressorInputStream { -043 -044 /** Mask used to determine the type of "tag" is being processed */ -045 private static final int TAG_MASK = 0x03; -046 -047 /** Default block size */ -048 public static final int DEFAULT_BLOCK_SIZE = 32768; -049 -050 /** The size of the uncompressed data */ -051 private final int size; -052 -053 /** Number of uncompressed bytes still to be read. */ -054 private int uncompressedBytesRemaining; -055 -056 /** Current state of the stream */ -057 private State state = State.NO_BLOCK; -058 -059 private boolean endReached = false; -060 -061 /** -062 * Constructor using the default buffer size of 32k. -063 * -064 * @param is -065 * An InputStream to read compressed data from -066 * -067 * @throws IOException if reading fails -068 */ -069 public SnappyCompressorInputStream(final InputStream is) throws IOException { -070 this(is, DEFAULT_BLOCK_SIZE); -071 } -072 -073 /** -074 * Constructor using a configurable buffer size. -075 * -076 * @param is -077 * An InputStream to read compressed data from -078 * @param blockSize -079 * The block size used in compression -080 * -081 * @throws IOException if reading fails -082 */ -083 public SnappyCompressorInputStream(final InputStream is, final int blockSize) -084 throws IOException { -085 super(is, blockSize); -086 uncompressedBytesRemaining = size = (int) readSize(); -087 } -088 -089 /** -090 * {@inheritDoc} -091 */ -092 @Override -093 public int read(final byte[] b, final int off, final int len) throws IOException { -094 if (endReached) { -095 return -1; -096 } -097 switch (state) { -098 case NO_BLOCK: -099 fill(len); -100 return read(b, off, len); -101 case IN_LITERAL: -102 int litLen = readLiteral(b, off, len); -103 if (!hasMoreDataInBlock()) { -104 state = State.NO_BLOCK; -105 } -106 return litLen; -107 case IN_BACK_REFERENCE: -108 int backReferenceLen = readBackReference(b, off, len); -109 if (!hasMoreDataInBlock()) { -110 state = State.NO_BLOCK; -111 } -112 return backReferenceLen; -113 default: -114 throw new IOException("Unknown stream state " + state); -115 } -116 } -117 -118 /** -119 * Try to fill the buffer with enough bytes to satisfy the current -120 * read request. -121 * -122 * @param len the number of uncompressed bytes to read -123 */ -124 private void fill(final int len) throws IOException { -125 if (uncompressedBytesRemaining == 0) { -126 endReached = true; -127 return; -128 } -129 -130 int b = readOneByte(); -131 if (b == -1) { -132 throw new IOException("Premature end of stream reading block start"); -133 } -134 int length = 0; -135 int offset = 0; +026 +027/** +028 * CompressorInputStream for the raw Snappy format. +029 * +030 * <p>This implementation uses an internal buffer in order to handle +031 * the back-references that are at the heart of the LZ77 algorithm. +032 * The size of the buffer must be at least as big as the biggest +033 * offset used in the compressed stream. The current version of the +034 * Snappy algorithm as defined by Google works on 32k blocks and +035 * doesn't contain offsets bigger than 32k which is the default block +036 * size used by this class.</p> +037 * +038 * @see <a href="https://github.com/google/snappy/blob/master/format_description.txt">Snappy compressed format description</a> +039 * @since 1.7 +040 */ +041public class SnappyCompressorInputStream extends AbstractLZ77CompressorInputStream { +042 +043 /** Mask used to determine the type of "tag" is being processed */ +044 private static final int TAG_MASK = 0x03; +045 +046 /** Default block size */ +047 public static final int DEFAULT_BLOCK_SIZE = 32768; +048 +049 /** The size of the uncompressed data */ +050 private final int size; +051 +052 /** Number of uncompressed bytes still to be read. */ +053 private int uncompressedBytesRemaining; +054 +055 /** Current state of the stream */ +056 private State state = State.NO_BLOCK; +057 +058 private boolean endReached = false; +059 +060 /** +061 * Constructor using the default buffer size of 32k. +062 * +063 * @param is +064 * An InputStream to read compressed data from +065 * +066 * @throws IOException if reading fails +067 */ +068 public SnappyCompressorInputStream(final InputStream is) throws IOException { +069 this(is, DEFAULT_BLOCK_SIZE); +070 } +071 +072 /** +073 * Constructor using a configurable buffer size. +074 * +075 * @param is +076 * An InputStream to read compressed data from +077 * @param blockSize +078 * The block size used in compression +079 * +080 * @throws IOException if reading fails +081 */ +082 public SnappyCompressorInputStream(final InputStream is, final int blockSize) +083 throws IOException { +084 super(is, blockSize); +085 uncompressedBytesRemaining = size = (int) readSize(); +086 } +087 +088 /** +089 * {@inheritDoc} +090 */ +091 @Override +092 public int read(final byte[] b, final int off, final int len) throws IOException { +093 if (endReached) { +094 return -1; +095 } +096 switch (state) { +097 case NO_BLOCK: +098 fill(); +099 return read(b, off, len); +100 case IN_LITERAL: +101 int litLen = readLiteral(b, off, len); +102 if (!hasMoreDataInBlock()) { +103 state = State.NO_BLOCK; +104 } +105 return litLen; +106 case IN_BACK_REFERENCE: +107 int backReferenceLen = readBackReference(b, off, len); +108 if (!hasMoreDataInBlock()) { +109 state = State.NO_BLOCK; +110 } +111 return backReferenceLen; +112 default: +113 throw new IOException("Unknown stream state " + state); +114 } +115 } +116 +117 /** +118 * Try to fill the buffer with the next block of data. +119 */ +120 private void fill() throws IOException { +121 if (uncompressedBytesRemaining == 0) { +122 endReached = true; +123 return; +124 } +125 +126 int b = readOneByte(); +127 if (b == -1) { +128 throw new IOException("Premature end of stream reading block start"); +129 } +130 int length = 0; +131 int offset = 0; +132 +133 switch (b & TAG_MASK) { +134 +135 case 0x00: 136 -137 switch (b & TAG_MASK) { -138 -139 case 0x00: -140 -141 length = readLiteralLength(b); -142 uncompressedBytesRemaining -= length; -143 startLiteral(length); -144 state = State.IN_LITERAL; -145 break; -146 -147 case 0x01: -148 -149 /* -150 * These elements can encode lengths between [4..11] bytes and -151 * offsets between [0..2047] bytes. (len-4) occupies three bits -152 * and is stored in bits [2..4] of the tag byte. The offset -153 * occupies 11 bits, of which the upper three are stored in the -154 * upper three bits ([5..7]) of the tag byte, and the lower -155 * eight are stored in a byte following the tag byte. -156 */ -157 -158 length = 4 + ((b >> 2) & 0x07); -159 uncompressedBytesRemaining -= length; -160 offset = (b & 0xE0) << 3; -161 b = readOneByte(); -162 if (b == -1) { -163 throw new IOException("Premature end of stream reading back-reference length"); -164 } -165 offset |= b; +137 length = readLiteralLength(b); +138 uncompressedBytesRemaining -= length; +139 startLiteral(length); +140 state = State.IN_LITERAL; +141 break; +142 +143 case 0x01: +144 +145 /* +146 * These elements can encode lengths between [4..11] bytes and +147 * offsets between [0..2047] bytes. (len-4) occupies three bits +148 * and is stored in bits [2..4] of the tag byte. The offset +149 * occupies 11 bits, of which the upper three are stored in the +150 * upper three bits ([5..7]) of the tag byte, and the lower +151 * eight are stored in a byte following the tag byte. +152 */ +153 +154 length = 4 + ((b >> 2) & 0x07); +155 uncompressedBytesRemaining -= length; +156 offset = (b & 0xE0) << 3; +157 b = readOneByte(); +158 if (b == -1) { +159 throw new IOException("Premature end of stream reading back-reference length"); +160 } +161 offset |= b; +162 +163 startBackReference(offset, length); +164 state = State.IN_BACK_REFERENCE; +165 break; 166 -167 startBackReference(offset, length); -168 state = State.IN_BACK_REFERENCE; -169 break; -170 -171 case 0x02: -172 -173 /* -174 * These elements can encode lengths between [1..64] and offsets -175 * from [0..65535]. (len-1) occupies six bits and is stored in -176 * the upper six bits ([2..7]) of the tag byte. The offset is -177 * stored as a little-endian 16-bit integer in the two bytes -178 * following the tag byte. -179 */ -180 -181 length = (b >> 2) + 1; -182 uncompressedBytesRemaining -= length; -183 -184 offset = (int) ByteUtils.fromLittleEndian(supplier, 2); +167 case 0x02: +168 +169 /* +170 * These elements can encode lengths between [1..64] and offsets +171 * from [0..65535]. (len-1) occupies six bits and is stored in +172 * the upper six bits ([2..7]) of the tag byte. The offset is +173 * stored as a little-endian 16-bit integer in the two bytes +174 * following the tag byte. +175 */ +176 +177 length = (b >> 2) + 1; +178 uncompressedBytesRemaining -= length; +179 +180 offset = (int) ByteUtils.fromLittleEndian(supplier, 2); +181 +182 startBackReference(offset, length); +183 state = State.IN_BACK_REFERENCE; +184 break; 185 -186 startBackReference(offset, length); -187 state = State.IN_BACK_REFERENCE; -188 break; -189 -190 case 0x03: -191 -192 /* -193 * These are like the copies with 2-byte offsets (see previous -194 * subsection), except that the offset is stored as a 32-bit -195 * integer instead of a 16-bit integer (and thus will occupy -196 * four bytes). -197 */ -198 -199 length = (b >> 2) + 1; -200 uncompressedBytesRemaining -= length; -201 -202 offset = (int) ByteUtils.fromLittleEndian(supplier, 4) & 0x7fffffff; -203 -204 startBackReference(offset, length); -205 state = State.IN_BACK_REFERENCE; -206 break; -207 } -208 } -209 -210 /* -211 * For literals up to and including 60 bytes in length, the -212 * upper six bits of the tag byte contain (len-1). The literal -213 * follows immediately thereafter in the bytestream. - For -214 * longer literals, the (len-1) value is stored after the tag -215 * byte, little-endian. The upper six bits of the tag byte -216 * describe how many bytes are used for the length; 60, 61, 62 -217 * or 63 for 1-4 bytes, respectively. The literal itself follows -218 * after the length. -219 */ -220 private int readLiteralLength(final int b) throws IOException { -221 int length; -222 switch (b >> 2) { -223 case 60: -224 length = readOneByte(); -225 if (length == -1) { -226 throw new IOException("Premature end of stream reading literal length"); -227 } -228 break; -229 case 61: -230 length = (int) ByteUtils.fromLittleEndian(supplier, 2); -231 break; -232 case 62: -233 length = (int) ByteUtils.fromLittleEndian(supplier, 3); -234 break; -235 case 63: -236 length = (int) ByteUtils.fromLittleEndian(supplier, 4); -237 break; -238 default: -239 length = b >> 2; -240 break; -241 } -242 -243 return length + 1; -244 } -245 -246 /** -247 * The stream starts with the uncompressed length (up to a maximum of 2^32 - -248 * 1), stored as a little-endian varint. Varints consist of a series of -249 * bytes, where the lower 7 bits are data and the upper bit is set iff there -250 * are more bytes to be read. In other words, an uncompressed length of 64 -251 * would be stored as 0x40, and an uncompressed length of 2097150 (0x1FFFFE) -252 * would be stored as 0xFE 0xFF 0x7F. -253 * -254 * @return The size of the uncompressed data -255 * -256 * @throws IOException -257 * Could not read a byte -258 */ -259 private long readSize() throws IOException { -260 int index = 0; -261 long sz = 0; -262 int b = 0; -263 -264 do { -265 b = readOneByte(); -266 if (b == -1) { -267 throw new IOException("Premature end of stream reading size"); -268 } -269 sz |= (b & 0x7f) << (index++ * 7); -270 } while (0 != (b & 0x80)); -271 return sz; -272 } -273 -274 /** -275 * Get the uncompressed size of the stream -276 * -277 * @return the uncompressed size -278 */ -279 @Override -280 public int getSize() { -281 return size; -282 } -283 -284 private enum State { -285 NO_BLOCK, IN_LITERAL, IN_BACK_REFERENCE -286 } -287} +186 case 0x03: +187 +188 /* +189 * These are like the copies with 2-byte offsets (see previous +190 * subsection), except that the offset is stored as a 32-bit +191 * integer instead of a 16-bit integer (and thus will occupy +192 * four bytes). +193 */ +194 +195 length = (b >> 2) + 1; +196 uncompressedBytesRemaining -= length; +197 +198 offset = (int) ByteUtils.fromLittleEndian(supplier, 4) & 0x7fffffff; +199 +200 startBackReference(offset, length); +201 state = State.IN_BACK_REFERENCE; +202 break; +203 default: +204 // impossible as TAG_MASK is two bits and all four possible cases have been covered +205 break; +206 } +207 } +208 +209 /* +210 * For literals up to and including 60 bytes in length, the +211 * upper six bits of the tag byte contain (len-1). The literal +212 * follows immediately thereafter in the bytestream. - For +213 * longer literals, the (len-1) value is stored after the tag +214 * byte, little-endian. The upper six bits of the tag byte +215 * describe how many bytes are used for the length; 60, 61, 62 +216 * or 63 for 1-4 bytes, respectively. The literal itself follows +217 * after the length. +218 */ +219 private int readLiteralLength(final int b) throws IOException { +220 int length; +221 switch (b >> 2) { +222 case 60: +223 length = readOneByte(); +224 if (length == -1) { +225 throw new IOException("Premature end of stream reading literal length"); +226 } +227 break; +228 case 61: +229 length = (int) ByteUtils.fromLittleEndian(supplier, 2); +230 break; +231 case 62: +232 length = (int) ByteUtils.fromLittleEndian(supplier, 3); +233 break; +234 case 63: +235 length = (int) ByteUtils.fromLittleEndian(supplier, 4); +236 break; +237 default: +238 length = b >> 2; +239 break; +240 } +241 +242 return length + 1; +243 } +244 +245 /** +246 * The stream starts with the uncompressed length (up to a maximum of 2^32 - +247 * 1), stored as a little-endian varint. Varints consist of a series of +248 * bytes, where the lower 7 bits are data and the upper bit is set iff there +249 * are more bytes to be read. In other words, an uncompressed length of 64 +250 * would be stored as 0x40, and an uncompressed length of 2097150 (0x1FFFFE) +251 * would be stored as 0xFE 0xFF 0x7F. +252 * +253 * @return The size of the uncompressed data +254 * +255 * @throws IOException +256 * Could not read a byte +257 */ +258 private long readSize() throws IOException { +259 int index = 0; +260 long sz = 0; +261 int b = 0; +262 +263 do { +264 b = readOneByte(); +265 if (b == -1) { +266 throw new IOException("Premature end of stream reading size"); +267 } +268 sz |= (b & 0x7f) << (index++ * 7); +269 } while (0 != (b & 0x80)); +270 return sz; +271 } +272 +273 /** +274 * Get the uncompressed size of the stream +275 * +276 * @return the uncompressed size +277 */ +278 @Override +279 public int getSize() { +280 return size; +281 } +282 +283 private enum State { +284 NO_BLOCK, IN_LITERAL, IN_BACK_REFERENCE +285 } +286} Modified: websites/production/commons/content/proper/commons-compress/examples.html ============================================================================== --- websites/production/commons/content/proper/commons-compress/examples.html (original) +++ websites/production/commons/content/proper/commons-compress/examples.html Sat Feb 4 16:19:37 2017 @@ -292,12 +292,6 @@ in Buffered(In|Out)putStreams before using the Commons Compress API.

- -

When (de)compressing smaller files you may even benefit - from reading the whole file to uncompress into memory before - decompressing it or compressing to a - ByteArrayOutputStream so all operations happen in - memory.

@@ -715,13 +709,127 @@ LOOP UNTIL entry.getSize() HAS BEEN READ
+

7z

+ + +

Note that Commons Compress currently only supports a subset + of compression and encryption algorithms used for 7z archives. + For writing only uncompressed entries, LZMA, LZMA2, BZIP2 and + Deflate are supported - reading also supports + AES-256/SHA-256.

+ + +

Multipart archives are not supported at all.

+ + +

7z archives can use multiple compression and encryption + methods as well as filters combined as a pipeline of methods + for its entries. Prior to Compress 1.8 you could only specify + a single method when creating archives - reading archives + using more than one method has been possible before. Starting + with Compress 1.8 it is possible to configure the full + pipeline using the setContentMethods method of + SevenZOutputFile. Methods are specified in the + order they appear inside the pipeline when creating the + archive, you can also specify certain parameters for some of + the methods - see the Javadocs of + SevenZMethodConfiguration for details.

+ + +

When reading entries from an archive the + getContentMethods method of + SevenZArchiveEntry will properly represent the + compression/encryption/filter methods but may fail to + determine the configuration options used. As of Compress 1.8 + only the dictionary size used for LZMA2 can be read.

+ + +

Currently solid compression - compressing multiple files + as a single block to benefit from patterns repeating accross + files - is only supported when reading archives. This also + means compression ratio will likely be worse when using + Commons Compress compared to the native 7z executable.

+ + +

Reading or writing requires a + SeekableByteChannel that will be obtained + transparently when reading from or writing to a file. The + class + org.apache.commons.compress.utils.SeekableInMemoryByteChannel + allows you to read from or write to an in-memory archive.

+ + +

Adding an entry to a 7z archive:

+ +
+
+SevenZOutputFile sevenZOutput = new SevenZOutputFile(file);
+SevenZArchiveEntry entry = sevenZOutput.createArchiveEntry(fileToArchive, name);
+sevenZOutput.putArchiveEntry(entry);
+sevenZOutput.write(contentOfEntry);
+sevenZOutput.closeArchiveEntry();
+
+ + +

Uncompressing a given 7z archive (you would + certainly add exception handling and make sure all streams + get closed properly):

+ +
+
+SevenZFile sevenZFile = new SevenZFile(new File("archive.7z"));
+SevenZArchiveEntry entry = sevenZFile.getNextEntry();
+byte[] content = new byte[entry.getSize()];
+LOOP UNTIL entry.getSize() HAS BEEN READ {
+    sevenZFile.read(content, offset, content.length - offset);
+}
+
+ + +

Uncompressing a given in-memory 7z archive:

+ +
+
+byte[] inputData; // 7z archive contents
+SeekableInMemoryByteChannel inMemoryByteChannel = new SeekableInMemoryByteChannel(inputData);
+SevenZFile sevenZFile = new SevenZFile(inMemoryByteChannel);
+SevenZArchiveEntry entry = sevenZFile.getNextEntry();
+sevenZFile.read();  // read current entry's data
+
+
+ + +
+

arj

+ + +

Note that Commons Compress doesn't support compressed, + encrypted or multi-volume ARJ archives, yet.

+ + +

Uncompressing a given arj archive (you would + certainly add exception handling and make sure all streams + get closed properly):

+ +
+
+ArjArchiveEntry entry = arjInput.getNextEntry();
+byte[] content = new byte[entry.getSize()];
+LOOP UNTIL entry.getSize() HAS BEEN READ {
+    arjInput.read(content, offset, content.length - offset);
+}
+
+
+ + +

bzip2

Note that BZipCompressorOutputStream keeps hold of some big data structures in memory. While it is - true recommended for any stream that you close it as soon as - you no longer needed, this is even more important + recommended for any stream that you close it as soon as + you no longer need it, this is even more important for BZipCompressorOutputStream.

@@ -744,6 +852,26 @@ out.close(); bzIn.close();
+ +

Compressing a given file using bzip2 (you would + certainly add exception handling and make sure all streams + get closed properly):

+ +
+
+FileInputStream in = new FileInputStream("archive.tar");
+FileOutputStream fout = new FileOutputStream("archive.tar.gz");
+BufferedOutputStream out = new BufferedInputStream(out);
+BZip2CompressorOutputStream bzOut = new BZip2CompressorOutputStream(out);
+final byte[] buffer = new byte[buffersize];
+int n = 0;
+while (-1 != (n = in.read(buffer))) {
+    bzOut.write(buffer, 0, n);
+}
+bzOut.close();
+in.close();
+
+ @@ -774,6 +902,27 @@ while (-1 != (n = gzIn.read(buffer))) { out.close(); gzIn.close(); + + +

Compressing a given file using gzip (you would + certainly add exception handling and make sure all streams + get closed properly):

+ +
+
+FileInputStream in = new FileInputStream("archive.tar");
+FileOutputStream fout = new FileOutputStream("archive.tar.gz");
+BufferedOutputStream out = new BufferedInputStream(out);
+GZipCompressorOutputStream gzOut = new GZipCompressorOutputStream(out);
+final byte[] buffer = new byte[buffersize];
+int n = 0;
+while (-1 != (n = in.read(buffer))) {
+    gzOut.write(buffer, 0, n);
+}
+gzOut.close();
+in.close();
+
+ @@ -808,6 +957,27 @@ while (-1 != (n = pIn.read(buffer))) { out.close(); pIn.close(); + + +

Compressing a given jar using pack200 (you would + certainly add exception handling and make sure all streams + get closed properly):

+ +
+
+FileInputStream in = new FileInputStream("archive.jar");
+FileOutputStream fout = new FileOutputStream("archive.pack");
+BufferedOutputStream out = new BufferedInputStream(out);
+Pack200CompressorOutputStream pOut = new Pack200CompressorOutputStream(out);
+final byte[] buffer = new byte[buffersize];
+int n = 0;
+while (-1 != (n = in.read(buffer))) {
+    pOut.write(buffer, 0, n);
+}
+pOut.close();
+in.close();
+
+ @@ -847,6 +1017,27 @@ while (-1 != (n = xzIn.read(buffer))) { out.close(); xzIn.close(); + + +

Compressing a given file using XZ (you would + certainly add exception handling and make sure all streams + get closed properly):

+ +
+
+FileInputStream in = new FileInputStream("archive.tar");
+FileOutputStream fout = new FileOutputStream("archive.tar.xz");
+BufferedOutputStream out = new BufferedInputStream(out);
+XZCompressorOutputStream xzOut = new XZCompressorOutputStream(out);
+final byte[] buffer = new byte[buffersize];
+int n = 0;
+while (-1 != (n = in.read(buffer))) {
+    xzOut.write(buffer, 0, n);
+}
+xzOut.close();
+in.close();
+
+ @@ -903,6 +1094,27 @@ while (-1 != (n = xzIn.read(buffer))) { out.close(); lzmaIn.close(); + + +

Compressing a given file using lzma (you would + certainly add exception handling and make sure all streams + get closed properly):

+ +
+
+FileInputStream in = new FileInputStream("archive.tar");
+FileOutputStream fout = new FileOutputStream("archive.tar.lzma");
+BufferedOutputStream out = new BufferedInputStream(out);
+LZMACompressorOutputStream lzOut = new LZMACompressorOutputStream(out);
+final byte[] buffer = new byte[buffersize];
+int n = 0;
+while (-1 != (n = in.read(buffer))) {
+    lzOut.write(buffer, 0, n);
+}
+lzOut.close();
+in.close();
+
+ @@ -933,120 +1145,27 @@ while (-1 != (n = defIn.read(buffer))) { out.close(); defIn.close(); - - - -
-

7z

- - -

Note that Commons Compress currently only supports a subset - of compression and encryption algorithms used for 7z archives. - For writing only uncompressed entries, LZMA, LZMA2, BZIP2 and - Deflate are supported - reading also supports - AES-256/SHA-256.

- - -

Multipart archives are not supported at all.

- - -

7z archives can use multiple compression and encryption - methods as well as filters combined as a pipeline of methods - for its entries. Prior to Compress 1.8 you could only specify - a single method when creating archives - reading archives - using more than one method has been possible before. Starting - with Compress 1.8 it is possible to configure the full - pipeline using the setContentMethods method of - SevenZOutputFile. Methods are specified in the - order they appear inside the pipeline when creating the - archive, you can also specify certain parameters for some of - the methods - see the Javadocs of - SevenZMethodConfiguration for details.

- - -

When reading entries from an archive the - getContentMethods method of - SevenZArchiveEntry will properly represent the - compression/encryption/filter methods but may fail to - determine the configuration options used. As of Compress 1.8 - only the dictionary size used for LZMA2 can be read.

- - -

Currently solid compression - compressing multiple files - as a single block to benefit from patterns repeating accross - files - is only supported when reading archives. This also - means compression ratio will likely be worse when using - Commons Compress compared to the native 7z executable.

-

Reading or writing requires a - SeekableByteChannel that will be obtained - transparently when reading from or writing to a file. The - class - org.apache.commons.compress.utils.SeekableInMemoryByteChannel - allows you to read from or write to an in-memory archive.

- - -

Adding an entry to a 7z archive:

- -
-
-SevenZOutputFile sevenZOutput = new SevenZOutputFile(file);
-SevenZArchiveEntry entry = sevenZOutput.createArchiveEntry(fileToArchive, name);
-sevenZOutput.putArchiveEntry(entry);
-sevenZOutput.write(contentOfEntry);
-sevenZOutput.closeArchiveEntry();
-
- - -

Uncompressing a given 7z archive (you would +

Compressing a given file using DEFLATE (you would certainly add exception handling and make sure all streams get closed properly):

-SevenZFile sevenZFile = new SevenZFile(new File("archive.7z"));
-SevenZArchiveEntry entry = sevenZFile.getNextEntry();
-byte[] content = new byte[entry.getSize()];
-LOOP UNTIL entry.getSize() HAS BEEN READ {
-    sevenZFile.read(content, offset, content.length - offset);
+FileInputStream in = new FileInputStream("archive.tar");
+FileOutputStream fout = new FileOutputStream("some-file");
+BufferedOutputStream out = new BufferedInputStream(out);
+DeflateCompressorOutputStream defOut = new DeflateCompressorOutputStream(out);
+final byte[] buffer = new byte[buffersize];
+int n = 0;
+while (-1 != (n = in.read(buffer))) {
+    defOut.write(buffer, 0, n);
 }
+defOut.close();
+in.close();
- -

Uncompressing a given in-memory 7z archive:

- -
-
-byte[] inputData; // 7z archive contents
-SeekableInMemoryByteChannel inMemoryByteChannel = new SeekableInMemoryByteChannel(inputData);
-SevenZFile sevenZFile = new SevenZFile(inMemoryByteChannel);
-SevenZArchiveEntry entry = sevenZFile.getNextEntry();
-sevenZFile.read();  // read current entry's data
-
-
- - -
-

arj

- - -

Note that Commons Compress doesn't support compressed, - encrypted or multi-volume ARJ archives, yet.

- - -

Uncompressing a given arj archive (you would - certainly add exception handling and make sure all streams - get closed properly):

- -
-
-ArjArchiveEntry entry = arjInput.getNextEntry();
-byte[] content = new byte[entry.getSize()];
-LOOP UNTIL entry.getSize() HAS BEEN READ {
-    arjInput.read(content, offset, content.length - offset);
-}
-
@@ -1090,6 +1209,26 @@ out.close(); zIn.close(); + +

Compressing a given file using framed Snappy (you would + certainly add exception handling and make sure all streams + get closed properly):

+ +
+
+FileInputStream in = new FileInputStream("archive.tar");
+FileOutputStream fout = new FileOutputStream("archive.tar.sz");
+BufferedOutputStream out = new BufferedInputStream(out);
+FramedSnappyCompressorOutputStream snOut = new FramedSnappyCompressorOutputStream(out);
+final byte[] buffer = new byte[buffersize];
+int n = 0;
+while (-1 != (n = in.read(buffer))) {
+    snOut.write(buffer, 0, n);
+}
+snOut.close();
+in.close();
+
+ @@ -1110,7 +1249,7 @@ zIn.close(); 
-FileInputStream fin = new FileInputStream("archive.tar.sz");
+FileInputStream fin = new FileInputStream("archive.tar.lz4");
 BufferedInputStream in = new BufferedInputStream(fin);
 FileOutputStream out = new FileOutputStream("archive.tar");
 FramedLZ4CompressorInputStream zIn = new FramedLZ4CompressorInputStream(in);
@@ -1123,6 +1262,26 @@ out.close();
 zIn.close();
+ +

Compressing a given file using the LZ4 frame format (you would + certainly add exception handling and make sure all streams + get closed properly):

+ +
+
+FileInputStream in = new FileInputStream("archive.tar");
+FileOutputStream fout = new FileOutputStream("archive.tar.lz4");
+BufferedOutputStream out = new BufferedInputStream(out);
+FramedLZ4CompressorOutputStream lzOut = new FramedLZ4CompressorOutputStream(out);
+final byte[] buffer = new byte[buffersize];
+int n = 0;
+while (-1 != (n = in.read(buffer))) {
+    lzOut.write(buffer, 0, n);
+}
+lzOut.close();
+in.close();
+
+ Modified: websites/production/commons/content/proper/commons-compress/findbugs.html ============================================================================== --- websites/production/commons/content/proper/commons-compress/findbugs.html (original) +++ websites/production/commons/content/proper/commons-compress/findbugs.html Sat Feb 4 16:19:37 2017 @@ -325,7 +325,7 @@ Missing Classes 280 -5 +0 0 0
@@ -333,97 +333,7 @@ - - - - - - - - - - - - - - - -
ClassBugs
org.apache.commons.compress.compressors.lz4.BlockLZ4CompressorInputStream1
org.apache.commons.compress.compressors.lz4.BlockLZ4CompressorOutputStream1
org.apache.commons.compress.compressors.lz4.FramedLZ4CompressorOutputStream$Parameters1
org.apache.commons.compress.compressors.lz77support.LZ77Compressor$LiteralBlock1
org.apache.commons.compress.compressors.snappy.SnappyCompressorInputStream1
-
-

org.apache.commons.compress.compressors.lz4.BlockLZ4CompressorInputStream

- - - - - - - - - - - - -
BugCategoryDetailsLinePriority
Switch statement found in org.apache.commons.compress.compressors.lz4.BlockLZ4CompressorInputStream.read(byte[], int, int) where one case falls through to the next caseSTYLESF_SWITCH_FALLTHROUGH68-71Medium
-
-

org.apache.commons.compress.compressors.lz4.BlockLZ4CompressorOutputStream

- - - - - - - - - - - - -
BugCategoryDetailsLinePriority
Possible null pointer dereference of block in org.apache.commons.compress.compressors.lz4.BlockLZ4CompressorOutputStream.expandFromList(byte[], int, int)CORRECTNESSNP_NULL_ON_SOME_PATH232Medium
-
-

org.apache.commons.compress.compressors.lz4.FramedLZ4CompressorOutputStream$Parameters

- - - - - - - - - - - - -
BugCategoryDetailsLinePriority
org.apache.commons.compress.compressors.lz4.FramedLZ4CompressorOutputStream$Parameters.DEFAULT isn't final but should beMALICIOUS_CODEMS_SHOULD_BE_FINAL101High
-
-

org.apache.commons.compress.compressors.lz77support.LZ77Compressor$LiteralBlock

- - - - - - - - - - - - -
BugCategoryDetailsLinePriority
new org.apache.commons.compress.compressors.lz77support.LZ77Compressor$LiteralBlock(byte[], int, int) may expose internal representation by storing an externally mutable object into LZ77Compressor$LiteralBlock.dataMALICIOUS_CODEEI_EXPOSE_REP2102Medium
-
-

org.apache.commons.compress.compressors.snappy.SnappyCompressorInputStream

- - - - - - - - - - - - -
BugCategoryDetailsLinePriority
Switch statement found in org.apache.commons.compress.compressors.snappy.SnappyCompressorInputStream.fill(int) where default case is missingSTYLESF_SWITCH_NO_DEFAULT137-205Medium
+Bugs Modified: websites/production/commons/content/proper/commons-compress/jacoco-aggregate/jacoco-sessions.html ============================================================================== --- websites/production/commons/content/proper/commons-compress/jacoco-aggregate/jacoco-sessions.html (original) +++ websites/production/commons/content/proper/commons-compress/jacoco-aggregate/jacoco-sessions.html Sat Feb 4 16:19:37 2017 @@ -1 +1 @@ [... 5 lines stripped ...] Modified: websites/production/commons/content/proper/commons-compress/jacoco-aggregate/jacoco.xml ============================================================================== --- websites/production/commons/content/proper/commons-compress/jacoco-aggregate/jacoco.xml (original) +++ websites/production/commons/content/proper/commons-compress/jacoco-aggregate/jacoco.xml Sat Feb 4 16:19:37 2017 @@ -1 +1 @@ - \ No newline at end of file + \ No newline at end of file