tomcat-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject DO NOT REPLY [Bug 52121] corrupt gzip output when compression=on in server.xml
Date Fri, 04 Nov 2011 12:08:34 GMT
https://issues.apache.org/bugzilla/show_bug.cgi?id=52121

--- Comment #6 from Konstantin Kolinko <knst.kolinko@gmail.com> 2011-11-04 12:08:34
UTC ---
Several additional notes

A link to the thread by Christopher Taylor mentioned by OP:
http://tomcat.markmail.org/thread/v56xpi5uz4end7cd

Note, that the Christopher Taylor's thread contains an attachment but only
containing the corrupted gzip stream.

1. I can observe the CRC error with 3-rd party tool, 7-zip.

It cannot decompress the file in Christopher Taylor's message, nether it can
the one produced by the test case for this issue before I committed my changes.
It reports CRC error.

Christopher Taylor's file was produced by Tomcat 7.0.14.

Thus not only Java report that the stream is broken.

2. According to the sources of deflate.c of zlib library that can be found in
many placed, changing compression level which is performed by "deflateParams"
call indeed causes deflate(strm, Z_PARTIAL_FLUSH) flushing the buffer.

That is in some old version of the code. In more recent ones it is just a
deflate(strm, Z_BLOCK).


3. Looking at the content of the streams produced by compression. Both the
stream in Christopher Taylor's message and the one produced by the JUnit test
case for this bug have uncompressed data near the end of the stream.

That is "</id><version>"... or "<div "... respectively - about 200 bytes of
plain text.

It just means that the flush in deflateParams() call failed to flush the whole
stream.

That happens when the output buffer used by deflate.c (that is configured by
'size' parameter in GZIPOutputStream constructor) is too small. There is a loop
that deals with this in DeflaterOutputStream.deflate(). Because the effective
compression level at that time is Deflater.NO_COMPRESSION the data is flushed
uncompressed.

The following follows:
1) If I increase size of the buffer by the following change in
FlushableGZIPOutputStream constructor, the problem reported here disappears:

-        super(os);
+        super(os, 64*1024);


4. The Java bugs 4255743 and 4813885 that are mentioned in the Javadoc for the
FlushableGZIPOutputStream class are both fixed in Java 7.

There is an additional boolean argument to GZIPOutputStream and
DeflaterOutputStream constructors in Java 7, "syncFlush". If it is set to true,
then calling mere flush() results in flushing the compressor.

5. Some documentation mentions that there is a trick in gzip when partial flush
is performed. It sends either one block or two blocks.

http://www.bolet.org/~pornin/deflate-flush.html

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


Mime
View raw message