tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacob Kjome <h...@visi.com>
Subject RE: GZIP filter problem....
Date Tue, 07 Jan 2003 21:45:12 GMT
Hi Charlie,

See comments below...

Quoting "Cox, Charlie" <ccox@cincom.com>:

> just noticed this. Two more lines reversed. Set the content lenght *before*
> you write to the response. IE can be really anal if it doesn't know the
> content length.
> 
> you have:
> 
> out.write(compressed.toByteArray());
> response.setContentLength(compressed.size());
> 
> it is possible that the default servlet set the content-length for your
> static file and that could be why it worked and not your servlet.
> 

Yes, I know I had that in the original post.  I had been moving stuff around so
much I missed that originally.  As of the last email I sent, I switched the
calls around so it is now:

response.setContentLength(compressed.size());
out.write(compressed.toByteArray());

Sorry for the confusion there.  And it is certainly possible, and quite likely,
that the default servlet set the contentLength().  However, in all cases where I
am GZIP'ing content, I reset the contentLength() as above.  It works with the
static html files but not with content served by .jsp or my servlets.

> overall:
> 
> httpResponse.setHeader("Vary", "Accept-Encoding");
> httpResponse.setHeader("Content-Encoding", "gzip");
> OutputStream out = response.getOutputStream();
> ...
> response.setContentLength(compressed.size());
> out.write(compressed.toByteArray());
> out.close();
> 

As far as setting the header before calling response.getOuputStream(), I tried
that and it made no difference.  I got the same behavior.  The fact that I can
grab the output stream and then set the gzip header and make it work with at
least some content (static html files) proves, I think, that one can set headers
at any point as long as it is before the out.write().

> Another thing from above code. Is 'httpResponse' and 'response' the same
> object? I am assuming that one is the 'ServletResponse' and the other casted
> as the 'HttpServletResponse'. I would use the same reference in both places.
> You could be skipping some functionality in the HttpServletResponse since it
> has your headers.
> 

Yeah, the response is just cast to httpResponse.  It is all working against the
same object.  I can try using the httpResponse for everything, but it really
shouldn't make a difference...and Marty Hall's example, along with all the other
 examples on the web that I've seen, do exactly the same thing I am doing.  I
really don't think this is the problem.

I'll test it and report back if it does make a difference.  In the meantime, if
you can think of anything else, let me know.

For the record, here is the content of the entire doFilter() method...


public void doFilter(final ServletRequest request,
                         final ServletResponse response,
                         FilterChain chain)
                                throws IOException, ServletException {

    HttpServletResponse httpResponse = (HttpServletResponse) response;
    HttpServletRequest httpRequest = (HttpServletRequest)request;
    GenericResponseWrapper wrapper = new GenericResponseWrapper(httpResponse);
    chain.doFilter(request, wrapper);
    httpResponse.setHeader("Vary", "Accept-Encoding");
    OutputStream out = response.getOutputStream();

    byte[] origBytes = wrapper.getData();
    if (!isCached(wrapper) && !isIncluded(request) && accepts(httpRequest,
"gzip")) {
        httpResponse.setHeader("Content-Encoding", "gzip");
        ByteArrayOutputStream compressed = new ByteArrayOutputStream();
        GZIPOutputStream gzout = new GZIPOutputStream(compressed);
        gzout.write(origBytes);
        gzout.finish();
        gzout.close();

        if (logger.isDebugEnabled()) {
            logger.debug("compressed data...");
            logger.debug(compressed);
            ByteArrayInputStream bais = new
ByteArrayInputStream(compressed.toByteArray());
            GZIPInputStream gzin = new GZIPInputStream(bais);
            byte[] buffer = new byte[1024];
            int n, i = 0, m = buffer.length;
            while ((n = gzin.read (buffer, i, m - i)) >= 0) {
                i += n;
                if (i >= m) {
                    byte[] newBuffer = new byte[m *= 2];
                    System.arraycopy (buffer, 0, newBuffer, 0, i);
                    buffer = newBuffer;
                }
            }
            byte[] result = new byte[i];
            System.arraycopy (buffer, 0, result, 0, i);
            ByteArrayOutputStream decompressed = new ByteArrayOutputStream();
            DataOutputStream daos = new DataOutputStream(decompressed);
            daos.write(result);
            daos.flush();
            daos.close();
            logger.debug("decompressed data...");
            logger.debug(decompressed);
        }

        response.setContentLength(compressed.size());
        out.write(compressed.toByteArray());

        if (logger.isDebugEnabled()) {
            logger.debug("GZIP filtered data totals...");
            logger.debug("Orig data size: "+origBytes.length+" bytes");
            logger.debug("GZIP data size: "+compressed.size()+" bytes");
        }
    }
     else {
        out.write(origBytes);

        if (logger.isDebugEnabled()) {
            logger.debug("Bypassed GZIP filtering...");
            logger.debug("Orig data size: "+origBytes.length+" bytes");
        }
    }
    out.flush();
    response.flushBuffer();
    out.close();
}


thanks,

Jake

> Charlie
> 
> > -----Original Message-----
> > From: Jacob Kjome [mailto:hoju@visi.com]
> > Sent: Tuesday, January 07, 2003 2:55 PM
> > To: Tomcat Users List
> > Subject: RE: GZIP filter problem....
> >
> >
> >
> > Hi Charlie,
> >
> > Thanks for the tip.  Actually, that seems to make no
> > difference.  I finally
> > decided to test it with a plain HTML page.  It successfully
> > GZIP'ed an html
> > document with an original data size of 636 bytes down to 369
> > bytes and the
> > output didn't get garbled on the browser side.  This was with
> > specifying the
> > compressed header after I did response.getOuputStream() but
> > before I actually
> > wrote the data.  So, that validates that my GZIP filter
> > generally works correctly.
> >
> > The curious thing is that I also tried it with a simple .jsp
> > page.  I got the
> > same sort of garbled data as I got when sending xml data from
> > a servlet.  In all
> > cases, I can see the decompressed data just fine in debugging.
> >
> > So, this leads me to believe there is either a bug in Tomcat
> > or there is some
> > other header or something that I need to set.  GZIP'ing
> > static files, such as
> > .html files, works fine.  GZIP'ing data coming from a servlet
> > or .jsp doesn't work.
> >
> > This narrows it down.  Can you think of anything else which
> > might be the issue
> > here?  Should I report a bug against Tomcat?  I hate to do
> > that until I really
> > understand the problem, though.
> >
> > Thanks for continued your help!
> >
> > Jake
> >
> > Quoting "Cox, Charlie" <ccox@cincom.com>:
> >
> > > try setting the encoding before you request the output
> > stream. You may be
> > > losing the header by requesting the output stream first.
> > >
> > > 	httpResponse.setHeader("Content-Encoding", "gzip");
> > > 	OutputStream out = response.getOutputStream();
> > >
> > > I know you can't set headers after writing to the output
> > stream, but I'm not
> > > sure if you can after getting a reference to the
> > outputstream. IIRC, the
> > > getOutputStream() determines whether you have written
> > anything or not(i.e.
> > > try getWriter() immediately after getOutputStream() and it
> > will throw an
> > > error)
> > >
> > > Charlie
> > >
> > > > -----Original Message-----
> > > > From: Jacob Kjome [mailto:hoju@visi.com]
> > > > Sent: Tuesday, January 07, 2003 9:19 AM
> > > > To: Tomcat Users List
> > > > Subject: RE: GZIP filter problem....
> > > >
> > > >
> > > >
> > > > Hi Charlie,
> > > >
> > > > I looked at it a little bit, but, to be honest, that one is less
> > > > straightforward (kind of messy, actually) and I'm not even
> > > > sure it is doing
> > > > any compression as it doesn't provide any logging stating the
> > > > difference
> > > > between the original size of the response and the compressed
> > > > size (although
> > > > it logs lots of other stuff).  I'll continue to investigate,
> > > > however, I
> > > > guess my point is that I've seen a good number of examples of
> > > > how to do the
> > > > GZIP compression and the example I am following pretty much
> > > > does exactly
> > > > what all of them recommend.  There is one from "More Servlets
> > > > and Java
> > > > Server Pages" by Marty Hall that also doesn't work under
> > Tomcat-4.1.18
> > > > (
> > > > http://archive.moreservlets.com/Filter-Code/filters/WEB-INF/cl
> > > asses/moreservlets/filters/CompressionFilter.java
> > > > ).  I have a really hard time believing that all these people
> > > > are just
> > > > plain wrong about how they did this.  The code looks valid
> > > > based on all
> > > > examples I've seen, and I've proven that the GZIP compression
> > > > was done
> > > > properly because I was able to decompress and print that to
> > > > logging.  I
> > > > just can't get it to the browsers (IE5.5/6.0 and Mozilla) to
> > > > understand the
> > > > output.
> > > >
> > > > So, apart from looking at the example in Tomcat which,
> > > > presumably, works
> > > > (and I will continue to study it),  is there anything in my
> > > > code that is
> > > > just obviously incorrect?  If not, why doesn't it work?
> > > >
> > > > Jake
> > > >
> > > > At 07:03 AM 1/7/2003 -0500, you wrote:
> > > > >did you look at CompressionFilter.java that is part of
> > the examples
> > > > >distributed with Tomcat?
> > > > >
> > > > >Charlie
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Jacob Kjome [mailto:hoju@visi.com]
> > > > > > Sent: Monday, January 06, 2003 8:22 PM
> > > > > > To: Tomcat Users List
> > > > > > Subject: GZIP filter problem....
> > > > > >
> > > > > >
> > > > > >
> > > > > > I'm trying to use a GZIP servlet filter under Tomcat-4.1.18.
> > > > > > I am basing this filter on
> > > > > > an existing example at Orion (
> > > > > > http://www.orionserver.com/tutorials/filters/5.html )
> > > > > >
> > > > > > It GZIPs fine and, in my debugging, I can decompress the
> > > > data back to
> > > > > > what it was originally (more on that below).  The problem
> > > > > > is, all I get when I send the data out to the browser is
> > > > > > non-decompressed garbled data.  That is, the browser doesn't
> > > > > > seem to understand that it is
> > > > > > gzip'ed stream and doesn't decode it.  Here is the relevant
> > > > > > code.  Can anyone see what the
> > > > > > problem might be?  Note that in the no compression case where
> > > > > > I simply call
> > > > > > dofilter(res, wrapper.getData()) it works just fine,
> > so it isn't a
> > > > > > problem with the filter in general....
> > > > > >
> > > > > >
> > > > > > ...
> > > > > > ...
> > > > > > ...
> > > > > > httpResponse.setHeader("Vary", "Accept-Encoding");
> > > > > > OutputStream out = response.getOutputStream();
> > > > > > httpResponse.setHeader("Content-Encoding", "gzip");
> > > > > > ByteArrayOutputStream compressed = new
> > ByteArrayOutputStream();
> > > > > > GZIPOutputStream gzout = new GZIPOutputStream(compressed);
> > > > > > gzout.write(wrapper.getData());
> > > > > > gzout.finish();
> > > > > > gzout.close();
> > > > > >
> > > > > > if (logger.isDebugEnabled()) {
> > > > > >     logger.debug("compressed data...");
> > > > > >     logger.debug(compressed);
> > > > > >
> > > > > >     ByteArrayInputStream bais = new
> > > > > > ByteArrayInputStream(compressed.toByteArray());
> > > > > >     GZIPInputStream gzin = new GZIPInputStream(bais);
> > > > > >     byte[] buffer = new byte[1024];
> > > > > >     int n, i = 0, m = buffer.length;
> > > > > >     while ((n = gzin.read (buffer, i, m - i)) >= 0) {
> > > > > >         i += n;
> > > > > >         if (i >= m) {
> > > > > >             byte[] newBuffer = new byte[m *= 2];
> > > > > >             System.arraycopy (buffer, 0, newBuffer, 0, i);
> > > > > >             buffer = newBuffer;
> > > > > >         }
> > > > > >     }
> > > > > >     byte[] result = new byte[i];
> > > > > >     System.arraycopy (buffer, 0, result, 0, i);
> > > > > >     ByteArrayOutputStream decompressed = new
> > > > ByteArrayOutputStream();
> > > > > >     DataOutputStream daos = new
> > DataOutputStream(decompressed);
> > > > > >     daos.write(result);
> > > > > >     daos.flush();
> > > > > >     daos.close();
> > > > > >     logger.debug("decompressed data...");
> > > > > >     logger.debug(decompressed);
> > > > > > }
> > > > > >
> > > > > > out.write(compressed.toByteArray());
> > > > > > response.setContentLength(compressed.size());
> > > > > > if (logger.isDebugEnabled()) logger.debug("Wrote filter
> > > > > > compressed data: "+compressed.size()+" bytes");
> > > > > > out.flush();
> > > > > > response.flushBuffer();
> > > > > > out.close();
> > > > > > ...
> > > > > > ...
> > > > > > ...
> > > > > >
> > > > > >
> > > > > > Here is an example of the debugging output so you can see
> > > > that I can
> > > > > > compress and decompress the data without issue...
> > > > > >
> > > > > > compressed data...
> > > > > > <       ³±¯ÈÍQ(K-*ÎÌϳU2Ô3P²·ã²qÉÏMÌÌ+¶f2òsSm*'ós
> > > > > > ó**ôÑÄ
> > > > > > Ró
> > > > > > rRAÂú0? ºÓ'sX
> > > > > > decompressed data...
> > > > > > <?xml version="1.0"?>
> > > > > > <Root><MyElement name="someName"/><MyElement
> > > > > > name="someOtherName"/></Root>
> > > > > >
> > > > > > Note that the contentType was set by the servlet as
> > > > "text/xml" and was
> > > > > > not reset by the filter.
> > > > > >
> > > > > > Any ideas?  It wouldn't be a bug in Tomcat, would it?  I
> > > > just don't
> > > > > > see anything wrong with the code???
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jake
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > >  Jacob                          mailto:hoju@visi.com
> 
> --
> To unsubscribe, e-mail:   <mailto:tomcat-user-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: <mailto:tomcat-user-help@jakarta.apache.org>

--
To unsubscribe, e-mail:   <mailto:tomcat-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:tomcat-user-help@jakarta.apache.org>


Mime
View raw message