hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ingo Meyer" <dj-sl...@gmx.net>
Subject AW: Does (or will?) HttpClient handle compressed content ?
Date Sun, 11 Dec 2005 10:51:26 GMT
Hi all,

I agree with the point that encoding should be a part of HttpClient. 
But for all who don't want to wait until it's implemented here my solution:

First, add this to your HostConfiguration:

		final Collection<Header> c = new ArrayList<Header> ();
		c.add (new Header ("Accept",
"text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q
=0.8,image/png,*/*;q=0.5"));
		c.add (new Header ("Accept-Language",
"de-de,de;q=0.8,en-us;q=0.5,en;q=0.3"));
	-->	c.add (new Header ("Accept-Encoding", "gzip, deflate"));
		c.add (new Header ("Accept-Charset",
"ISO-8859-1,utf-8;q=0.7,*;q=0.7"));
		hostConfiguration.getParams ().setParameter
("http.default-headers", c);


Secondly, after the transfer is proceeded call the following method with the
response bytes:

	protected byte[] decodeResponse (final byte[] responseBytes) throws
Exception
	{
		LOG.debug ("request enter decodeResponse()");

		// Getting Content-Encoding
		final String sContentEncoding = getResponseHeader
("Content-Encoding");

		// If available...
		if (sContentEncoding != null)
		{
			InputStream encoded = null;

			// GZIP
			if (sContentEncoding.equals ("gzip"))
			{
				LOG.debug ("method received gzip-encoded
content");
				
				encoded = new GZIPInputStream (new
ByteArrayInputStream (responseBytes));
			}
			// DEFLATE
			else if (sContentEncoding.equals ("deflate"))
			{
				LOG.debug ("method received deflate-encoded
content");
				
				encoded = new InflaterInputStream (new
ByteArrayInputStream (responseBytes));
			}
			
			final ByteArrayOutputStream decoded = new
ByteArrayOutputStream ();
			final byte buffer[] = new byte[1024];

			for (int length; (length = encoded.read (buffer, 0,
1024)) != -1;)
			{
				decoded.write (buffer, 0, length);
			}

			// closing
			encoded.close ();
			decoded.close ();

			return decoded.toByteArray ();
		}
		
		LOG.debug ("method received uncoded content");
			
		return responseBytes;
	}


If anyone want's to use it one thing still not work. The deflate part!
I have no idea about it, all works fine with gzip. So to get it running
better
change to just  gzip:

	-->	c.add (new Header ("Accept-Encoding", "gzip"));

So any help with the deflate part will be recommened.

Thanks 

Ingo Meyer



> -----Urspr√ľngliche Nachricht-----
> Von: Roland Weber [mailto:http-async@dubioso.net] 
> Gesendet: Samstag, 10. Dezember 2005 21:29
> An: HttpClient User Discussion
> Betreff: Re: Does (or will?) HttpClient handle compressed content ?
> 
> RFC 2616 distinguishes between content codings (section 3.5) 
> and transport codings (section 3.6). Content coding is out of 
> scope, as Oleg mentioned. HttpClient will not automatically 
> decompress for example a gzipped tar into a plain tar archive.
> 
> Transport coding is in scope for HttpClient/HttpComponents, 
> as we already support "chunked". We can consider adding 
> support for other transport codings, such as "gzip". The 
> pluggable filter architecture of HttpComponents will at least 
> allow you to add your own support for transport codings. I 
> guess that seamless interoperability of the HttpComponents 
> "chunked" implementation with additional custom transport 
> encodings will be made possible on request.
> I don't think we'll find time to add new transport codings 
> out of the box in HttpClient nor HttpComponents. Which 
> implies that the interaction of chunked with other transport 
> codings will not be tested until somebody tries to do it. It 
> is a tricky thing, since the response filter chain will have 
> to depend on the value of the Transfer-Encoding response 
> header. Given the complexity of the task, I see this item 
> dangling near the end of the to-do list forever.
> As always, contributions will be welcome.
> 
> If somebody should contribute code that performs 
> decompression for transport codings, we sure won't stop you 
> from re-using that code for decompression of content codings. 
> But that will remain the responsibility of the application 
> using HttpComponents.
> 
> cheers,
>   Roland
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: 
> httpclient-user-help@jakarta.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Mime
View raw message