hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Khosro Asgharifard <khosro_quest...@yahoo.com>
Subject Re: HttpClient does not seem to correctly handle chunked response
Date Wed, 20 Jan 2010 18:01:54 GMT
Hello,
I have the same problem :
 Error fetching news HTML for: Event[entryData=EntryData(title=Yemeni al-Qaeda branch a magnet
for jihadists, url=http://feeds.washingtonpost.com/click.phdo?i=2657414efdbc5d7710278204dac31246)].
java.io.IOException: CRLF expected at end of chunk: 49/54
    at org.apache.commons.httpclient.ChunkedInputStream.readCRLF(ChunkedInputStream.java:207)
    at org.apache.commons.httpclient.ChunkedInputStream.nextChunk(ChunkedInputStream.java:219)
    at org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream.java:176)
    at org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream.java:196)
    at org.apache.commons.httpclient.ChunkedInputStream.exhaustInputStream(ChunkedInputStream.java:369)
    at org.apache.commons.httpclient.ChunkedInputStream.close(ChunkedInputStream.java:346)
    at java.io.FilterInputStream.close(FilterInputStream.java:155)
    at org.apache.commons.httpclient.AutoCloseInputStream.notifyWatcher(AutoCloseInputStream.java:194)
    at org.apache.commons.httpclient.AutoCloseInputStream.close(AutoCloseInputStream.java:158)
    at java.io.BufferedInputStream.close(BufferedInputStream.java:451)
    at sun.nio.cs.StreamDecoder.implClose(StreamDecoder.java:358)
    at sun.nio.cs.StreamDecoder.close(StreamDecoder.java:173)
    at java.io.InputStreamReader.close(InputStreamReader.java:182)
    at net.htmlparser.jericho.Util.getString(Unknown Source)
    at net.htmlparser.jericho.Source.getString(Unknown Source)
    at net.htmlparser.jericho.Source.<init>(Unknown Source)
    at net.htmlparser.jericho.Source.<init>(Unknown Source)
    at ir.ideacenter.biz.service.NewsService.addNewNews(NewsService.java:116)
    at ir.ideacenter.biz.service.NewsService$$FastClassByCGLIB$$fa1e5631.invoke(<generated>)
    at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
    at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:692)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
    at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:625)
    at ir.ideacenter.biz.service.NewsService$$EnhancerByCGLIB$$a9589857.addNewNews(<generated>)
    at ir.ideacenter.biz.service.fetcher.NewsFetchListener.contentLoaded(NewsFetchListener.java:65)
    at ir.ideacenter.biz.crawler.AbstractCrawler.fireEvents(AbstractCrawler.java:56)
    at ir.ideacenter.biz.crawler.MultiThreadedNewsCrawler$L2Task.run(MultiThreadedNewsCrawler.java:199)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)

Khosro.




________________________________
From: Oleg Kalnichevski <olegk@apache.org>
To: HttpClient User Discussion <httpclient-users@hc.apache.org>
Sent: Wed, January 20, 2010 8:46:49 AM
Subject: Re: HttpClient does not seem to correctly handle chunked response

On Wed, 2010-01-20 at 08:20 -0800, Royan wrote:
> We have an XML API service which splits reply XML data into chunks if it is
> larger then certain amount of bytes. Here is the sample piece of the reply:
> 
> HTTP/1.1 200 OK
> Server: nginx/0.6.35
> Date: Wed, 20 Jan 2010 14:53:27 GMT
> Content-Type: text/xml;charset=UTF-8
> Transfer-Encoding: chunked
> Connection: keep-alive
> X-Powered-By: Servlet 2.4; JBoss-4.2.2.GA (build: SVNTag=JBoss_4_2_2_GA
> date=200710221139)/Tomcat-5.5
> Connection: close
> 
> 1f0d
> <?xml version='1.0' encoding='UTF-8'?>
> <root>
> [...]
> <Label><![CDATA[Some character data br
> 2000
> oken in the middle of the string]]></Label>
> [...]
> <root>
> 
> 0
> 
> 
> The problem is when this XML is retrieved via
> httpResponse.getEntity().getContent() I expect all chunks to be transformed
> into single XML with no service information (I'm talking about some strange
> 2000 number appearing in the middle of the string)
> 
> In fact returned content is not always correctly parsed and contains such
> service information, which in turn makes my XML parser throw an exception
> 
> httpResponse.getEntity().isChunked() always retrieves "true"
> 
> Can anyone advice on what am I doing wrong or otherwise provide information
> how to workaround such issue?
> 
> Thanks,
> Roman 

I cannon recall a single confirmed problem with the correctness of the
chunk coding code in HttpClient in the past 7 (seven) years I am a
committer on the project. 

Double-check your code. If you are reasonably sure this is not an issue
with your code, post a _COMPLETE_ wire / context log of the session and
a test case reproducing the problem (preferably self-contained).

Oleg




---------------------------------------------------------------------
To unsubscribe, e-mail: mailto:httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: mailto:httpclient-users-help@hc.apache.org


      
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message