tomcat-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rainer Jung <rainer.j...@kippdata.de>
Subject Re: 5.5.27 blocker: URIEncoding UTF-8 broken for 5.5.trunk
Date Mon, 25 Aug 2008 19:16:27 GMT
Unfortunately removing the available() method agains gives us back
44494. But there is some hope.

B2CConverter gets used for two different things:

1) It is associated with a request and used inside the adapter to decode
the URI.

In this case, it seems that during the lifetime of the  ByteChunk
underlying the B2CConverter (the URI ByteChunk) we only call
conv.convert() once. So there is no real need for any conv.recycle()
before conv.convert() (cleaning up some unrelated old ByteChunk; if we
would need to do that, then we should do it at the end of convertURI for
some request and not near the beginning of convertURI for some later
request).

2) It is associated with the InputBuffer to transform input data.

In this case we mightcall convert() multiple times and handle a
difficult relationship between direct access to the Chunks and via
convert. Here we need to clean up the ByteChunk at the end of the
request handling.

As far as I can see, the instances of B2CConverter used for 1) and for
2) are independant of each other.

B2CConvertor.recycle() has only one action, it calls
ReadConvertor.recycle(), which reads the remaining bytes in the
ByteChunk() and throws them away. This seems to be good for 2) (e.g. BZ
44494), but it eats data from a ByteChunk instance associated with the
B2CConverter during an earlier request in case 1).

I would suggest to simply remove the conv.recycle in case 1). I dn't see
any use of it. If we really think we need to eat up remaining bytes in
the ByteChunk in 1), then we should do the conv.recycle() near the end
of convertURI.

Regards,

Rainer

Rainer Jung schrieb:
> It's the available() methde of the class IntermediateInputStream
> contained in B2CConverter. It doesn't exist in 6.0. If I comment it out
> in 5.5 trunk. the problem is gone.
> 
> The method was first introduced in
> 
> http://svn.apache.org/viewvc/tomcat/connectors/trunk/util/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=481614&r2=568699&pathrev=568699&diff_format=h
> 
> and the changed to it's final contents in
> 
> http://svn.apache.org/viewvc/tomcat/connectors/trunk/util/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=568699&r2=569970&diff_format=h
> 
> I didn't yet check, what negative consequences removing of the method
> has, or if there is a better implementation for it. We should carefully
> check the consequences of changing it w.r.t. BZ 44494 with the test
> webapp, we got for that BZ.
> 
> Regards,
> 
> Rainer
> 
> 
> Rainer Jung schrieb:
>> OK, cancelled my appointment. More info:
>>
>> I backported all functional changes in o.a.tomcat.util.buf from tc6.x to
>> tc5.5 and can't reproduce the problem any more. Those are very few
>> changes. I'll narrow it down some more during the next hour. Stay tuned.
>>
>> Rainer
>>
>> Rainer Jung schrieb:
>>> Filip Hanik - Dev Lists schrieb:
>>>> Thanks Rainer, I will take a look at it tonight
>>> Thank you!
>>>
>>> Last info chunk for today: in CoyoteAdapter.convertURI, before the
>>> try/catch block that either creates or recycles the B2CConverter, the
>>> ByteChunk bc coming from the decodedURI contains the correct URI. After
>>> the recycle of the B2CConverter, the ByteChunk is empty and thus we end
>>> up in a default redirect.
>>>
>>> Although we already read the correct URI from the request, the
>>> B2CConverter associated with the request detroys the already read URI in
>>> the recycle. I don't see the delta to 6.0. CoyoteAdapter seems fine,
>>> maybe in ByteChunk?
>>>
>>> Regards,
>>>
>>> Rainer
>>>
>>>> Filip
>>>>
>>>> Rainer Jung wrote:
>>>>> Remy Maucherat schrieb:
>>>>>  
>>>>>> On Mon, 2008-08-25 at 17:16 +0200, Rainer Jung wrote:
>>>>>>    
>>>>>>> If we revert the backport of
>>>>>>>
>>>>>>> http://svn.eu.apache.org/viewvc/tomcat/trunk/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=642819&r2=647307&diff_format=h
>>>>>>>
>>>>>>>
>>>>>>> then the redirect loop is gone, and the usual content gets served,
but
>>>>>>> we know, that this change was needed to fix the "remaining garbage"
>>>>>>> part
>>>>>>> of 44494. So reverting it without any alternative is not really
an
>>>>>>> option.
>>>>>>>       
>>>>>> Most likely, some backport is missing (many patches made up this
input
>>>>>> fixes). OTOH, I don't think this particular problem is that critical,
so
>>>>>> I would be in favor of dropping it if fixing the issue is complex.
>>>>>>     
>>>>> Got some more info:
>>>>>
>>>>> 1) What happens inside the new ReadConvertor.recycle(): actually if I
>>>>> print out, which "superfluous" bytes are eaten by the read() loop inside
>>>>> ReadConvertor.recycle(), I can see, that it's all the bytes making up
>>>>> the PATH in the URL. If I request http://myserver:8080/, recycle read
>>>>> one character, namely "/", if I request http://myserver:8080/index.jsp,
>>>>> then recycle reads all characters from "/index.jsp". In my understanding
>>>>> of those patches, the time recycle in ReadConvertor gets called, those
>>>>> should have alrady been read and only body bytes left over after request
>>>>> processing should be eaten.
>>>>>
>>>>> 2) So I checked, when recycle() gets called, and I see, that during the
>>>>> first few (here: 2) requests it doesn't get called at all (and those
>>>>> work), and during the following broken requests, it gets called in the
>>>>> following stack:
>>>>>
>>>>>         at
>>>>> org.apache.tomcat.util.buf.ReadConvertor.recycle(B2CConverter.java:222)
>>>>>         at
>>>>> org.apache.tomcat.util.buf.B2CConverter.recycle(B2CConverter.java:64)
>>>>>         at
>>>>> org.apache.catalina.connector.CoyoteAdapter.convertURI(CoyoteAdapter.java:475)
>>>>>
>>>>>         at
>>>>> org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:265)
>>>>>
>>>>>         at
>>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:172)
>>>>>
>>>>>         at
>>>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
>>>>>
>>>>>         at
>>>>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
>>>>>
>>>>>         at
>>>>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
>>>>>
>>>>>         at
>>>>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
>>>>>
>>>>>         at
>>>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>>>>>
>>>>>         at java.lang.Thread.run(Thread.java:595)
>>>>>
>>>>> Didn't yet go further into it, but maybe something is wrong in
>>>>> CoyoteAdapter.convertURI?
>>>>>
>>>>> Regards,
>>>>>
>>>>> Rainer
>>>>>
>>>>> P.S: I'll soon need to stop investigating this for today. If anyone can
>>>>> take over that will be nice, because we really should have a working
>>>>> 5.5.27 soon.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


Mime
View raw message