tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Mikusa <dmik...@gopivotal.com>
Subject Re: Tomcat 8.0.0-RC5: asynchron IO and back pressure with ReadListener
Date Thu, 30 Jan 2014 22:02:28 GMT
On Jan 30, 2014, at 3:38 PM, Yann Simon <yann.simon.fr@gmail.com> wrote:

> 2014-01-30 Daniel Mikusa <dmikusa@gopivotal.com>:
>> On Jan 30, 2014, at 11:18 AM, Yann Simon <yann.simon.fr@gmail.com> wrote:
>> 
>>> Hi,
>>> 
>>> I wrote a sample app to demonstrate the problem:
>>> https://github.com/yanns/servlet31_async
>>> 
>>> You can generate an exploded war with maven: mvn war:exploded
>>> I deployed the application in tomcat 8.0.0-RC10.
>>> 
>>> The 2 upload form does work.
>>> The 1st upload form uses a new thread in , and that does not work.
>>> https://github.com/yanns/servlet31_async/blob/master/src/main/java/com/yann/ReadListenerImpl.java#L22
>> 
>> I’m not sure I see the point of the code here.  If you force it to block with Thread.sleep()
you’re going to tie up the thread that you’ve created and you’re going to be back to
having threads sitting around and doing nothing.  If that’s the case, you may as well save
yourself some trouble and use the blocking apis.
> 
> This sample application is only a simple way to show what I want to
> achieve with a more complex application that I mentioned at the
> beginning of the thread:
> https://github.com/yanns/play2-war-plugin/blob/servlet31/project-code/core/servlet31/src/main/scala/play/core/server/servlet31/RequestHandler31.scala#L80

I haven’t looked at your code.  I’m not a Scala guy, so I can’t begin to comment on
that.  What you’ve described below sounds feasible though.  Including some notes below.

> Let me try to explain the use case:
> let's say we want to save the uploaded file chunk by chunk in a
> database, for which we have an asynchronous API.
> - we receive the first chunk, that we can read synchronously in
> onDataAvailable, no problem so far

Ok.  Because this is our first call to onDataAvailable, the container is going to handle that
for us.  Question.  Are we reading until isReady() returns false here?  or are we existing
onDataAvailable with data that still needs to be read?  This is important to know so we know
who’s responsibility it is to continue the reading process.

> - we trigger the saving of this chunk in the database, and of course,
> we do not wait for the completion of the operation

Ok

> - we receive the signal that a new chunk is available - the container
> call onDataAvailable

This is not quite correct.  The container won’t call onDataAvailable here, unless you read
all of the data that was available in the first step (i.e. read till isReady() returns false).
 It’s not specified, but it doesn’t sound like that’s your intent here.

> - we cannot push this chunk into the database, as the first operation
> is not completed

Ok.  At this point, you’re going to have to either buffer data or stop reading.  It sounds
like you want to do the later, so you’re going to need to make sure something in your code
starts reading again when your code can process more data, since the container is not going
to handle the call back.

> - we asynchronously wait for the completion of the database operation.
> We do not read any data so that the container push the pressure back
> to the browser, telling it to slowdown. We do not want to block any
> thread here. That's why we must exit the onDataAvailable without
> having read any data.

Sounds OK, but you’re now responsible for making sure that the rest of the data is consumed.
 If you don’t do this, the request will appear to hang until it times out.

> - when the database says us that it is ready for a new operation (by a
> callback, or an event), then we can read the second chunk and trigger
> the save operation in the database

Once the database write is complete, you’re just going to need to call onDataAvailable manually
some how.  The container won’t do it because you previously exited before isReady() returned
false.

> This a what I simulated with the new Thread and Thread.sleep.
> In reality, there are no "new Thread" - it just simulates that at some
> point later, the database driver informs us that the database is ready
> for a new operation.
> There are no Thread.sleep neither - it just simulates a slow database
> to check the back-pressure mechanism that the container should do on
> the browser.

Ok, I understand what you’re going for here.

> Also, if you said that I must consume the chunks synchronously in
> onDataAvailable, I can simply not implement this double triggering
> (from browser and from database) without blocking any thread. For me,
> it means that I cannot totally use the asynchronous IO mechanism of
> the OS.

The non-blocking API that Servlet 3.1 exposes is pretty flexible, but things get complicated
quick.  I don’t see any reason why you couldn’t do what you’ve described.  It’s just
going to be complicated.  You’re going to have to know when the container will call onDataAvailable
and when you need to call it.  

Hope that helps.

Dan


> 
> I hope I made my goals clear.
> 
> Cheers,
> Yann
> 
>> 
>> Here’s what I’d suggest to make this work.
>> 
>>  - in onDataAvailable read as much data as possible
>>  - if you read until input.isReady() is false then just exit the function.  The container
will call back when more data can be read.
>>  - if you need to stop reading for some reason but input.isReady() is still true,
use a thread pool to schedule your own call back to onDataAvailable at some point in the future.
 You can then continue reading at that point in time.
>>  - Repeat until you’ve read all the data.
>> 
>> You still need additional threads with this approach, but it’s not one to one.
 A small thread pool can service many requests because the thread is only active when data
is being read.
>> 
>> Here’s an example of this in action.
>> 
>>  http://git.eclipse.org/c/jetty/org.eclipse.jetty.project.git/tree/jetty-servlets/src/main/java/org/eclipse/jetty/servlets/DataRateLimitedServlet.java
>> 
>> Dan
>> 
>>> 
>>> The "onDataAvailable" is called only one time.
>>> 
>>> With jetty, it does work (mvn jetty:run)
>>> 
>>> I hope this can help.
>>> 
>>> Yann
>>> 
>>> 2014-01-08 Yann Simon <yann.simon.fr@gmail.com>:
>>>> 2014/1/8 Daniel Mikusa <dmikusa@gopivotal.com>:
>>>>> On Jan 8, 2014, at 12:04 PM, Yann Simon <yann.simon.fr@gmail.com>
wrote:
>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> I am trying to write a servlet that asynchronously read data from
the
>>>>>> servlet request input stream.
>>>>>> I tested my servlet with tomcat 8.0.0-RC5.
>>>>> 
>>>>> If possible, you might want to try 8.0.0-RC10 or trunk and see if you're
getting the same behavior.
>>>>> 
>>>>>> 
>>>>>> the symptoms:
>>>>>> - I must synchronously read the input stream in onDataAvailable()
so
>>>>>> that the upload works
>>>>>> 
>>>>>> what I expected:
>>>>>> I want to be more "reactive" (buzzword of the moment) and not read
the
>>>>>> input stream until I am ready (=until the previous chunk is processed)
>>>>>> I though I could return from onDataAvailable() before having read
all
>>>>>> the data, read the data in another thread.
>>>>> 
>>>>> Do you have a code sample?  It would help to see what you're doing.
>>>>> 
>>>>>> I expected onDataAvailable() to be called again when I consumed all
the data
>>>>>> (servletInputStream.isReady becomes false)
>>>>> 
>>>>> Generally this sounds OK.  When you call ServletInputStream.isReady()
and it returns false, that should trigger the container to call onDataAvailable() when more
data is available to be read.  If you return from onDataAvailable() and ServletInputStream.isReady()
is still true the container won't call onDataAvailable() again.  You'll be responsible for
calling it when you're ready to read more.
>>>>> 
>>>>>> That way, I could implement back pressure on the browser as long
as my
>>>>>> servlet has not finished its work with the actual chunk
>>>>>> 
>>>>>> But if I do that, neither onDataAvailable() nor onAllDataRead() is
called again.
>>>>> 
>>>>> Again, a code sample would be helpful.  Debugging non-blocking IO is
tricky.  If you can include a sample Servlet or test case, it would greatly increase your
chance of getting feedback.
>>>> 
>>>> Thanks for the quick answer!
>>>> 
>>>> I have a code sample, but it may be too complicated to help debugging
>>>> the problem.
>>>> It is written in Scala. Its purpose is to provide a servlet that runs
>>>> asynchronous action from Playframework (http://www.playframework.com/)
>>>> 
>>>> https://github.com/yanns/play2-war-plugin/blob/servlet31/project-code/core/servlet31/src/main/scala/play/core/server/servlet31/RequestHandler31.scala#L74
>>>> 
>>>> The line 80 (iteratee = iteratee.pureFlatFold ) use a closure that
>>>> consumes the input stream asynchronously in another thread.
>>>> 
>>>> I'll try to take the time to write a much simpler code sample in Java.
>>>> 
>>>>> 
>>>>> Dan
>>>>> 
>>>>>> 
>>>>>> When I consume the input stream synchronously in onDataAvailable(),
it
>>>>>> works as expected.
>>>>>> 
>>>>>> Am I misunderstanding the asynchron IO in Tomcat 8?
>>>>>> 
>>>>>> Thanks in advance for any ideas!
>>>>>> Yann
>>>>>> 
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>> 
>>>>> 
>>>>> 
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>> 
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message