cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sylvain Wallez <sylv...@apache.org>
Subject Re: Weird multithreading bug in Cron block
Date Thu, 09 Jun 2005 13:01:04 GMT
Vadim Gritsenko wrote:

> Sylvain Wallez wrote:
>
>> Sylvain Wallez wrote:
>>
>>> Weird, weird, weird! Anybody having a hint about why fireJob() is 
>>> doing this environment mixture?
>>
>>
>>
>> Actually fireJobAt() is broken also when using another test case. 
>
>
> What's wrong with it?
>
>
>> Desperately searching for the cause, I went back to basics, i.e. "new 
>> Thread(runnable).start()". Also broken, but helped me to finally find 
>> the cause :-)
>>
>> The problems lies in CocoonComponentManager.addForAutomaticRelease().
>>
>> The environmentStack is a CloningInheritableThreadLocal. That means 
>> that when we create a new thread, it inherits the environment stack 
>> of its parent thread.
>>
>> The result is that threads created by Cocoon *always* inherit an 
>> environment stack of at least size 1:
>> - in the cron block, that's the environment of the first http 
>> request, which created the Cocoon object
>> - for "new Thread()", that's the same as above, plus all sitemaps 
>> that we've been through when we create the thread.
>>
>> Now let's look at InvokeContext.getProcessingPipeline() (in 
>> treeprocessor): if this is an internal request, the pipeline object 
>> is added for automatic release. I guess this is to avoid memory leaks 
>> if ever we forget to call resolver.release() on a sitemap source.
>>
>> Following this path, let's go to 
>> CocoonComponentManager.addForAutomaticRelease(). The component that 
>> has to be autoreleased is added to a list attached to the *first* 
>> environment of the stack (because of "stack.get(0)"), and is 
>> therefore released when we exit this environment.
>>
>> Now what happens when we create a thread that runs in the background? 
>> The end of processing of the *http* request releases pipeline objects 
>> of all child threads of the servlet engine's thread (the one which 
>> processes the http request). If the background thread uses a 
>> "cocoon:" URL and is currently executing the corresponding pipeline, 
>> recycle() is called on all pipeline components and bang, NPEs all 
>> around the place!!
>>
>> And this leads to very random bugs: since servlet engines uses a 
>> thread pools, this erroneous pipeline release happens only when the 
>> servlet engine reuses the thread that intially loaded CocoonServlet. 
>> And NPEs happen when this first thread is used *and* a scheduler 
>> thread is executing a "cocoon:" pipeline. Weird...
>>
>> So the question is:
>> - why does the environment stack have to be inherited by child 
>> threads? Is it to keep the current context? Then isn't inheriting the 
>> current processor and uri context enough?
>> - why is the pipeline automatically released? Is to avoid memory leaks?
>>
>> Possible remedies would be to remove one of the above features, but I 
>> guest they're there for a reason.
>
>
> Remedy is to use thread pool(s), and not create local Threads - with 
> the exception of situation when local thread lives no longer than 
> original request.


I used the scheduler's thread pool, and that's when the problem appeared.

> If thread lives longer than request, use RunnableManager (or Cron), 
> which are using thread pools. Thread from the thread pool should be 
> set up with environment / processor, and it will be independent of 
> http environment which triggered the job.


Yes, I know that (having largely contributed to the CocoonJobExecutor). 
The problem is the inheritance of the enviroment stack between threads 
(either in the pool or brand new ones) and their parent, which happens 
to be the servlet engine's thread that processed the very first request 
in the case of RunnableManager.

> CocoonQuartzJobExecutor already has enterEnv/leaveEnv, so it should 
> work, if you have issues with it - what are they?


Well, read the explanation above :-)

If a CronJob uses the SourceResolver to process a "cocoon:" URI, the 
corresponding pipeline occasionally gets recycled in the middle of its 
processing.

Sylvain

-- 
Sylvain Wallez                        Anyware Technologies
http://apache.org/~sylvain            http://anyware-tech.com
Apache Software Foundation Member     Research & Technology Director


Mime
View raw message