cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ellis Pritchard <el...@nukinetics.com>
Subject Re: [jira] Created: (COCOON-1985) AbstractCachingProcessingPipeline locking with IncludeTransformer may hang pipeline
Date Thu, 18 Jan 2007 16:02:09 GMT
Hi,

The crux is that the sub-pipeline is called twice within the context of 
the master pipeline (once by the root pipeline, once by an include); 
thus the pipeline keys which are the same are those for the 
sub-pipeline, not the master pipeline.

My 'broken' pipeline is too complex to explain, but it's basically 
something like:

<map:match pattern="master">
  <map:generate src="cocoon:/foo">
  <map:transform src="page.xsl"/> <!-- generates include element for 
"cocoon:/included" -->
  <map:transform type="include"/> <!-- includes "included" sub-pipeline -->
  <map:serialize/>
</map:match>

<map:match pattern="included">
  <map:generate src="cocoon:/foo">
  <map:transform src="included-page.xsl"/>
  <map:serialize/>
</map:match>

<map:match pattern="foo"> <!-- this gets called twice -->
  <map:generate ... />
  <map:serialize/>
</map:match>

Ellis.


Ard Schrijvers wrote:

>Hello,
>
>  
>
>>Cocoon 2.1.9 introduced the concept of a lock in 
>>AbstractCachingProcessingPipeline, an optimization to prevent 
>>two concurrent requests from generating the same cached 
>>content. The first request adds the pipeline key to the 
>>transient cache to 'lock' the cache entry for that pipeline, 
>>subsequent concurrent requests wait for the first request to 
>>cache the content (by Object.lock()ing the pipeline key 
>>entry) before proceeding, and can then use the newly cached content.
>>
>>However, this has introduced an incompatibility with the 
>>IncludeTransformer: if the inclusions access the same 
>>yet-to-be-cached content as the root pipeline, the whole 
>>assembly hangs, since a lock will be made on a lock already 
>>held by the same thread, and which cannot be satisfied.
>>
>>e.g.
>>i) Root pipeline generates using sub-pipeline cocoon:/foo.xml
>>ii) the cocoon:/foo.xml sub-pipeline adds it's pipeline key 
>>to the transient store as a lock.
>>iii) subsequently in the root pipeline, the IncludeTransformer is run.
>>iv) one of the inclusions also generates with 
>>cocoon:/foo.xml, this sub-pipeline locks in 
>>AbstractProcessingPipeline.waitForLock() because the 
>>sub-pipeline key is already present.
>>v) deadlock.
>>    
>>
>
>I do not understand one part of it. If a sub-pipeline is called, cocoon:/foo.xml, there
is lock generated for this sub-pipeline seperately, right? (if not, I do not understand why
it is not like this. I suppose a lock is generated for the root pipeline, but as well for
every sub-pipeline individually. I suppose though, because i did not actually look at the
code). 
>
>Now, if the include transformer calls this same sub-pipeline, which is having its own
lock, I do not see why a deadlock can occur? The root-pipeline is locked, the sub-pipeline
is locked as well. The include transformer wants to include the same sub-pipeline, waits untill
this one is finished, then can includes it, right? 
>
>I most be missing something, 
>
>Regards Ard
>
>  
>
>>I've found a (partial, see below) solution for this: instead 
>>of a plain Object being added to the transient store as the 
>>lock object, the Thread.currentThread() is added; when 
>>waitForLock() is called, if the lock object exists, it checks 
>>that it is not the same thread before attempting to lock it; 
>>if it is the same thread, then waitForLock() returns success, 
>>which allows generation to proceed. You loose the efficiency 
>>of generating the cache only once in this case, but at least 
>>it doesn't hang! With JDK1.5 this can be made neater by using 
>>Thread#holdsLock() instead of adding the thread object itself 
>>to the transient store.
>>
>>See patch file.
>>
>>However, even with this fix, parallel includes (when enabled) 
>>may still hang, because they pass the not-the-same-thread 
>>test, but fail because the root pipeline, which holds the 
>>initial lock, cannot complete (and therefore statisfy the 
>>lock condition for the parallel threads), before the threads 
>>themselves have completed, which then results in a deadlock again.
>>
>>The complete solution is probably to avoid locking if the 
>>lock is held by the same top-level Request, but that requires 
>>more knowledge of Cocoon's processing than I (currently) have!
>>
>>IMHO unless a complete solution is found to this, then this 
>>optimization should be removed completely, or else made 
>>optional by configuration, since it renders the 
>>IncludeTransformer dangerous.
>>
>>
>>-- 
>>This message is automatically generated by JIRA.
>>-
>>If you think it was sent incorrectly contact one of the 
>>administrators: 
>>    
>>
>https://issues.apache.org/jira/secure/Administrators.jspa
>-
>For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>        
>  
>


Mime
View raw message