cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christoph Gaffga" <cgaf...@triplemind.com>
Subject ContentAggregator and multithreating (migrating C2 -> C2.1M2), is it thread-safe?
Date Sat, 05 Jul 2003 20:42:17 GMT
Hi,
I worked before on doing multithreading in the ContentAggregator, but whith
out succes.

I had a modified ContentAggregator class with Part extending thread, getting
all output buffered, and a second loop in the aggregators generate-method to
join and stream the buffers.

This solution did work in c-2.0.4, but not in c2.1m3. It sometimes throws
exceptions during the release of components.

It the meantime there was some interest from Tony Culshaw about this issue.
I sent him my aggregator, but he came up with his own way of doing it. It
was mainly the same as my SAX-buffer-solution, but storing the parts as DOM
and then stream them.

Carsten Ziegeler gave me the advise to have a look at the
CIncludeTransformer, where parallel retrieval of source should work. I tried
it out. I wrote a aggregator that uses exactly the same
IncludeCacheManager/Session mechanism as CIncludeTransformer.
When I switch the parameter "parallel" for IncludeCacheManagerSession to
"true", I run into the same problem as before. For parallel=false ist works.

So it seems to me that the problem is somewhere deeper in cocoon or avalon
code.

He wrote:
> multi-threading in Cocoon is unfortunately a little bit difficult.
> Cocoon uses some ThreadLocal variables that have to set and used
> properly in any subtasks

I have no idea witch variables that could be. I thing some method for the
release has to be decleare as synchronized. Has anybody any idea?

As I understand the code, it does (by useing the cache manager/session
classes) nothing more with the variables then when I have two loops, the
first that starts the source threads, using SourceUtil.toSAX to fetch the
content, the second loop joins the threads and streams them. What is the
cache manager good for regarding multithreating?

I think everything, that worked in cocoon 2.0.4, should work in 2.1.
Especialy a that importen feature as multithreating.

I would apriciate any help, I'm totaly confused at the moment.

Regards
Christoph Gaffga
cgaffga@triplemind.com

P.S.: If anybody want to have a look on the aggregator code, I can send you
the java file (dont want attachement on the mailinglist).

P.P.S.: The Exception thrown (sometimes, depends on the pool I think)

java.lang.IndexOutOfBoundsException: Index: 23, Size: 23

at java.util.ArrayList.RangeCheck(ArrayList.java:507)

at java.util.ArrayList.remove(ArrayList.java:392)

at
org.apache.cocoon.components.EnvironmentDescription.removeFromAutoRelease(Co
coonComponentManager.java:620)

at
org.apache.cocoon.components.CocoonComponentManager.removeFromAutomaticRelea
se(CocoonComponentManager.java:420)

at
org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.release(Abs
tractProcessingPipeline.java:203)

at
org.apache.cocoon.components.source.impl.SitemapSource.reset(SitemapSource.j
ava:454)

at
org.apache.cocoon.components.source.impl.SitemapSource.recycle(SitemapSource
.java:474)

at
org.apache.cocoon.components.source.impl.SitemapSourceFactory.release(Sitema
pSourceFactory.java:113)

at
org.apache.excalibur.source.impl.SourceResolverImpl.release(SourceResolverIm
pl.java:308)

at
org.apache.cocoon.components.CocoonComponentManager.release(CocoonComponentM
anager.java:467)

at
org.apache.cocoon.environment.AbstractEnvironment.release(AbstractEnvironmen
t.java:521)

at org.apache.cocoon.generation.FileGenerator.recycle(FileGenerator.java:90)

at
org.apache.avalon.excalibur.pool.ResourceLimitingPool.put(ResourceLimitingPo
ol.java:438)

at
org.apache.avalon.excalibur.component.PoolableComponentHandler.doPut(Poolabl
eComponentHandler.java:245)

at
org.apache.avalon.excalibur.component.ComponentHandler.put(ComponentHandler.
java:452)

at
org.apache.avalon.excalibur.component.ExcaliburComponentSelector.release(Exc
aliburComponentSelector.java:336)

at
org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo
nentSelector.java:320)

at
org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo
nentSelector.java:317)

at
org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.recycle(Abs
tractProcessingPipeline.java:637)

at
org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline
.recycle(AbstractCachingProcessingPipeline.java:959)

at
org.apache.avalon.excalibur.pool.ResourceLimitingPool.put(ResourceLimitingPo
ol.java:438)

at
org.apache.avalon.excalibur.component.PoolableComponentHandler.doPut(Poolabl
eComponentHandler.java:245)

at
org.apache.avalon.excalibur.component.ComponentHandler.put(ComponentHandler.
java:452)

at
org.apache.avalon.excalibur.component.ExcaliburComponentSelector.release(Exc
aliburComponentSelector.java:336)

at
org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo
nentSelector.java:320)

at
org.apache.cocoon.components.EnvironmentDescription.removeFromAutoRelease(Co
coonComponentManager.java:615)

at
org.apache.cocoon.components.CocoonComponentManager.removeFromAutomaticRelea
se(CocoonComponentManager.java:420)

at
org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.release(Abs
tractProcessingPipeline.java:203)

at
org.apache.cocoon.components.source.impl.SitemapSource.reset(SitemapSource.j
ava:454)

at
org.apache.cocoon.components.source.impl.SitemapSource.toSAX(SitemapSource.j
ava:446)

at org.apache.cocoon.components.source.SourceUtil.toSAX(SourceUtil.java:159)

at org.apache.cocoon.components.source.SourceUtil.toSAX(SourceUtil.java:126)

at
org.apache.cocoon.transformation.helpers.LoaderThread.run(DefaultIncludeCach
eManager.java:467)

at java.lang.Thread.run(Thread.java:534)

java.lang.NullPointerException

at
org.apache.cocoon.environment.AbstractEnvironment.release(AbstractEnvironmen
t.java:521)

at org.apache.cocoon.generation.FileGenerator.recycle(FileGenerator.java:90)

at
org.apache.avalon.excalibur.pool.ResourceLimitingPool.put(ResourceLimitingPo
ol.java:438)

at
org.apache.avalon.excalibur.component.PoolableComponentHandler.doPut(Poolabl
eComponentHandler.java:245)

at
org.apache.avalon.excalibur.component.ComponentHandler.put(ComponentHandler.
java:452)

at
org.apache.avalon.excalibur.component.ExcaliburComponentSelector.release(Exc
aliburComponentSelector.java:336)

at
org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo
nentSelector.java:320)

at
org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo
nentSelector.java:317)

at
org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.recycle(Abs
tractProcessingPipeline.java:637)

at
org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline
.recycle(AbstractCachingProcessingPipeline.java:959)

at
org.apache.avalon.excalibur.pool.ResourceLimitingPool.put(ResourceLimitingPo
ol.java:438)

at
org.apache.avalon.excalibur.component.PoolableComponentHandler.doPut(Poolabl
eComponentHandler.java:245)

at
org.apache.avalon.excalibur.component.ComponentHandler.put(ComponentHandler.
java:452)

at
org.apache.avalon.excalibur.component.ExcaliburComponentSelector.release(Exc
aliburComponentSelector.java:336)

at
org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo
nentSelector.java:320)

at
org.apache.cocoon.components.EnvironmentDescription.release(CocoonComponentM
anager.java:547)

at
org.apache.cocoon.components.CocoonComponentManager.endProcessing(CocoonComp
onentManager.java:199)

at org.apache.cocoon.Cocoon.process(Cocoon.java:641)

at org.apache.cocoon.servlet.CocoonServlet.service(CocoonServlet.java:1083)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)

at
org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.
java:684)

at
org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatch
er.java:432)

at
org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher
.java:356)

at com.triplemind.asp.UrlMappingServlet.service(UrlMappingServlet.java:314)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)

at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:247)

at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:193)

at com.triplemind.asp.server.AuthFilter.doFilter(AuthFilter.java:100)

at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:213)

at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:193)

at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
va:260)

at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
eNext(StandardPipeline.java:643)

at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)

at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)

at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
va:191)

at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
eNext(StandardPipeline.java:643)

at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)

at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)

at
org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2415)

at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180
)

at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
eNext(StandardPipeline.java:643)

at
org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.
java:170)

at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
eNext(StandardPipeline.java:641)

at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172
)

at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
eNext(StandardPipeline.java:641)

at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)

at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)

at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
:174)

at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
eNext(StandardPipeline.java:643)

at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)

at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)

at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:223)

at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:432)

at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConne
ction(Http11Protocol.java:386)

at
org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:534)

at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.jav
a:530)

at java.lang.Thread.run(Thread.java:534)

java.lang.NullPointerException

at
org.apache.cocoon.environment.AbstractEnvironment.release(AbstractEnvironmen
t.java:521)

at org.apache.cocoon.generation.FileGenerator.recycle(FileGenerator.java:90)

at
org.apache.avalon.excalibur.pool.ResourceLimitingPool.put(ResourceLimitingPo
ol.java:438)

at
org.apache.avalon.excalibur.component.PoolableComponentHandler.doPut(Poolabl
eComponentHandler.java:245)

at
org.apache.avalon.excalibur.component.ComponentHandler.put(ComponentHandler.
java:452)

at
org.apache.avalon.excalibur.component.ExcaliburComponentSelector.release(Exc
aliburComponentSelector.java:336)

at
org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo
nentSelector.java:320)

at
org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo
nentSelector.java:317)

at
org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.recycle(Abs
tractProcessingPipeline.java:637)

at
org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline
.recycle(AbstractCachingProcessingPipeline.java:959)

at
org.apache.avalon.excalibur.pool.ResourceLimitingPool.put(ResourceLimitingPo
ol.java:438)

at
org.apache.avalon.excalibur.component.PoolableComponentHandler.doPut(Poolabl
eComponentHandler.java:245)

at
org.apache.avalon.excalibur.component.ComponentHandler.put(ComponentHandler.
java:452)

at
org.apache.avalon.excalibur.component.ExcaliburComponentSelector.release(Exc
aliburComponentSelector.java:336)

at
org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo
nentSelector.java:320)

at
org.apache.cocoon.components.EnvironmentDescription.release(CocoonComponentM
anager.java:547)

at
org.apache.cocoon.components.CocoonComponentManager.endProcessing(CocoonComp
onentManager.java:199)

at org.apache.cocoon.Cocoon.process(Cocoon.java:641)

at org.apache.cocoon.servlet.CocoonServlet.service(CocoonServlet.java:1083)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)

at
org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.
java:684)

at
org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatch
er.java:432)

at
org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher
.java:356)

at com.triplemind.asp.UrlMappingServlet.service(UrlMappingServlet.java:314)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)

at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:247)

at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:193)

at com.triplemind.asp.server.AuthFilter.doFilter(AuthFilter.java:100)

at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:213)

at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:193)

at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
va:260)

at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
eNext(StandardPipeline.java:643)

at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)

at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)

at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
va:191)

at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
eNext(StandardPipeline.java:643)

at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)

at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)

at
org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2415)

at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180
)

at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
eNext(StandardPipeline.java:643)

at
org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.
java:170)

at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
eNext(StandardPipeline.java:641)

at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172
)

at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
eNext(StandardPipeline.java:641)

at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)

at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)

at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
:174)

at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
eNext(StandardPipeline.java:643)

at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)

at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)

at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:223)

at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:432)

at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConne
ction(Http11Protocol.java:386)

at
org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:534)

at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.jav
a:530)

at java.lang.Thread.run(Thread.java:534)



Mime
View raw message