cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefan Pietschmann" <dev.sen...@gmail.com>
Subject AW: AW: Desparately seeking help with pipeline implementation
Date Mon, 01 May 2006 07:17:30 GMT
Hi Sylvain (and others),

thank you for the quick response which finally helped me understand how the
pipeline is used at runtime.

What I do is to throw transformers out depending on their attributes (it's a
bit more complicated, but anyway..). Now that I see that the transformerlist
is built anew for every request I do not need to backup the original list
and load it everytime. I thought it would be built only once at the
beginning.

So my choices now are to overload the addTransformer() method, only adding a
transformer under certain circumstances, or leave my current implementation
which goes through this.transformers, this.transformerSources and
this.transformerParams, checks the parameters and throws the transformers,
sources and parameters out of the lists.
The first way seems easier, but it means that I have to copy the whole
method from the super class to my pipeline and change it there - I don't
like that because you I have to check for changes with every Cocoon update.
With the second way I can simply outsource the code going through the lists
to my own method and call it at the beginning of setupPipeline().

I do have one more question - to make it right this time. If I prevent a
transformer from being added to the list in addTransformer(), or if I kick
it out in my method, I see I have to release it. This is done by the
following in AbstractProcessingPipeline:

int size = this.transformerSelectors.size();
for (int i = 0; i < size; i++) {
  final ComponentSelector selector = (ComponentSelector)
this.transformerSelectors.get(i);
  selector.release((Component) this.transformers.get(i));
  this.newManager.release(selector);
}

In my method it would look like:

Iterator transformerItt = this.transformers.iterator();
...
int i = 0;
while (transformerItt.hasNext()) {
      Transformer trans = (Transformer) transformerItt.next();
	...
	// check if the transformer should be run
	..
	// kick it out
      transformerItt.remove();
	// release transformer
	final ComponentSelector selector = (ComponentSelector)
this.transformerSelectors.get(i);
	  selector.release((Component) trans);
	this.newManager.release(selector);
	i++;
}     

Right?

Stefan                   

| -----Urspr√ľngliche Nachricht-----
| Von: Sylvain Wallez [mailto:sylvain@apache.org]
| Gesendet: Montag, 1. Mai 2006 00:01
| An: dev@cocoon.apache.org
| Betreff: Re: AW: Desparately seeking help with pipeline implementation
| 
| Stefan Pietschmann wrote:
| > | -----Urspr√ľngliche Nachricht-----
| > | Von: Sylvain Wallez [mailto:sylvain@apache.org]
| > | Gesendet: Sonntag, 30. April 2006 14:00
| > | An: dev@cocoon.apache.org
| > | Betreff: Re: Desparately seeking help with pipeline implementation
| > |
| > | Stefan Pietschmann wrote:
| > | >
| > | > Hi guys,
| > | >
| > | > for my thesis I have implemented a custom pipeline, which modifies
| the
| > | > list of transformers to be run during pipeline setup. This is done
| > | > with every request in setupPipeline(), so I need to reset to the
| > | > original every time, before I modify again.
| > | >
| > |
| > | Why do you need that? Can't it be done by overloading one of
| > | setGenerator/addTransformer/setSerializer?
| >
| > I adapt the pipeline with every request according to some external data.
| > Isn't addTransformer() called only once when the pipeline is built? If
| it
| > is, I couldn't use it, because this would mean I adapt the pipeline
| once,
| > and all subsequent requests use the adapted one.
| > However, if this.transformers (and the other ArrayLists with the sources
| and
| > parameters) is built anew for every request, then this might be an easy
| way
| > out.
| >
| 
| The contents of a pipeline is not reused across requests! All components
| are releases and cleared in recycle(), which occurs once the pipeline
| has been executed.
| 
| The setGenerator/addTransformer/setSerializer are called by the sitemap
| engine when executing <map:generate>, <map:transform> and
| <map:serialize> respectively.
| 
| This is probably a good place if you want to automatically add some
| components to the pipeline, particularly considering that pipeline cache
| key and validity computation happens after these methods. Now if you
| remove some of the components that were already in the pipeline, don't
| forget to release them or you may encounter some memory leaks in object
| pools (see the code in AbstractProcessingPipeline)

So my 
| 
| > | Giving more details about the errors you obtain would be useful, but I
| > | guess this is more related to component pools than to multithreading.
| >
| > I can attach some stacktraces if that helps...
| >
| 
| Yes, it always helps.
| 
| > I only see that I only get problems when using multiple threads. The
| problem
| > might be related to anything else - I don't seem to understand how a
| > pipeline is used by multiple threads. I thought - since it's recycleable
| -
| > that each thread gets its own pipeline object.
| >
| 
| Yes, pipelines are not shared among threads. But if you do some "nasty"
| things with the pipeline components, it may happen that the pipeline is
| not be correctly recycled and therefore components used by multiple
| threads...
| 
| Sylvain
| 
| --
| Sylvain Wallez - http://bluxte.net


Mime
View raw message