Mailing-List: contact dev-help@cocoon.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cocoon.apache.org
Received-SPF: pass (asf.osuosl.org: domain of dev.sensei@gmail.com designates
 64.233.182.184 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:from:to:subject:date:message-id:mime-version:content-type:content-transfer-encoding:x-mailer:in-reply-to:thread-index:x-mimeole;
        b=J0/k8PgABNinu32PLvS2d2zHX/YC5lXpUdEtuXKvJwSlaL6GafMqy0rOvUi4lm9s7dtGBIczVdR2bn1bDwA1sjHgjJJ/FIdKkmexO9FvAO429wAktmz0vu0jIvS6868Nj7ONYjMRIKBAfz/TgCSKe//i13d/h9enko0f7Jx+izs=
From: "Stefan Pietschmann" <dev.sensei@gmail.com>
To: <dev@cocoon.apache.org>
Subject: AW: AW: Desparately seeking help with pipeline implementation
Date: Mon, 1 May 2006 09:17:30 +0200
Message-ID: <000601c66cef$4ff9eac0$16b2a8c0@blackdragon>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
In-Reply-To: <445533AC.20400@apache.org>
Thread-Index: AcZsoawg3oEdf5nsQImp/Z3vJgZ7cgASpicQ

Hi Sylvain (and others),

thank you for the quick response which finally helped me understand how =
the
pipeline is used at runtime.

What I do is to throw transformers out depending on their attributes =
(it's a
bit more complicated, but anyway..). Now that I see that the =
transformerlist
is built anew for every request I do not need to backup the original =
list
and load it everytime. I thought it would be built only once at the
beginning.

So my choices now are to overload the addTransformer() method, only =
adding a
transformer under certain circumstances, or leave my current =
implementation
which goes through this.transformers, this.transformerSources and
this.transformerParams, checks the parameters and throws the =
transformers,
sources and parameters out of the lists.
The first way seems easier, but it means that I have to copy the whole
method from the super class to my pipeline and change it there - I don't
like that because you I have to check for changes with every Cocoon =
update.
With the second way I can simply outsource the code going through the =
lists
to my own method and call it at the beginning of setupPipeline().

I do have one more question - to make it right this time. If I prevent a
transformer from being added to the list in addTransformer(), or if I =
kick
it out in my method, I see I have to release it. This is done by the
following in AbstractProcessingPipeline:

int size =3D this.transformerSelectors.size();
for (int i =3D 0; i < size; i++) {
  final ComponentSelector selector =3D (ComponentSelector)
this.transformerSelectors.get(i);
  selector.release((Component) this.transformers.get(i));
  this.newManager.release(selector);
}

In my method it would look like:

Iterator transformerItt =3D this.transformers.iterator();
...
int i =3D 0;
while (transformerItt.hasNext()) {
      Transformer trans =3D (Transformer) transformerItt.next();
	...
	// check if the transformer should be run
	..
	// kick it out
      transformerItt.remove();
	// release transformer
	final ComponentSelector selector =3D (ComponentSelector)
this.transformerSelectors.get(i);
	  selector.release((Component) trans);
	this.newManager.release(selector);
	i++;
}    =20

Right?

Stefan                  =20

| -----Urspr=FCngliche Nachricht-----
| Von: Sylvain Wallez [mailto:sylvain@apache.org]
| Gesendet: Montag, 1. Mai 2006 00:01
| An: dev@cocoon.apache.org
| Betreff: Re: AW: Desparately seeking help with pipeline implementation
|=20
| Stefan Pietschmann wrote:
| > | -----Urspr=FCngliche Nachricht-----
| > | Von: Sylvain Wallez [mailto:sylvain@apache.org]
| > | Gesendet: Sonntag, 30. April 2006 14:00
| > | An: dev@cocoon.apache.org
| > | Betreff: Re: Desparately seeking help with pipeline implementation
| > |
| > | Stefan Pietschmann wrote:
| > | >
| > | > Hi guys,
| > | >
| > | > for my thesis I have implemented a custom pipeline, which =
modifies
| the
| > | > list of transformers to be run during pipeline setup. This is =
done
| > | > with every request in setupPipeline(), so I need to reset to the
| > | > original every time, before I modify again.
| > | >
| > |
| > | Why do you need that? Can't it be done by overloading one of
| > | setGenerator/addTransformer/setSerializer?
| >
| > I adapt the pipeline with every request according to some external =
data.
| > Isn't addTransformer() called only once when the pipeline is built? =
If
| it
| > is, I couldn't use it, because this would mean I adapt the pipeline
| once,
| > and all subsequent requests use the adapted one.
| > However, if this.transformers (and the other ArrayLists with the =
sources
| and
| > parameters) is built anew for every request, then this might be an =
easy
| way
| > out.
| >
|=20
| The contents of a pipeline is not reused across requests! All =
components
| are releases and cleared in recycle(), which occurs once the pipeline
| has been executed.
|=20
| The setGenerator/addTransformer/setSerializer are called by the =
sitemap
| engine when executing <map:generate>, <map:transform> and
| <map:serialize> respectively.
|=20
| This is probably a good place if you want to automatically add some
| components to the pipeline, particularly considering that pipeline =
cache
| key and validity computation happens after these methods. Now if you
| remove some of the components that were already in the pipeline, don't
| forget to release them or you may encounter some memory leaks in =
object
| pools (see the code in AbstractProcessingPipeline)

So my=20
|=20
| > | Giving more details about the errors you obtain would be useful, =
but I
| > | guess this is more related to component pools than to =
multithreading.
| >
| > I can attach some stacktraces if that helps...
| >
|=20
| Yes, it always helps.
|=20
| > I only see that I only get problems when using multiple threads. The
| problem
| > might be related to anything else - I don't seem to understand how a
| > pipeline is used by multiple threads. I thought - since it's =
recycleable
| -
| > that each thread gets its own pipeline object.
| >
|=20
| Yes, pipelines are not shared among threads. But if you do some =
"nasty"
| things with the pipeline components, it may happen that the pipeline =
is
| not be correctly recycled and therefore components used by multiple
| threads...
|=20
| Sylvain
|=20
| --
| Sylvain Wallez - http://bluxte.net