incubator-droids-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thorsten Scherler <thorsten.scherler....@juntadeandalucia.es>
Subject Re: HandlerFactory fails with multithreaded implementation
Date Mon, 09 Nov 2009 12:37:00 GMT
On Fri, 2009-11-06 at 14:29 +0100, Javier Puerto wrote:
> Hi, I'm working with Droids and made some URL crawlers to save a lot of web
> pages in disk. In JUnit test, I run a little http server and crawl 20 pages,
> the most times everything works ok but in rare cases I get an error. I found
> the problem in the HandlerFactory implementation, in the example the call to
> handlers is like this:
> 
> protected void handle(ContentEntity entity, Link link)
>     throws DroidsException, IOException
> {
>   droid.getHandlerFactory().handle(link.getURI(), entity);
> }
> 
> 
> If two or more workers is trying to handle at same time, the HandlerFactory
> will handle the all with the same instance of the handler. The solution
> could be saving memory or improving performance.
> 
> The first solution could be implemented adding a "synchronized" to
> HandlerFactory.handle like this.
> 
> public synchronized boolean handle(URI uri, ContentEntity entity)
>     throws DroidsException, IOException {
>   for (Handler handler : getMap().values()) {
>     handler.handle(uri, entity);
>   }
>   return true;
> }
> 
> Only one handler to share with all workers but this solution is a
> performance killer. The other approx should be the opposite, each worker had
> his own instance of
> the handlerfactory or handler.
> 
> Solution that you think might be more appropriate?

It depends on the usecase I guess. However I think the second option is
the more common solution. 

salu2

> 
> Salu2.
-- 
Thorsten Scherler <thorsten.at.apache.org>
Open Source Java <consulting, training and solutions>

Sociedad Andaluza para el Desarrollo de la Sociedad 
de la InformaciĆ³n, S.A.U. (SADESI)





Mime
View raw message