incubator-droids-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Javier Puerto <jpue...@gmail.com>
Subject HandlerFactory fails with multithreaded implementation
Date Fri, 06 Nov 2009 13:29:46 GMT
Hi, I'm working with Droids and made some URL crawlers to save a lot of web
pages in disk. In JUnit test, I run a little http server and crawl 20 pages,
the most times everything works ok but in rare cases I get an error. I found
the problem in the HandlerFactory implementation, in the example the call to
handlers is like this:

protected void handle(ContentEntity entity, Link link)
    throws DroidsException, IOException
{
  droid.getHandlerFactory().handle(link.getURI(), entity);
}


If two or more workers is trying to handle at same time, the HandlerFactory
will handle the all with the same instance of the handler. The solution
could be saving memory or improving performance.

The first solution could be implemented adding a "synchronized" to
HandlerFactory.handle like this.

public synchronized boolean handle(URI uri, ContentEntity entity)
    throws DroidsException, IOException {
  for (Handler handler : getMap().values()) {
    handler.handle(uri, entity);
  }
  return true;
}

Only one handler to share with all workers but this solution is a
performance killer. The other approx should be the opposite, each worker had
his own instance of
the handlerfactory or handler.

Solution that you think might be more appropriate?

Salu2.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message