lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fuad Efendi (JIRA)" <>
Subject [jira] Commented: (SOLR-2233) DataImportHandler - JdbcDataSource is not thread safe
Date Sun, 14 Nov 2010 15:53:13 GMT


Fuad Efendi commented on SOLR-2233:

It is 3 times faster "after" I applied changes:
Before: 729 documents/minute
After: 2639 documents/minute

In my test, with 10 sub-entities some of them are multi-valued (and hard to use CachedJdbcDataSource
for composite PKs). I can't explain it by only threads="16" option (which this patch makes
possible). It is probably "Connection Close / Connection Open" issue which is very expensive
for SQL-Server (except MySQL JDBC driver which internally uses connection pooling)

> DataImportHandler - JdbcDataSource is not thread safe
> -----------------------------------------------------
>                 Key: SOLR-2233
>                 URL:
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 1.5
>            Reporter: Fuad Efendi
>         Attachments: FE-patch.txt, SOLR-2233-JdbcDataSource.patch, SOLR-2233-JdbcDataSource.patch
> Whenever Thread A spends more than 10 seconds on a Connection (by retrieving records
in a batch), Thread B will close connection.
> Related exceptions happen when we use "threads=" attribute for entity; usually exception
stack contains message "connection already closed"
> It shouldn't happen with some JNDI data source, where Connection.close() simply returns
Connection to a pool of available connections, but we might get different errors.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message