lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bernd Fehling (Commented) (JIRA)" <>
Subject [jira] [Commented] (SOLR-3011) DIH MultiThreaded bug
Date Fri, 30 Mar 2012 13:02:26 GMT


Bernd Fehling commented on SOLR-3011:

Just tried multi-threaded. It produces the required number of threads (seen in debugger) but
only runs once.
My configuration is:
        <dataSource name="filetraverser" type="FileDataSource" encoding="UTF-8" />
        <entity name="basedata" processor="FileListEntityProcessor"
                <entity name="records" processor="XPathEntityProcessor"
                        <field column="id"      xpath="/documents/document/@id" />
                        <field column="dctitle" xpath="/documents/document/element[@name='dctitle']/value"

It should read all files below baseDir and build documents from the records inside the files.
Works fine in non-multi-threaded but only reads the first file in multi-threaded mode.
Any idea?

And another thing to mention, in there are the lines:
  public void testCachedThreadless_FullImport() throws Exception {
    runFullImport(getCachedConfig(random.nextBoolean(), random.nextBoolean(), 0));
  public void testCachedSingleThread_FullImport() throws Exception {
    runFullImport(getCachedConfig(random.nextBoolean(), random.nextBoolean(), 1));
  public void testCachedThread_FullImport() throws Exception {
    int numThreads = random.nextInt(9) + 1; // between one and 10
    String config = getCachedConfig(random.nextBoolean(), random.nextBoolean(), numThreads);

This will test 0, 1 and random between 1 to 9. But 1 is already covered.
So wouldn't it be better to have "random.nextInt(8) + 2" for the range 2 to 9?

> DIH MultiThreaded bug
> ---------------------
>                 Key: SOLR-3011
>                 URL:
>             Project: Solr
>          Issue Type: Sub-task
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.5
>            Reporter: Mikhail Khludnev
>            Assignee: James Dyer
>            Priority: Minor
>             Fix For: 3.6
>         Attachments: SOLR-3011.patch, SOLR-3011.patch, SOLR-3011.patch, SOLR-3011.patch,
SOLR-3011.patch, patch-3011-EntityProcessorBase-iterator.patch, patch-3011-EntityProcessorBase-iterator.patch
> current DIH design is not thread safe. see last comments at SOLR-2382 and SOLR-2947.
I'm going to provide the patch makes DIH core threadsafe. Mostly it's a SOLR-2947 patch from
28th Dec. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message