lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (Commented) (JIRA)" <>
Subject [jira] [Commented] (SOLR-3307) DIH FileListEntityProcessor not multi-threading after applying patch SOLR-3011
Date Tue, 03 Apr 2012 13:22:24 GMT


Robert Muir commented on SOLR-3307:

Now its getting more difficult because when loading a large amount of records each index segment
gets a ".del" file of different size.
Nevertheless all data is loaded without loss and an optimize will clean up everything.
Obviously another bad side effect of multi-threading and NOT seen with version 3.5.
Will be hard to find out what happens.
An overlapping of solrwriter?

Are you sure its not just because DIH no longer optimizes itself by default in 3.6?
* SOLR-3142: Imports no longer default optimize to true, instead false. If you want to force
all segments to be merged
             into one, you can specify this parameter yourself. NOTE: this can be very expensive
operation and usually
             does not make sense for delta-imports.

Of course if you are seeing a lot of .dels after importing data, it sounds like you have
some kind of impedence mismatch (duplicate unique ids) in your source data... 

> DIH FileListEntityProcessor not multi-threading after applying patch SOLR-3011
> ------------------------------------------------------------------------------
>                 Key: SOLR-3307
>                 URL:
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.6
>            Reporter: Bernd Fehling
>            Assignee: James Dyer
>             Fix For: 3.6
>         Attachments: SOLR-3307-UnitTest.patch, SOLR-3307.patch
> As reported in issue SOLR-3011 the FileListEntityProcessor is not recursing through all
sub-directories and files after applying SOLR-3011.patch.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message