lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wenca Petr (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-3011) DIH MultiThreaded bug
Date Thu, 01 Mar 2012 13:19:58 GMT

    [ https://issues.apache.org/jira/browse/SOLR-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220016#comment-13220016
] 

Wenca Petr commented on SOLR-3011:
----------------------------------

Hi Mikhail,
I know about 2804, I solved it by disabling logging as someone adviced (I think).

Without multithreading a was able to index about 15k documents per minute, with 4 threads
average about 45k per minute. After applying your patch it seems to me that it fell to 30k
per minute. But the number of processed documents is wrong. I have 50000 documents to be indexed.
I start a full dump, it precesses about 44k documents during the first minute, but it continues
after 50k to total 200k of processed with decreasing number of docs per minute with total
time of more than 7 minutes. After the commit the index contains 50k documents which is right.
                
> DIH MultiThreaded bug
> ---------------------
>
>                 Key: SOLR-3011
>                 URL: https://issues.apache.org/jira/browse/SOLR-3011
>             Project: Solr
>          Issue Type: Sub-task
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.5, 4.0
>            Reporter: Mikhail Khludnev
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-3011.patch, SOLR-3011.patch
>
>
> current DIH design is not thread safe. see last comments at SOLR-2382 and SOLR-2947.
I'm going to provide the patch makes DIH core threadsafe. Mostly it's a SOLR-2947 patch from
28th Dec. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message