lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Simmons (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-3484) LogUpdateProcessor throws ConcurrentModificationException under multi-threading calls
Date Wed, 06 Jun 2012 19:34:23 GMT

    [ https://issues.apache.org/jira/browse/SOLR-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290348#comment-13290348
] 

Aaron Simmons commented on SOLR-3484:
-------------------------------------

I have read the related tickets [SOLR-2694, SOLR-2804, SOLR-2694] .. and I still think this
is a bug in LogUpdateProcessor, or maybe SolrWriter's usage of LogUpdateProcessor?!?

My environment is a Solr Cloud instance with 21 nodes.  (solr-spec-version=4.0.0.2012.03.14.16.31.07)

When we kicked off a regular full-import of a data source (~5 million rows) .. it starts off
fine, but then the import crashes.  Looking over the log files, I find this on one of the
nodes:

INFO: [core_NAME_slice11_shard1] webapp=/solr path=/select params={command=status&qt=/dataimport}
status=0 QTime=1
Jun 5, 2012 7:24:22 PM org.apache.solr.handler.dataimport.SolrWriter upload

WARNING: Error creating document : SolrInputDocument[{ --LOTS OF DATA HERE-- }]
java.util.ConcurrentModificationException
        at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
        at java.util.HashMap$KeyIterator.next(HashMap.java:828)
        at org.apache.solr.common.util.IteratorChain.next(IteratorChain.hava:73)
        at org.apache.solr.common.params.ModifiableSolrParams.add(ModifiableSolrParams.java:116)
        at org.apache.solr.common.params.ModifiableSolrParams.<init>(ModifiableSolrParams.java:56)
        at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:268)
        at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessor.java:115)
        at org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:70)
        at org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:289)
        at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:722)
        at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:618)
        at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:324)
        at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:222)
        at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:382)
        at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:450)
        at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:431)

We are using the 'out of the box' DataImportHandler, connecting to a SQL Server, etc.  This
seems like a real issue/bug that in org.apache.solr.update.processor.LogUpdateProcessor.

I saw the patch from SOLR-2804 on 1/1/2012 ... and this seems like a valid fix.  But some
people made comments and indicated it was not a valid fix, perhaps communicating that it's
not a real problem?  And SOLR-2804 was closed with "Won't Fix" ... and this bug report is
marked as "Not a Problem".

I see this as a very real problem .. but maybe I'm not understanding something.  Thoughts
anyone?
                
> LogUpdateProcessor throws ConcurrentModificationException under multi-threading calls

> --------------------------------------------------------------------------------------
>
>                 Key: SOLR-3484
>                 URL: https://issues.apache.org/jira/browse/SOLR-3484
>             Project: Solr
>          Issue Type: Bug
>          Components: update
>    Affects Versions: 4.0
>         Environment: linux
>            Reporter: Ethan Tao
>
> Using the LogUpdateProcessor in a singleton chain for concurrent processing throws exception.
The issue has been reported in SOLR-2694 (closed), and an unoffical patch can be found in
related bug-id SOLR-2804 patch dated 1/1/12.
> If the patch won't become official for LogUpdateProcessor, suggested to have new class
"ConcurrentLogUpdateProcessorFactory" to address the thread safe issue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message