Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Wed, 6 Jun 2012 19:34:23 +0000 (UTC)
From: "Aaron Simmons (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: <566299353.44581.1339011263091.JavaMail.jiratomcat@issues-vm>
In-Reply-To: <991611645.13989.1337817100927.JavaMail.jiratomcat@issues-vm>
Subject: [jira] [Commented] (SOLR-3484) LogUpdateProcessor throws
 ConcurrentModificationException under multi-threading calls
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/SOLR-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290348#comment-13290348 ] 

Aaron Simmons commented on SOLR-3484:
-------------------------------------

I have read the related tickets [SOLR-2694, SOLR-2804, SOLR-2694] .. and I still think this is a bug in LogUpdateProcessor, or maybe SolrWriter's usage of LogUpdateProcessor?!?

My environment is a Solr Cloud instance with 21 nodes.  (solr-spec-version=4.0.0.2012.03.14.16.31.07)

When we kicked off a regular full-import of a data source (~5 million rows) .. it starts off fine, but then the import crashes.  Looking over the log files, I find this on one of the nodes:

INFO: [core_NAME_slice11_shard1] webapp=/solr path=/select params={command=status&qt=/dataimport} status=0 QTime=1
Jun 5, 2012 7:24:22 PM org.apache.solr.handler.dataimport.SolrWriter upload

WARNING: Error creating document : SolrInputDocument[{ --LOTS OF DATA HERE-- }]
java.util.ConcurrentModificationException
        at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
        at java.util.HashMap$KeyIterator.next(HashMap.java:828)
        at org.apache.solr.common.util.IteratorChain.next(IteratorChain.hava:73)
        at org.apache.solr.common.params.ModifiableSolrParams.add(ModifiableSolrParams.java:116)
        at org.apache.solr.common.params.ModifiableSolrParams.<init>(ModifiableSolrParams.java:56)
        at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:268)
        at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessor.java:115)
        at org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:70)
        at org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:289)
        at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:722)
        at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:618)
        at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:324)
        at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:222)
        at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:382)
        at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:450)
        at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:431)

We are using the 'out of the box' DataImportHandler, connecting to a SQL Server, etc.  This seems like a real issue/bug that in org.apache.solr.update.processor.LogUpdateProcessor.

I saw the patch from SOLR-2804 on 1/1/2012 ... and this seems like a valid fix.  But some people made comments and indicated it was not a valid fix, perhaps communicating that it's not a real problem?  And SOLR-2804 was closed with "Won't Fix" ... and this bug report is marked as "Not a Problem".

I see this as a very real problem .. but maybe I'm not understanding something.  Thoughts anyone?
                
> LogUpdateProcessor throws ConcurrentModificationException under multi-threading calls 
> --------------------------------------------------------------------------------------
>
>                 Key: SOLR-3484
>                 URL: https://issues.apache.org/jira/browse/SOLR-3484
>             Project: Solr
>          Issue Type: Bug
>          Components: update
>    Affects Versions: 4.0
>         Environment: linux
>            Reporter: Ethan Tao
>
> Using the LogUpdateProcessor in a singleton chain for concurrent processing throws exception. The issue has been reported in SOLR-2694 (closed), and an unoffical patch can be found in related bug-id SOLR-2804 patch dated 1/1/12.
> If the patch won't become official for LogUpdateProcessor, suggested to have new class "ConcurrentLogUpdateProcessorFactory" to address the thread safe issue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org