lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Dyer (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SOLR-2947) DIH caching bug - EntityRunner destroys child entity processor
Date Mon, 12 Dec 2011 20:03:32 GMT

     [ https://issues.apache.org/jira/browse/SOLR-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

James Dyer updated SOLR-2947:
-----------------------------

    Attachment: SOLR-2947.patch

Mikhail,

I appreciate your finding this bug and providing a fix.  Indeed I wasn't sure I was handling
"destroy()" correctly for the multi-threaded case in SOLR-2382 but then again all tests were
passing.

I think you're absolutely correct we don't have enough threaded tests.  To help with this,
I expanded "TestEphemeralCache" (see updated patch) to run it 3 times: first single-threaded,
second single-threaded with 'threads=1', and third as truly multi-threaded (2-6 threads randomly
chosen).

Obviously the third case still fails but you indicate you plan on fixing that one next?  :)
 That would be great if you could.  Let me know if there is any way I can help although I
admit I'm a bit unsure of myself in the DIH multi-threaded code.  Is it your impression that
DIH caching ever worked threaded, or do you think SOLR-2382 might have broken it?

I also agree that it shouldn't use the "threaded" code if "threads=1"  For now, it provides
us with a nice testing scenario to fix the 'destroy()' semantics without having to tackle
the multi-threaded cacheing beast.

Although I'm with you that it would be great if we could just combine the single-threaded
and multi-threaded cases into the same code.  But I would think before that can happen all
of these threaded bugs need to be worked through.

I apologize for taking so long to look at this.  On the other hand, I'm not going to be much
help to you in getting this committed.  But if I can help in any other way let me know.
                
> DIH caching bug - EntityRunner destroys child entity processor
> --------------------------------------------------------------
>
>                 Key: SOLR-2947
>                 URL: https://issues.apache.org/jira/browse/SOLR-2947
>             Project: Solr
>          Issue Type: Sub-task
>          Components: contrib - DataImportHandler
>    Affects Versions: 4.0
>            Reporter: Mikhail Khludnev
>              Labels: noob
>             Fix For: 4.0
>
>         Attachments: SOLR-2947.patch, SOLR-2947.patch, dih-cache-destroy-on-threads-fix.patch,
dih-cache-threads-enabling-bug.patch
>
>
> My intention is fix multithread import with SQL cache. Here is the 2nd stage. If I enable
DocBuilder.EntityRunner flow even for single thread, it breaks the pretty basic functionality:
parent-child join.
> the reason is [line 473 entityProcessor.destroy();|http://svn.apache.org/viewvc/lucene/dev/trunk/solr/contrib/dataimporthandler/src/java/org/apache/solr/handler/dataimport/DocBuilder.java?revision=1201659&view=markup]
breaks children entityProcessor.
> see attachement comments for more details. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message