lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Dyer (Commented) (JIRA)" <>
Subject [jira] [Commented] (SOLR-3011) DIH MultiThreaded bug
Date Mon, 19 Mar 2012 14:54:38 GMT


James Dyer commented on SOLR-3011:

what is the importance of delta import scenario?  I see it can be done via full import instead
This just provides two ways you can set up your deltas.  Personally I like to use "full-import
w/ clean=false" to do deltas.  Its much more flexible.  With that said, people use the other
way and it works.  The bad thing, I think, is it is not documented that "threads" doesn't
work with it.  Also, surely the code can be cleaned up to not be an entirely different branch
from "full-import".

It's hard to maintain code ever. 
This is why I think removing "threads" in trunk is still the way to go.  This will give us
a good starting point for improving the code.  We can add add a better-designed version of
"threads" later.

With that said, I think you've probably fixed "threads" for a lot of use-cases.  Why can't
we back-port this to 3.x and document for the users what works and doesn't work with "threads",
with warnings to test thoroughly before going to production?  If it means back-porting the
cache refactorings first, so be it.  

I know you were thinking we really should start over with "Ultimate DIH".  That's fine if
people want to do that.  But I'm using the existing DIH for some pretty complex things and
it works great.  My issue with DIH is not that it isn't a good product.  Its just that it
needs some work internally if we're going to be able to continue to improve it from here.
> DIH MultiThreaded bug
> ---------------------
>                 Key: SOLR-3011
>                 URL:
>             Project: Solr
>          Issue Type: Sub-task
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.5, 4.0
>            Reporter: Mikhail Khludnev
>            Priority: Minor
>             Fix For: 4.0
>         Attachments: SOLR-3011.patch, SOLR-3011.patch, SOLR-3011.patch, SOLR-3011.patch,
patch-3011-EntityProcessorBase-iterator.patch, patch-3011-EntityProcessorBase-iterator.patch
> current DIH design is not thread safe. see last comments at SOLR-2382 and SOLR-2947.
I'm going to provide the patch makes DIH core threadsafe. Mostly it's a SOLR-2947 patch from
28th Dec. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message