lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Miller (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-3621) Fix concurrency race around newIndexWriter
Date Tue, 17 Jul 2012 14:03:33 GMT

    [ https://issues.apache.org/jira/browse/SOLR-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416208#comment-13416208
] 

Mark Miller commented on SOLR-3621:
-----------------------------------

I think this is related to the fail that kept happening in DIH tests previously (I was actually
hoping this would help fix that test and is why I started looking at this the other day again).

Part of the problem with that test may be that it uses RAMDir though - I don't think that
in Solr that is compatible with booting a new IndexWriter - which happens during rollback,
which happens in some error situations with DIH. I think that's the "no segments file found
issue" that sometimes fails.

Anyhow, my best current guess for this case is that opening a new writer fails, causes an
exception, the getWriter lock is never released, someone tries to get a writer, they wait
forever. I'm changing to release that lock in a finally so hopefully the true error is more
visible and there is no hang.

The other fix may be to hard code fsdir for these tests that might rollback - but I'll wait
on that.
                
> Fix concurrency race around newIndexWriter 
> -------------------------------------------
>
>                 Key: SOLR-3621
>                 URL: https://issues.apache.org/jira/browse/SOLR-3621
>             Project: Solr
>          Issue Type: Bug
>          Components: update
>            Reporter: Mark Miller
>            Assignee: Mark Miller
>            Priority: Minor
>             Fix For: 4.0, 5.0
>
>         Attachments: SOLR--3621.patch
>
>
> When I did the first big refactor on update handler, I was trying to never close the
index writer - I had to give in on this goal due to the replication handler - it requires
rebooting the indexwriter. At the time, I settled for allowing a little race that didn't show
up as an issue in tests - this IW reboot was always a bit of a hack in the past anyhow.
> Now that the dust has settled, we should make this air tight though. I'd like to make
opening a new indexwriter a full class citizen rather than a hacky method only used minimally
for replication to reboot things. It should be a solid API that is valid for any uses down
the road.
> For some IW config changes, we may want to do it in 'some' cases on reload.
> To do this, we have to start ref counting iw use - so that we only actually open a new
one and close the old one when it's not in use at all.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message