lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-2762) Don't leak deleted open file handles with pooled readers
Date Tue, 24 May 2011 22:04:47 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038847#comment-13038847
] 

Michael McCandless commented on LUCENE-2762:
--------------------------------------------

We have a test, TestNRTThreads, that stress-tests NRT to try to catch issues like this.

I've backported to 3.0.x, and was able to run it for 30 minutes with max 1024 file handles,
successfully... so if there is a file handle leak, this test failed to uncover it.

Not sure what's up... can you try to boil down your test case to a small test showing the
problem?

Note that when you incRef the searcher you have to do that sync'd somehow with your reopen
logic to prevent the final decRef from running at the same time.  (I don't think that's related
to the file handle issue but it's important to get right else you have a thread hazard).

> Don't leak deleted open file handles with pooled readers
> --------------------------------------------------------
>
>                 Key: LUCENE-2762
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2762
>             Project: Lucene - Java
>          Issue Type: Bug
>    Affects Versions: 2.9.4, 3.0.3, 3.1, 4.0
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: 2.9.4, 3.0.3, 3.1, 4.0
>
>         Attachments: LUCENE-2762.patch
>
>
> If you have CFS enabled today, and pooling is enabled (either directly
> or because you've pulled an NRT reader), IndexWriter will hold open
> SegmentReaders against the non-CFS format of each merged segment.
> So even if you close all NRT readers you've pulled from the writer,
> you'll still see file handles open against files that have been
> deleted.
> This count will not grow unbounded, since it's limited by the number
> of segments in the index, but it's still a serious problem since the
> app had turned off CFS in the first place presumably to avoid risk of
> too-many-open-files.  It's also bad because it ties up disk space
> since these files would otherwise be deleted.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message