lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Busch (JIRA)" <>
Subject [jira] Commented: (LUCENE-669) finalize()-methods of FSDirectory.FSIndexInput and FSDirectory.FSIndexOutput try to close already closed file
Date Tue, 28 Nov 2006 08:27:23 GMT
    [ ] 
Michael Busch commented on LUCENE-669:


you are right, it is very weird that 1) this problem happens and 2) my patch fixes it. It
took me quite a while to figure out what the real problem is but I think I have at least a
guess now. 

The problem again only happens on an IBM JVM. It happens rarely with your unmodified TestStressIndexing.
If I change this testcase so that is does not perform the stress test on a RAMDirectory but
only on a FSDirectory it happens almost always. This is VERY weird, because I did not change
the stress test at all... I just commented the lines

//    // First in a RAM directory:
//    Directory directory = new RAMDirectory();
//    runStressTest(directory);
//    directory.close();

and suddenly every test fails with the IOException "The handle is invalid". So that makes
me think it has something to do with garbage collection and the finalize() methods.

Now if I just comment out the finalize() method of FSDirectory.FSIndexOutput the test passes.
This is even stranger, because the IOException occurred in FSDirectory.FSIndexInput.close(),
not in FSIndexOutput, so in a different class which uses hence a different file descriptor.

So I checked how is implemented: It simply wraps an int value. I can't
see how these int values are computed, because the open() method of RandomAccessFile is native.

So I believe the following happens:
(1) FSIndexOutput uses a FileDescriptor with value x
(2) FSIndexOutput.close() is called, so the underlying file is being closed
(3) A new FSIndexInput instance is created by a searcher thread. This opens a RandomAccessFile.
Because FileDescriptor x is not in use anymore, x is used as the value for the new FileDescriptor.
(4) Now garbage collection kicks in. It removes the old instance of FSIndexOutput for which
close() has been called already. So the garbage collector calls finalize() which calls RandomAccessFile.close()
again which still uses the descriptor with value x. So this call of close() actually closes
the file used by the IndexInput instance created in (3).
(5) FSIndexInput.close() is called and tries to close the file which has been closed already
in (4) and thus the IOException occurs.

So it seems to me that the IBM JVM makes file descriptor values available after a file has
been closed, whereas Sun waits until the FileDescriptor instance is destroyed. This might
be a bug in the JVM, but since this patch is very simple we could just use it to be on the
safe side.

Do you think this makes sense? Or does anybody have a better idea why commenting out the finalize()
method in FSIndexOutput prevents FSIndexInput.close() from throwing the IOException? 

> finalize()-methods of FSDirectory.FSIndexInput and FSDirectory.FSIndexOutput try to close
already closed file
> -------------------------------------------------------------------------------------------------------------
>                 Key: LUCENE-669
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Store
>            Reporter: Michael Busch
>         Assigned To: Michael Busch
>            Priority: Trivial
>         Attachments: FSDirectory_close_file2.patch
> Hi all,
> I found a small problem in FSDirectory: The finalize()-methods of FSDirectory.FSIndexInput
and FSDirectory.FSIndexOutput try to close the underlying file. This is not a problem unless
the file has been closed before by calling the close() method. If it has been closed before,
the finalize method throws an IOException saying that the file is already closed. Usually
this IOException would go unnoticed, because the GarbageCollector, which calls finalize(),
just eats it. However, if I use the Eclipse debugger the execution of my code will always
be suspended when this exception is thrown.
> Even though this exception probably won't cause problems during normal execution of Lucene,
the code becomes cleaner if we apply this small patch. Might this IOException also have a
performance impact, if it is thrown very frequently?
> I attached the patch which applies cleanly on the current svn HEAD. All testcases pass
and I verfied with the Eclipse debugger that the IOException is not longer thrown.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message