lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Tucker" <>
Subject FSDirectory patch for file renaming
Date Sun, 16 Feb 2003 22:35:49 GMT
Hello all,

For quite some time, we've been experiencing intermittent indexing failures
that look similar to the following: couldn't rename to segments 
String)V(Unknown Source) 
y)V(Unknown Source)

This is caused by File.renameTo failing. From previous run-ins with that
method, it's actually fairly flaky, especially under various Windows JVMs.
The fix is pretty easy -- if File.renameTo fails, manually copy the file
from the old to new destination.

Old code (FSDirectory.renameFile): 

    if (!old.renameTo(nu))
      throw new IOException("couldn't rename " + from + " to " + to);

New code ("buf" variable is declared at top of file as null):

    // Rename the old file to the new one. Unfortunately, the renameTo()
method doesn't
    // work reliably under some JVMs. Therefore, if the rename fails, we
manually rename by
    // copying the old file to the new one.
    if (!old.renameTo(nu)) {
      try { in = new FileInputStream(old); out = new FileOutputStream(nu);
        int len;
        // See if the buf needs to be initialized. Initialization is only
done on-demand
        // since many VM's will never run into the renameTo bug and hence
shouldn't waste
        // 1K of mem for no reason.
        if (buf == null) {
          buf = new byte[1024];
        while ((len = >= 0) {
           out.write(buf, 0, len);

        // Delete the old file.
      catch (IOException ioe) {
        throw new IOException("couldn't rename " + from + " to " + to);

Advantages to this fix:

 * Makes indexing more reliable, especially under VM's where the renameTo
method doesn't always work.
 * Uses no additional memory or resources when File.renameTo works normally.


 * It always sucks to add workarounds to bugs in other libraries, but what
is one to do?
 * We've run into File.renameTo issues on multiple machines and VM versions,
but it would be good if others could confirm they've had problems as well. 

I'd suggest this fix as a good candidate for Lucene 1.3 unless others see
problems with it.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message