lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Becker <>
Subject 2.9 - leftover (deleted) filehandles after upgrade
Date Wed, 29 Jul 2009 08:53:15 GMT
Hi everybody,

we just recently upgraded to lucene 2.9 nightly build, since we needed some of the comparator
fixes. We got ooms with some ScoreDocComparators which are now replaced with FieldComparators.
I refactored the usages of deprecated apis and changed only small things. However during index
updates, lucene now seems to leave files open.

The files are deleted, but the jvm is still holding filehandles on them:

-bash-3.2$ /usr/sbin/lsof | grep 31159| grep deleted
java      31159  asuser   72r      REG              253,3   18596499  3680841 /var/APP/ME.local/lucene/indexWriter/editorialContent/_pzd.cfs
java      31159  asuser   78r      REG              253,3 2750904146  3035091 /var/APP/ME.local/lucene/indexWriter/items/_3e.cfs
java      31159  asuser  381r      REG              253,3 2750298670  3713134 /var/APP/ME.local/lucene-local/indexWriter/items/_3c.cfs
java      31159  asuser  384r      REG              253,3   41138379  3713132 /var/APP/ME.local/lucene-local/indexWriter/contributors/_gp.cfs
java      31159  asuser  386r      REG              253,3   18596353  3713127 /var/APP/ME.local/lucene-local/indexWriter/editorialContent/_pzb.cfs
java      31159  asuser  429r      REG              253,3 2750298670  3035086 /var/APP/ME.local/lucene/indexWriter/items/_3c.cfs

Our lucene index is based on a shared index lying around on nfs. The shared index is the one
which gets all updates from our database. The lucene instances will notice the updates and
have a running
copy on a tmpfs ramdisk for performance reasons.
Since searching on the nfs index is very slow, we copy the index first to local disk and use
it temporarily for indexSearcher during update. Then we delete the ramdisk index and replace

This is the method responsible for an indexSearcher update:
	public void updateIndexSearcher() {
		if (!isReaderUp2Date() && !disableUpdates) {
			if (fileLockService.lock("locked for indexSearcher update", false)) {
				if (isIndexValid(getIndexWriterPath())) {
					try {"updateIndexSearcher: {} updating IndexSearcher", indexAlias);
						long startTime = System.currentTimeMillis();
						// Temporary we need a copy of the current local index
						FileUtils.deleteDirectory(new File(getTempIndexWriterLocalPath()));
						Directory tmpDir = File(getTempIndexWriterLocalPath()), lockFactory);
						Directory.copy(directory, tmpDir, false);"updateIndexSearcher: finished copy of {} to {}", directory, tmpDir);
						// Before switching the reference of indexSearcher we
						// create
						// the new indexSearcher in a temporary reference. Going
						// this way we guarantee that we've never a corrupt
						// indexSearcher during recreation.
						IndexSearcher indexSearcherTmp = new IndexSearcher(tmpDir, true);
						// temporary store the indexSearcher ref. for closing
						IndexSearcher tempIndexSearcher = indexSearcher;"updateIndexSearcher: Now starting to read from {}", indexSearcherTmp.getIndexReader().directory());
						indexSearcher = indexSearcherTmp;
						// Give all remaining searches a chance to finish,
						// before closing the indexSearcher. Lucene doesn't seem
						// to gracefully close IndexSearcher (at least not in
						// the current release
						try {
						} catch (InterruptedException e) {
							log.error("updateIndexSearcher: InterruptedException", e);
						if (tempIndexSearcher != null)
						localDirectory.close();"updateIndexSearcher: Now deleting {}. And starting to copy {} to {}", new
Object[] { getIndexWriterLocalPath(), tmpDir,
								localDirectory });
						FileUtils.deleteDirectory(new File(getIndexWriterLocalPath()));
						// indexSearcher is now reading from the tmp local index
						// so
						// we recreate the localDirectory
						localDirectory = File(getIndexWriterLocalPath()), lockFactory);
						Directory.copy(tmpDir, localDirectory, true);
						// and finally switch back to a new localDirectory
						tempIndexSearcher = new IndexSearcher(localDirectory, true);
						indexSearcher = tempIndexSearcher;
						tmpDir.close();"updateIndexSearcher: {} finished {}ms", indexAlias, System.currentTimeMillis()
- startTime);
						// set lastIndexSearcherUpdate to value from
						// since indexReader is now up2date
						lastIndexSearcherUpdate = getLastUpdateTimestmp();
					} catch (CorruptIndexException e) {
						log.error("updateIndexSearcher: " + indexAlias + " Index CorruptIndexException ", e);
					} catch (IOException e) {
						log.error("updateIndexSearcher: " + indexAlias + " Index: IOException ", e);
					} finally {
				} else {
					log.warn("updateIndexSearcher: {} index validation failed. Keeping old index.", indexAlias);
			} else {"updateIndexSearcher: {} Couldn't get file I assume an index update
is update of IndexSearcher", indexAlias);

As far as I can see each IndexSearcher and Directory is closed properly. It worked well with
2.4. Part of the refactoring I did which might be related is to replace:

Directory tmpDir = FSDirectory.getDirectory(getTempIndexWriterLocalPath());
Directory tmpDir = File(getTempIndexWriterLocalPath()), lockFactory);

IndexSearcher indexSearcherTmp = new IndexSearcher(tmpDir);
IndexSearcher indexSearcherTmp = new IndexSearcher(tmpDir, true);

No errors in the logfiles, no catched exceptions, etc. I'm a kinda out of ideas at the moment.
I googled and tried couple of things (IndexWriter.setUseCompoundFile(true), etc.) but didn't
find a
solution. Any help is appreciated.


Thomas Becker
Senior JEE Developer

net mobile AG
Zollhof 17
40221 Düsseldorf

Phone:    +49 211 97020-195
Fax:      +49 211 97020-949
Mobile:   +49 173 5146567 (private)

Registergericht:  Amtsgericht Düsseldorf, HRB 48022
Vorstand:         Theodor Niehues (Vorsitzender), Frank Hartmann,
                 Kai Markus Kulas, Dieter Plassmann
Vorsitzender des
Aufsichtsrates:   Dr. Michael Briem

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message