lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: Dangerous reflection access to sun.misc.Cleaner by class org.apache.lucene.store.MMapDirectory$MMapIndexInput$1 detected!
Date Mon, 03 Nov 2014 22:30:11 GMT
Hi Jean-Claude,

> We have a NetBeans RCP application using Lucene 4.7 for indexing and
> searching.
> 
> When indexing, the following message is displayed:
> 
> Dangerous reflection access to sun.misc.Cleaner by class
> org.apache.lucene.store.MMapDirectory$MMapIndexInput$1 detected!

This message is not created by Apache Lucene, it must be some instrumentation/safety library
in your classpath. Lucene's fast disk i/o layer uses memory mapping and needs to call sun.misc.Cleaner
to unmap memory mapped byte buffers "in time" when index files are closed. This is implemented
in a safe way in Lucene, so there is no real risk in it (unless you get unlucky and access
an already closed IndexReader from many threads - in most cases this is detected, but may
cause a SIGSEGV of the JVM). If the client code is implemented in a correct way, this is no
problem at all (otherwise servers like Apache Solr or Elasticsearch would not use it :-)

In conclusion: The warning is correct - its risky, but only if you don't know what you do
- but Lucene knows :-) So you can safely ignore the warning - or better: disable this safety
instrumentation kit - it may kill performance.

If this is a client application (RCP = Rich Client?), the use of MMAP may not be wanted at
all. By default, Lucene chooses the "optimal" disk IO implementation for the platform (and
this is mmap on 64 bits), if you use FSDirectory.open(...) to open index files (this is a
static factory method). If you don't want to use MMap, use new NIOFSDirectory or new SimpleFSDirectory
instead by directly instantiating, so automatism is not used. MMap is intended to be used
for huge indexes on server-side machines, that have high concurrency and need to access the
index in a very fast way. For client applications, this is in most cases not needed.

If you want to use MMapDirectory as implementation, you can disable it's use of sun.misc.Cleaner
by calling MMapDirectory.setUseUnmap(false) [this is also only possible if you instantiate
directly, not through static factory). But this will cause it to sit on file buffers for possibly
very long time and this may be a problem on windows (where files cannot be deleted while memory
mapped) or can block disk space. The reason is delayed byte buffer deallocation by the garbage
collector.

For more information about MMapDirectory, read the javadocs and this blog post: http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html

> It does not stop indexing but it is quite annoying. Any idea on how to get rid
> of this message or should we ignore it ??

See above!

Uwe


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message