lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kameron Cole" <kameronc...@us.ibm.com>
Subject RE: MappedByteBuffer duplicates
Date Fri, 24 Feb 2017 19:31:57 GMT
Actually, at a certain point, they have crashed the machine. The native 
file mappings are deallocated (unmapped) by the JVM when the 
MappedByteBuffers are eligible for garbage collection. The problem we're 
seeing  is that there are thousands of MappedByteBuffers which are not 
eligible for garbage collection. The native memory is retained because the 
Lucene code is still referencing the MappedByteBuffer objects on the Java 
heap. This isn't the fault of Windows or the JVM. It appears to be a fault 
in Lucen, but we can't diagnose it - we can't see why the MappedByteBuffer 
objects are being retained.





From:   "Uwe Schindler" <uwe@thetaphi.de>
To:     <java-user@lucene.apache.org>
Date:   02/24/2017 01:39 PM
Subject:        RE: MappedByteBuffer duplicates



Hi,

that is not an issue, the duplicates are required for so called IndexInput 
clones and splices. Every search request will create many of them. But 
there is no need to worry, they are just thin wrappers - they don't 
allocate any extra off-heap memory. They are just there to have a separate 
position(), limit() and other settings for each searcher thread.

Why do you worry?
Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Kameron Cole [mailto:kameroncole@us.ibm.com]
> Sent: Friday, February 24, 2017 7:19 PM
> To: java-user@lucene.apache.org
> Subject: MappedByteBuffer duplicates
> 
> We have a Lucene engine that creates MappedByteBuffer objects when
> creating the Lucene index.  I don't know Lucene well enough to know if
> this standard behavior.
> 
> The mapped files are being created by Lucene, via the JRE's NIO APIs
> native file mapping underneath each MappedByteBuffer object. We see an
> issue where duplicate MappedByteBuffer objects are being created.  Has
> anyone seen this?
> 
> Thank you!
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org






Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message