lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roberto Fonti <roberto.fo...@gmail.com>
Subject Re: JVM Crash org.apache.lucene.store.DataInput.readVInt()I
Date Tue, 22 Nov 2011 20:08:55 GMT
Thanks Uwe and Simon and for your detailed answers.

I'm reviewing my code to find where I'm using a closed index.

Roberto


On Nov 22, 2011, at 11:53 AM, Uwe Schindler wrote:

> Hi,
> 
> The JDK issues you are talking about are also fixed in 1.6.9_29 (not only
> 1.7.0u1). So your crash is not related to any of the "famous java6/7 bugs"
> (in addition, the vint bug does *not* crash your JVM at all it just corrupts
> your index by returning wrong values - and this one is definitely fixed
> since Lucene 3.1).
> 
> But there is another chance you can crash your JVM: You are on a 64 bit
> platform (Linux), so the default directory implementation is MMapDirectory.
> Lucene uses a hack to be able to unmap mapped files from virtual address
> space. This is not allowed by the JVM itsself, but makes unmapping dependent
> on garbage collector, which is a problem for Lucene. By default
> MMapDirectory unmaps the files after closing the IndexInputs. MMapDirectory
> is not synchronized at all, so when another thread or even the same thread
> tries to access the IndexInput after unmapping it will access an unmapped
> address and will SIGSEGV.
> 
> If your code would be correct this cannot happen, but it looks like you are
> using an already closed IndexReader/IndexWriter to access the index. Before
> Lucene 3.5 (will come out soon), missing checks in IndexReader will make it
> possible that an already closed IndexReader with all its closed (and
> unmapped) IndexInputs tries to access index data and segfaults.
> 
> In 3.5 we added additional safety checks to prevent this illegal access, but
> its not 100% (as synchronization is missing for performance). I would review
> the code and check that nothing accesses closed index.
> 
> A simple check to see if this is your issue would be to use NIOFSDirectory
> (slower on Linux) instead of MMapDirectory. If it does not crash and
> possibly throws AlreadyClosedExceptions in 3.5, the bug is that you are
> accessing closed indexes.
> 
> Uwe
> 
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
> 
> 
>> -----Original Message-----
>> From: Roberto Fonti [mailto:roberto.fonti@gmail.com]
>> Sent: Tuesday, November 22, 2011 8:03 PM
>> To: java-user@lucene.apache.org
>> Subject: JVM Crash org.apache.lucene.store.DataInput.readVInt()I
>> 
>> My JVM (1.6.0_29) keeps crashing on intensive use when indexing documents
>> with Lucene. I get:
>> 
>> 
>> #
>> # A fatal error has been detected by the Java Runtime Environment:
>> #
>> #  SIGSEGV (0xb) at pc=0x00002b6b196d767c, pid=26417, tid=1183217984 # #
>> JRE version: 6.0_29-b11 # Java VM: Java HotSpot(TM) 64-Bit Server VM
> (20.4-
>> b02 mixed mode linux-amd64 compressed oops) # Problematic frame:
>> # J  org.apache.lucene.store.DataInput.readVInt()I
>> #
>> # If you would like to submit a bug report, please visit:
>> #   http://java.sun.com/webapps/bugreport/crash.jsp
>> #
>> 
>> Environment:
>> 
>> JDK: 1.6u29 (same issue with 1.6_02) Lucene Version 3.4.0
>> 
>> vm_info: Java HotSpot(TM) 64-Bit Server VM (20.4-b02) for linux-amd64 JRE
>> (1.6.0_29-b11), built on Oct 3 2011 01:19:20 by "java_re" with gcc 3.2.2
> (SuSE
>> Linux)
>> 
>> OS:CentOS release 5.0 (Final)
>> 
>> jvm_args: -Dcatalina.home=/var/local/tomcat-8081 -
>> Dcatalina.base=/var/local/tomcat-8081 -Djava.io.tmpdir=/var/tmp -
>> Dfile.encoding=UTF-8 -Xmx1024M -XX:MaxPermSize=96m
>> 
>> It seems to be a jdk issue that was fixed in jdk 1.7, but other issues
> where
>> introduced. https://issues.apache.org/jira/browse/LUCENE-3335 "Java 7
>> contains a fix to the readVInt issue since 1.6.0_21 (approx, LUCENE-2975)"
>> 
>> So, how can I fix this issue using JDK 1.6? Should I upgrade to jdk 1.7?
>> 
>> 
>> 
>> Thanks,
>> Roberto
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message