lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Smith (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-1342) 64bit JVM crashes on Linux
Date Tue, 18 Nov 2008 21:33:44 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12648768#action_12648768
] 

Paul Smith commented on LUCENE-1342:
------------------------------------

java version "1.6.0_10"
Java(TM) SE Runtime Environment (build 1.6.0_10-b33)
Java HotSpot(TM) 64-Bit Server VM (build 11.0-b15, mixed mode)

For clarity, there's 2 Paul's, myself included, and Alison here on the discussion thread,
all from Aconex (we're all talking about the same problem at the same company, but are sharing
in the discussion based on different analysis we're doing.

We've recently upgraded to using Lucene 2.2 from 2.0 (yes, way behind, but we're cautious
here..), and about 4 days from going into production with it.  

First off, an observation.  The original bug report here was reported against Lucene 2.0,
which we've been using in production for nearly 2 years against a few different JVM's (Java
1.5, plus a few builds of Java 1.6 up to and including 1.6.04).  We've never encountered this
in production or in our load test area using Lucene 2.0.  However as soon as we switched to
Lucene 2.2, using the same JRE as production (1.6.04), we started seeing these problems. 
After reviewing another HotSpot crash bug (LUCENE-1282) we decided to see if JRE 1.6.010 made
a difference.  Initially it did, we didn't find a problem with several load testing runs and
we thought we were fine.  Then a few weeks later, we started to see it occurring more frequently,
yet none of the code changes in our application since the initial 1.6.010 switch could logically
be connected to the indexing system at all (our application is spilt between an App, and an
Index/Search server, and the SVN diff between the load testing tag runs didn't have any code
change that was Indexer/Search related).

At the same time we had a strange network problem going on in the load testing area that was
causing problems with the App talking to the Indexer, which was caused by a local DNS problem.
 Inexplicably the JRE crash hasn't happened that I'm aware of; how that is related to the
JRE hotspot compilation of Lucene byte-code, I have no idea.. BUT, since we had several weeks
of stability and then several crashes, this is purely anecdotal/coincidental.  I'm still rubbing
my rabbits foot here.  I need to chat with Alison & Paul Cowan about this to get more
specific details about if/when the crash has occurred since the DNS problem was resolved,
because it could purely be a statistical anomaly (we simply may not have done many runs to
flush it out), and frankly I could be mistaken in the # crashes in the load testing env.

For incremental indexing (which is what is happening during the load test that crashes) we
are using compound file format, merge factor =default(10), minMergeDocs=200, maxMergeDocs=Default(MAX_INT).
it's pretty vanilla really.. (the reason for a low mergeFactor is that we have several hundred
indexes open at the same time for different projects, so open file handles becomes a problem).

I'll let Alison/Paul Cowan comment further, this is just my 5 Aussie cents worth.



> 64bit JVM crashes on Linux
> --------------------------
>
>                 Key: LUCENE-1342
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1342
>             Project: Lucene - Java
>          Issue Type: Bug
>    Affects Versions: 2.0.0
>         Environment: 2.6.18-53.el5 x86_64  GNU/Linux
> Java(TM) SE Runtime Environment (build 1.6.0_04-b12)
>            Reporter: Kevin Richards
>
> Whilst running lucene in our QA environment we received the following exception. This
problem was also reported here : http://confluence.atlassian.com/display/KB/JSP-20240+-+POSSIBLE+64+bit+JDK+1.6+update+4+may+have+HotSpot+problems.
> Is this a JVM problem or a problem in Lucene.
> #
> # An unexpected error has been detected by Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00002aaaaadb9e3f, pid=2275, tid=1085356352
> #
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (10.0-b19 mixed mode linux-amd64)
> # Problematic frame:
> # V  [libjvm.so+0x1fce3f]
> #
> # If you would like to submit a bug report, please visit:
> #   http://java.sun.com/webapps/bugreport/crash.jsp
> #
> ---------------  T H R E A D  ---------------
> Current thread (0x00002aab0007f000):  JavaThread "CompilerThread0" daemon [_thread_in_vm,
id=2301, stack(0x0000000040a13000,0x0000000040b14000)]
> siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x0000000000000000
> Registers:
> RAX=0x0000000000000000, RBX=0x00002aab0007f000, RCX=0x0000000000000000, RDX=0x00002aab00309aa0
> RSP=0x0000000040b10f60, RBP=0x0000000040b10fb0, RSI=0x00002aaab37d1ce8, RDI=0x00002aaaaaaad000
> R8 =0x00002aaaab40cd88, R9 =0x0000000000000ffc, R10=0x00002aaaab40cd90, R11=0x00002aaaab410810
> R12=0x00002aab00ae60b0, R13=0x00002aab0a19cc30, R14=0x0000000040b112f0, R15=0x00002aab00ae60b0
> RIP=0x00002aaaaadb9e3f, EFL=0x0000000000010246, CSGSFS=0x0000000000000033, ERR=0x0000000000000004
>   TRAPNO=0x000000000000000e
> Top of Stack: (sp=0x0000000040b10f60)
> 0x0000000040b10f60:   00002aab0007f000 0000000000000000
> 0x0000000040b10f70:   00002aab0a19cc30 0000000000000001
> 0x0000000040b10f80:   00002aab0007f000 0000000000000000
> 0x0000000040b10f90:   0000000040b10fe0 00002aab0a19cc30
> 0x0000000040b10fa0:   00002aab0a19cc30 00002aab00ae60b0
> 0x0000000040b10fb0:   0000000040b10fe0 00002aaaaae9c2e4
> 0x0000000040b10fc0:   00002aaaab413210 00002aaaab413350
> 0x0000000040b10fd0:   0000000040b112f0 00002aab09796260
> 0x0000000040b10fe0:   0000000040b110e0 00002aaaaae9d7d8
> 0x0000000040b10ff0:   00002aaaab40f3d0 00002aab08c2a4c8
> 0x0000000040b11000:   0000000040b11940 00002aab09796260
> 0x0000000040b11010:   00002aab09795b28 0000000000000000
> 0x0000000040b11020:   00002aab08c2a4c8 00002aab009b9750
> 0x0000000040b11030:   00002aab09796260 0000000040b11940
> 0x0000000040b11040:   00002aaaab40f3d0 0000000000002023
> 0x0000000040b11050:   0000000040b11940 00002aab09796260
> 0x0000000040b11060:   0000000040b11090 00002aaaab0f199e
> 0x0000000040b11070:   0000000040b11978 00002aab08c2a458
> 0x0000000040b11080:   00002aaaab413210 0000000000002023
> 0x0000000040b11090:   0000000040b110e0 00002aaaab0f1fcf
> 0x0000000040b110a0:   0000000000002023 00002aab09796260
> 0x0000000040b110b0:   00002aab08c2a3c8 0000000040b123b0
> 0x0000000040b110c0:   00002aab08c2a458 0000000040b112f0
> 0x0000000040b110d0:   00002aaaab40f3d0 00002aab00043670
> 0x0000000040b110e0:   0000000040b11160 00002aaaab0e808d
> 0x0000000040b110f0:   00002aab000417c0 00002aab009b66a8
> 0x0000000040b11100:   0000000000000000 00002aab009b9750
> 0x0000000040b11110:   0000000040b112f0 00002aab009bb360
> 0x0000000040b11120:   0000000000000003 0000000040b113d0
> 0x0000000040b11130:   01002aab0052d0c0 0000000040b113d0
> 0x0000000040b11140:   00000000000000b3 0000000040b112f0
> 0x0000000040b11150:   0000000040b113d0 00002aab08c2a108 
> Instructions: (pc=0x00002aaaaadb9e3f)
> 0x00002aaaaadb9e2f:   48 89 5d b0 49 8b 55 08 49 8b 4c 24 08 48 8b 32
> 0x00002aaaaadb9e3f:   4c 8b 21 8b 4e 1c 49 8d 7c 24 10 89 cb 4a 39 34 
> Stack: [0x0000000040a13000,0x0000000040b14000],  sp=0x0000000040b10f60,  free space=1015k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
> V  [libjvm.so+0x1fce3f]
> V  [libjvm.so+0x2df2e4]
> V  [libjvm.so+0x2e07d8]
> V  [libjvm.so+0x52b08d]
> V  [libjvm.so+0x524914]
> V  [libjvm.so+0x51c0ea]
> V  [libjvm.so+0x519f77]
> V  [libjvm.so+0x519e7c]
> V  [libjvm.so+0x519ad5]
> V  [libjvm.so+0x1e0cf4]
> V  [libjvm.so+0x2a0bc0]
> V  [libjvm.so+0x528e03]
> V  [libjvm.so+0x51c0ea]
> V  [libjvm.so+0x519f77]
> V  [libjvm.so+0x519e7c]
> V  [libjvm.so+0x519ad5]
> V  [libjvm.so+0x1e0cf4]
> V  [libjvm.so+0x240eba]
> V  [libjvm.so+0x1e05c7]
> V  [libjvm.so+0x248ec8]
> V  [libjvm.so+0x248866]
> V  [libjvm.so+0x62a3f9]
> V  [libjvm.so+0x6246a1]
> V  [libjvm.so+0x505eea]
> Current CompileTask:
> C2:2408  !   org.apache.lucene.index.DocumentWriter.invertDocument(Lorg/apache/lucene/document/Document;)V
(482 bytes)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message