Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Message-ID: <1385190079.1227044024668.JavaMail.jira@brutus>
Date: Tue, 18 Nov 2008 13:33:44 -0800 (PST)
From: "Paul Smith (JIRA)" <jira@apache.org>
To: java-dev@lucene.apache.org
Subject: [jira] Commented: (LUCENE-1342) 64bit JVM crashes on Linux
In-Reply-To: <1319699007.1216736851675.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/LUCENE-1342?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D126=
48768#action_12648768 ]=20

Paul Smith commented on LUCENE-1342:
------------------------------------

java version "1.6.0_10"
Java(TM) SE Runtime Environment (build 1.6.0_10-b33)
Java HotSpot(TM) 64-Bit Server VM (build 11.0-b15, mixed mode)

For clarity, there's 2 Paul's, myself included, and Alison here on the disc=
ussion thread, all from Aconex (we're all talking about the same problem at=
 the same company, but are sharing in the discussion based on different ana=
lysis we're doing.

We've recently upgraded to using Lucene 2.2 from 2.0 (yes, way behind, but =
we're cautious here..), and about 4 days from going into production with it=
. =20

First off, an observation.  The original bug report here was reported again=
st Lucene 2.0, which we've been using in production for nearly 2 years agai=
nst a few different JVM's (Java 1.5, plus a few builds of Java 1.6 up to an=
d including 1.6.04).  We've never encountered this in production or in our =
load test area using Lucene 2.0.  However as soon as we switched to Lucene =
2.2, using the same JRE as production (1.6.04), we started seeing these pro=
blems.  After reviewing another HotSpot crash bug (LUCENE-1282) we decided =
to see if JRE 1.6.010 made a difference.  Initially it did, we didn't find =
a problem with several load testing runs and we thought we were fine.  Then=
 a few weeks later, we started to see it occurring more frequently, yet non=
e of the code changes in our application since the initial 1.6.010 switch c=
ould logically be connected to the indexing system at all (our application =
is spilt between an App, and an Index/Search server, and the SVN diff betwe=
en the load testing tag runs didn't have any code change that was Indexer/S=
earch related).

At the same time we had a strange network problem going on in the load test=
ing area that was causing problems with the App talking to the Indexer, whi=
ch was caused by a local DNS problem.  Inexplicably the JRE crash hasn't ha=
ppened that I'm aware of; how that is related to the JRE hotspot compilatio=
n of Lucene byte-code, I have no idea.. BUT, since we had several weeks of =
stability and then several crashes, this is purely anecdotal/coincidental. =
 I'm still rubbing my rabbits foot here.  I need to chat with Alison & Paul=
 Cowan about this to get more specific details about if/when the crash has =
occurred since the DNS problem was resolved, because it could purely be a s=
tatistical anomaly (we simply may not have done many runs to flush it out),=
 and frankly I could be mistaken in the # crashes in the load testing env.

For incremental indexing (which is what is happening during the load test t=
hat crashes) we are using compound file format, merge factor =3Ddefault(10)=
, minMergeDocs=3D200, maxMergeDocs=3DDefault(MAX_INT). it's pretty vanilla =
really.. (the reason for a low mergeFactor is that we have several hundred =
indexes open at the same time for different projects, so open file handles =
becomes a problem).

I'll let Alison/Paul Cowan comment further, this is just my 5 Aussie cents =
worth.


> 64bit JVM crashes on Linux
> --------------------------
>
>                 Key: LUCENE-1342
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1342
>             Project: Lucene - Java
>          Issue Type: Bug
>    Affects Versions: 2.0.0
>         Environment: 2.6.18-53.el5 x86_64  GNU/Linux
> Java(TM) SE Runtime Environment (build 1.6.0_04-b12)
>            Reporter: Kevin Richards
>
> Whilst running lucene in our QA environment we received the following exc=
eption. This problem was also reported here : http://confluence.atlassian.c=
om/display/KB/JSP-20240+-+POSSIBLE+64+bit+JDK+1.6+update+4+may+have+HotSpot=
+problems.
> Is this a JVM problem or a problem in Lucene.
> #
> # An unexpected error has been detected by Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=3D0x00002aaaaadb9e3f, pid=3D2275, tid=3D1085356352
> #
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (10.0-b19 mixed mode linux-a=
md64)
> # Problematic frame:
> # V  [libjvm.so+0x1fce3f]
> #
> # If you would like to submit a bug report, please visit:
> #   http://java.sun.com/webapps/bugreport/crash.jsp
> #
> ---------------  T H R E A D  ---------------
> Current thread (0x00002aab0007f000):  JavaThread "CompilerThread0" daemon=
 [_thread_in_vm, id=3D2301, stack(0x0000000040a13000,0x0000000040b14000)]
> siginfo:si_signo=3DSIGSEGV: si_errno=3D0, si_code=3D1 (SEGV_MAPERR), si_a=
ddr=3D0x0000000000000000
> Registers:
> RAX=3D0x0000000000000000, RBX=3D0x00002aab0007f000, RCX=3D0x0000000000000=
000, RDX=3D0x00002aab00309aa0
> RSP=3D0x0000000040b10f60, RBP=3D0x0000000040b10fb0, RSI=3D0x00002aaab37d1=
ce8, RDI=3D0x00002aaaaaaad000
> R8 =3D0x00002aaaab40cd88, R9 =3D0x0000000000000ffc, R10=3D0x00002aaaab40c=
d90, R11=3D0x00002aaaab410810
> R12=3D0x00002aab00ae60b0, R13=3D0x00002aab0a19cc30, R14=3D0x0000000040b11=
2f0, R15=3D0x00002aab00ae60b0
> RIP=3D0x00002aaaaadb9e3f, EFL=3D0x0000000000010246, CSGSFS=3D0x0000000000=
000033, ERR=3D0x0000000000000004
>   TRAPNO=3D0x000000000000000e
> Top of Stack: (sp=3D0x0000000040b10f60)
> 0x0000000040b10f60:   00002aab0007f000 0000000000000000
> 0x0000000040b10f70:   00002aab0a19cc30 0000000000000001
> 0x0000000040b10f80:   00002aab0007f000 0000000000000000
> 0x0000000040b10f90:   0000000040b10fe0 00002aab0a19cc30
> 0x0000000040b10fa0:   00002aab0a19cc30 00002aab00ae60b0
> 0x0000000040b10fb0:   0000000040b10fe0 00002aaaaae9c2e4
> 0x0000000040b10fc0:   00002aaaab413210 00002aaaab413350
> 0x0000000040b10fd0:   0000000040b112f0 00002aab09796260
> 0x0000000040b10fe0:   0000000040b110e0 00002aaaaae9d7d8
> 0x0000000040b10ff0:   00002aaaab40f3d0 00002aab08c2a4c8
> 0x0000000040b11000:   0000000040b11940 00002aab09796260
> 0x0000000040b11010:   00002aab09795b28 0000000000000000
> 0x0000000040b11020:   00002aab08c2a4c8 00002aab009b9750
> 0x0000000040b11030:   00002aab09796260 0000000040b11940
> 0x0000000040b11040:   00002aaaab40f3d0 0000000000002023
> 0x0000000040b11050:   0000000040b11940 00002aab09796260
> 0x0000000040b11060:   0000000040b11090 00002aaaab0f199e
> 0x0000000040b11070:   0000000040b11978 00002aab08c2a458
> 0x0000000040b11080:   00002aaaab413210 0000000000002023
> 0x0000000040b11090:   0000000040b110e0 00002aaaab0f1fcf
> 0x0000000040b110a0:   0000000000002023 00002aab09796260
> 0x0000000040b110b0:   00002aab08c2a3c8 0000000040b123b0
> 0x0000000040b110c0:   00002aab08c2a458 0000000040b112f0
> 0x0000000040b110d0:   00002aaaab40f3d0 00002aab00043670
> 0x0000000040b110e0:   0000000040b11160 00002aaaab0e808d
> 0x0000000040b110f0:   00002aab000417c0 00002aab009b66a8
> 0x0000000040b11100:   0000000000000000 00002aab009b9750
> 0x0000000040b11110:   0000000040b112f0 00002aab009bb360
> 0x0000000040b11120:   0000000000000003 0000000040b113d0
> 0x0000000040b11130:   01002aab0052d0c0 0000000040b113d0
> 0x0000000040b11140:   00000000000000b3 0000000040b112f0
> 0x0000000040b11150:   0000000040b113d0 00002aab08c2a108=20
> Instructions: (pc=3D0x00002aaaaadb9e3f)
> 0x00002aaaaadb9e2f:   48 89 5d b0 49 8b 55 08 49 8b 4c 24 08 48 8b 32
> 0x00002aaaaadb9e3f:   4c 8b 21 8b 4e 1c 49 8d 7c 24 10 89 cb 4a 39 34=20
> Stack: [0x0000000040a13000,0x0000000040b14000],  sp=3D0x0000000040b10f60,=
  free space=3D1015k
> Native frames: (J=3Dcompiled Java code, j=3Dinterpreted, Vv=3DVM code, C=
=3Dnative code)
> V  [libjvm.so+0x1fce3f]
> V  [libjvm.so+0x2df2e4]
> V  [libjvm.so+0x2e07d8]
> V  [libjvm.so+0x52b08d]
> V  [libjvm.so+0x524914]
> V  [libjvm.so+0x51c0ea]
> V  [libjvm.so+0x519f77]
> V  [libjvm.so+0x519e7c]
> V  [libjvm.so+0x519ad5]
> V  [libjvm.so+0x1e0cf4]
> V  [libjvm.so+0x2a0bc0]
> V  [libjvm.so+0x528e03]
> V  [libjvm.so+0x51c0ea]
> V  [libjvm.so+0x519f77]
> V  [libjvm.so+0x519e7c]
> V  [libjvm.so+0x519ad5]
> V  [libjvm.so+0x1e0cf4]
> V  [libjvm.so+0x240eba]
> V  [libjvm.so+0x1e05c7]
> V  [libjvm.so+0x248ec8]
> V  [libjvm.so+0x248866]
> V  [libjvm.so+0x62a3f9]
> V  [libjvm.so+0x6246a1]
> V  [libjvm.so+0x505eea]
> Current CompileTask:
> C2:2408  !   org.apache.lucene.index.DocumentWriter.invertDocument(Lorg/a=
pache/lucene/document/Document;)V (482 bytes)

--=20
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org