Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of jbellis@gmail.com designates
 209.85.215.170 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAAnh3_-SzayNR51gQ=0KvhaRJ8pkT1MhrQsVFBmu3LTgToxXqQ@mail.gmail.com>
References: 
 <CAAnh3__3dukJuD7-FPPtNDsmHOifqt0B6yexBY+yQ2xWU50s4A@mail.gmail.com>
 <CAAnh3_8tZOkE=pF4LH=Faod9W=+vXNHi+Jv7izO5vxqAsJ3OKA@mail.gmail.com>
 <CALdd-zh3E=rf9y6L-WNuHzaTcK=mGprmTawHqwcaENY83=shPA@mail.gmail.com>
 <CAAnh3_8UHVvUUJLTDLwV5sfrE2X=or=kbyZh-b3R1WDNDM1XWQ@mail.gmail.com>
 <CAAnh3_-1o6y_jE8RSFf1SeAaQLv9zSu5-n3c6f0F6t_i4krzaQ@mail.gmail.com>
 <CAKkz8Q2pHNqOe+keG2tL9U3ZWnGpRdd+T60See9SL_hXxRJN2g@mail.gmail.com>
 <CAAnh3_-SzayNR51gQ=0KvhaRJ8pkT1MhrQsVFBmu3LTgToxXqQ@mail.gmail.com>
From: Jonathan Ellis <jbellis@gmail.com>
Date: Sun, 11 Sep 2011 17:35:39 -0500
Message-ID: 
 <CALdd-zjiMmNiZft45Q6V43vbsC09rSbC=zN93KVAeSinTqzf5w@mail.gmail.com>
Subject: Re: SIGSEGV during compaction?
To: user@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

The problem would be because we're trying to access a mappedbytebuffer
that has been unmapped, not because of OS deletion semantics.

On Sun, Sep 11, 2011 at 2:12 PM, Yang <teddyyyy123@gmail.com> wrote:
> unfortunately it appeared again, I confirmed it appeared with the
> -XX:-UseCompressedOops =A0.
>
> so, if it's due to accessing unmapped file, the problem should
> disappear if I use direct random access file, I'll try that.
>
> also if a process A (not only java) has a file mmapped, and another
> process deletes the file, I thought Unix should keep the file open for
> A, no ?? =A0if this is true, then we shouldn't get a SEGV due to forcing
> unmmap ????
>
>
> Thanks
> Yang
> On Thu, Sep 8, 2011 at 12:07 AM, Sylvain Lebresne <sylvain@datastax.com> =
wrote:
>> Are you using current trunk ? Or 0.8 ?
>>
>> Because if on trunk, a SIGSEGV could also be due to CASSANDRA-2521,
>> if we happen to force the unmapping of a file but tries to access it aft=
erwards
>> (which shouldn't happen but ...).
>>
>> --
>> Sylvain
>>
>> On Thu, Sep 8, 2011 at 7:36 AM, Yang <teddyyyy123@gmail.com> wrote:
>>> hmmmm, all other things remaining the same, I put jna.jar into classpat=
h,
>>> now it successfully completed a compaction without problems
>>>
>>> On Wed, Sep 7, 2011 at 10:06 PM, Yang <teddyyyy123@gmail.com> wrote:
>>>> thanks Jonathan.
>>>>
>>>> I tried openJdk too, same , filed bug to both Oracle and openJdk
>>>>
>>>>
>>>> tried -XX:-UseCompressedOops , same SEGV
>>>>
>>>> Oracle bug site asks "does it appear with -server and -Xint", I tried
>>>> these options, so far no SEGV yet, maybe slower, but haven't measured
>>>> exactly
>>>>
>>>>
>>>>
>>>> On Wed, Sep 7, 2011 at 8:56 PM, Jonathan Ellis <jbellis@gmail.com> wro=
te:
>>>>> You should report a bug to Oracle.
>>>>>
>>>>> In the meantime you could try turning off compressed oops -- that's
>>>>> been a source of a lot of GC bugs in the past.
>>>>>
>>>>> On Wed, Sep 7, 2011 at 8:22 PM, Yang <teddyyyy123@gmail.com> wrote:
>>>>>> some info in the debug file that JVM exported:
>>>>>>
>>>>>> #
>>>>>> # A fatal error has been detected by the Java Runtime Environment:
>>>>>> #
>>>>>> # =A0SIGSEGV (0xb) at pc=3D0x00002aaaab37cbfa, pid=3D7236, tid=3D117=
9806016
>>>>>> #
>>>>>> # JRE version: 6.0_27-b07
>>>>>> # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.2-b06 mixed mode
>>>>>> linux-amd64 compressed oops)
>>>>>> # Problematic frame:
>>>>>> # J =A0com.cgm.whisky.filter.WithinLimitIpFrequencyCap.isValid(Lorg/=
apache/avro/specific/SpecificRecord;)Lcom/cgm/whisky/EventsFilter$ValidityC=
ode;
>>>>>> #
>>>>>> # If you would like to submit a bug report, please visit:
>>>>>> # =A0 http://java.sun.com/webapps/bugreport/crash.jsp
>>>>>> #
>>>>>>
>>>>>> --------------- =A0T H R E A D =A0---------------
>>>>>>
>>>>>> Current thread (0x00002aaab80e2800): =A0JavaThread "pool-3-thread-8"
>>>>>> [_thread_in_Java, id=3D7669,
>>>>>> stack(0x0000000046426000,0x0000000046527000)]
>>>>>>
>>>>>> siginfo:si_signo=3DSIGSEGV: si_errno=3D0, si_code=3D1 (SEGV_MAPERR),
>>>>>> si_addr=3D0x00002aaabc000000
>>>>>>
>>>>>> Registers:
>>>>>> RAX=3D0x00000007914355e8, RBX=3D0x000000000000058a,
>>>>>> RCX=3D0x0000000791461b38, RDX=3D0x0000000000000000
>>>>>> RSP=3D0x00000000465259f0, RBP=3D0x00000000f222b894,
>>>>>> RSI=3D0x0000000791433f20, RDI=3D0x00002aaaab37ca60
>>>>>> R8 =3D0x00000000d0931f61, R9 =3D0x00000000f2286ab2,
>>>>>> R10=3D0x0000000000000000, R11=3D0x00002aaabc000000
>>>>>> R12=3D0x0000000000000000, R13=3D0x00000000465259f0,
>>>>>> R14=3D0x0000000000000002, R15=3D0x00002aaab80e2800
>>>>>> RIP=3D0x00002aaaab37cbfa, EFLAGS=3D0x0000000000010202,
>>>>>> CSGSFS=3D0x0100000000000033, ERR=3D0x0000000000000004
>>>>>> =A0TRAPNO=3D0x000000000000000e
>>>>>>
>>>>>> Top of Stack: (sp=3D0x00000000465259f0)
>>>>>> 0x00000000465259f0: =A0 000000068a828dc8 0000000791433f20
>>>>>> 0x0000000046525a00: =A0 000000079145ee60 000005890000058a
>>>>>>
>>>>>>
>>>>>> On Wed, Sep 7, 2011 at 6:21 PM, Yang <teddyyyy123@gmail.com> wrote:
>>>>>>> I started compaction using nodetool,
>>>>>>> then always reproducibly, I get a SEGV in a code that I added to th=
e
>>>>>>> Cassandra code, which simply calls get_slice().
>>>>>>>
>>>>>>> have you seen SEGV associated with compaction? anyone could suggest=
 a
>>>>>>> route on how to debug this?
>>>>>>>
>>>>>>> I filed a bug on sun website, right now the only possible approach =
I
>>>>>>> can try is to use another JDK
>>>>>>>
>>>>>>>
>>>>>>> Thanks
>>>>>>> Yang
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Jonathan Ellis
>>>>> Project Chair, Apache Cassandra
>>>>> co-founder of DataStax, the source for professional Cassandra support
>>>>> http://www.datastax.com
>>>>>
>>>>
>>>
>>
>


--=20
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com