ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan V." <iveselovs...@gridgain.com>
Subject Re: JVM Crash - SIGSEGV on GridUnsafe.copyMemory
Date Wed, 19 Oct 2016 16:27:15 GMT
Looks similar to this issue: https://github.com/netty/netty/issues/2950 .

On Wed, Oct 19, 2016 at 2:26 PM, Vladislav Pyatkov <vldpyatkov@gmail.com>
wrote:

> Hi,
>
> This look like in the system does not have enough memory.
>
> It is difficult to say anything else, without additional information.
> Memory consumption, system logs for the moment.
>
> On Wed, Oct 19, 2016 at 2:00 PM, yfernando <yohan.fernando@tudor.com>
> wrote:
>
>> Hi,
>>
>> We've had the multiple instances where a 16 node grid completely crashes
>> with the following error.
>>
>> we use all our caches OFF HEAP caches in our configuration.
>>
>> We are running Ignite 1.7 and running Java 1.8u60-b27 on Linux
>>
>> Has anyone seen a similar issue?
>>
>> Thanks
>>
>> Yohan
>>
>> I've attached a few of the hs_err log files. Please let us know if you
>> need
>> anymore information.
>>
>> #
>> # A fatal error has been detected by the Java Runtime Environment:
>> #
>> #  SIGSEGV (0xb) at pc=0x00007ff13f1c2524, pid=3402054,
>> tid=140670445545216
>> #
>> # JRE version: Java(TM) SE Runtime Environment (8.0_60-b27) (build
>> 1.8.0_60-b27)
>> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.60-b23 mixed mode
>> linux-amd64 compressed oops)
>> # Problematic frame:
>> # v  ~StubRoutines::jint_disjoint_arraycopy
>> #
>> # Core dump written. Default location:
>> /prod/Java/apps/data-grid-server-ignite/2.2.31/bin/core or core.3402054
>> #
>> # If you would like to submit a bug report, please visit:
>> #   http://bugreport.java.com/bugreport/crash.jsp
>> #
>>
>> ---------------  T H R E A D  ---------------
>>
>> Current thread (0x00007ff1552f9800):  JavaThread
>> "pub-#12%DataGridServer-Production%" [_thread_in_Java, id=3402569,
>> stack(0x00007ff063dfe000,0x00007ff063eff000)]
>>
>> siginfo: si_signo: 11 (SIGSEGV), si_code: 2 (SEGV_ACCERR), si_addr:
>> 0x000000076a800000
>>
>>
>> Stack: [0x00007ff063dfe000,0x00007ff063eff000],  sp=0x00007ff063efc490,
>> free space=1017k
>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
>> code)
>> v  ~StubRoutines::jint_disjoint_arraycopy
>> J 6572 C2
>> org.apache.ignite.internal.util.GridUnsafe.copyMemory(Ljava/
>> lang/Object;JLjava/lang/Object;JJ)V
>> (14 bytes) @ 0x00007ff13f52f1a8 [0x00007ff13f52f180+0x28]
>> j
>> org.apache.ignite.internal.util.io.GridUnsafeDataInput.readS
>> hortArray()[S+88
>> j
>> org.apache.ignite.marshaller.optimized.OptimizedObjectInputS
>> tream.readShortArray()[S+4
>> J 25716 C2
>> org.apache.ignite.marshaller.optimized.OptimizedObjectInputS
>> tream.readObjectOverride()Ljava/lang/Object;
>> (883 bytes) @ 0x00007ff1437bbc4c [0x00007ff1437ba2a0+0x19ac]
>> J 6523 C2 java.io.ObjectInputStream.readObject()Ljava/lang/Object; (124
>> bytes) @ 0x00007ff13f4e0d9c [0x00007ff13f4e0760+0x63c]
>> J 12272 C2
>> org.apache.ignite.marshaller.optimized.OptimizedObjectInputS
>> tream.readFields(Ljava/lang/Object;Lorg/apache/ignite/
>> marshaller/optimized/OptimizedClassDescriptor$ClassFields;)V
>> (336 bytes) @ 0x00007ff140833d78 [0x00007ff140833980+0x3f8]
>> J 12508 C2
>> org.apache.ignite.marshaller.optimized.OptimizedObjectInputS
>> tream.readSerializable(Ljava/lang/Class;Ljava/util/List;
>> Ljava/lang/reflect/Method;Lorg/apache/ignite/marshaller/
>> optimized/OptimizedClassDescriptor$Fields;)Ljava/lang/Object;
>> (176 bytes) @ 0x00007ff14178ee74 [0x00007ff14178e780+0x6f4]
>> J 16693 C2
>> org.apache.ignite.marshaller.optimized.OptimizedClassDescrip
>> tor.read(Lorg/apache/ignite/marshaller/optimized/Optimized
>> ObjectInputStream;)Ljava/lang/Object;
>> (130 bytes) @ 0x00007ff140804868 [0x00007ff1408044a0+0x3c8]
>> J 25716 C2
>> org.apache.ignite.marshaller.optimized.OptimizedObjectInputS
>> tream.readObjectOverride()Ljava/lang/Object;
>> (883 bytes) @ 0x00007ff1437bbe80 [0x00007ff1437ba2a0+0x1be0]
>> J 6523 C2 java.io.ObjectInputStream.readObject()Ljava/lang/Object; (124
>> bytes) @ 0x00007ff13f4e0d9c [0x00007ff13f4e0760+0x63c]
>> J 12272 C2
>> org.apache.ignite.marshaller.optimized.OptimizedObjectInputS
>> tream.readFields(Ljava/lang/Object;Lorg/apache/ignite/
>> marshaller/optimized/OptimizedClassDescriptor$ClassFields;)V
>> (336 bytes) @ 0x00007ff140833d78 [0x00007ff140833980+0x3f8]
>> J 12508 C2
>> org.apache.ignite.marshaller.optimized.OptimizedObjectInputS
>> tream.readSerializable(Ljava/lang/Class;Ljava/util/List;
>> Ljava/lang/reflect/Method;Lorg/apache/ignite/marshaller/
>> optimized/OptimizedClassDescriptor$Fields;)Ljava/lang/Object;
>> (176 bytes) @ 0x00007ff14178ee74 [0x00007ff14178e780+0x6f4]
>> J 16693 C2
>> org.apache.ignite.marshaller.optimized.OptimizedClassDescrip
>> tor.read(Lorg/apache/ignite/marshaller/optimized/Optimized
>> ObjectInputStream;)Ljava/lang/Object;
>> (130 bytes) @ 0x00007ff140804868 [0x00007ff1408044a0+0x3c8]
>> J 25716 C2
>> org.apache.ignite.marshaller.optimized.OptimizedObjectInputS
>> tream.readObjectOverride()Ljava/lang/Object;
>> (883 bytes) @ 0x00007ff1437bbe80 [0x00007ff1437ba2a0+0x1be0]
>> J 6560 C2
>> org.apache.ignite.marshaller.optimized.OptimizedObjectInputS
>> tream.readArrayList()Ljava/util/ArrayList;
>> (47 bytes) @ 0x00007ff140621050 [0x00007ff140620c80+0x3d0]
>> J 25716 C2
>> org.apache.ignite.marshaller.optimized.OptimizedObjectInputS
>> tream.readObjectOverride()Ljava/lang/Object;
>> (883 bytes) @ 0x00007ff1437bbbac [0x00007ff1437ba2a0+0x190c]
>> J 6523 C2 java.io.ObjectInputStream.readObject()Ljava/lang/Object; (124
>> bytes) @ 0x00007ff13f4e0d9c [0x00007ff13f4e0760+0x63c]
>> J 12272 C2
>> org.apache.ignite.marshaller.optimized.OptimizedObjectInputS
>> tream.readFields(Ljava/lang/Object;Lorg/apache/ignite/
>> marshaller/optimized/OptimizedClassDescriptor$ClassFields;)V
>> (336 bytes) @ 0x00007ff140833d78 [0x00007ff140833980+0x3f8]
>> J 12508 C2
>> org.apache.ignite.marshaller.optimized.OptimizedObjectInputS
>> tream.readSerializable(Ljava/lang/Class;Ljava/util/List;
>> Ljava/lang/reflect/Method;Lorg/apache/ignite/marshaller/
>> optimized/OptimizedClassDescriptor$Fields;)Ljava/lang/Object;
>> (176 bytes) @ 0x00007ff14178ee74 [0x00007ff14178e780+0x6f4]
>> J 16693 C2
>> org.apache.ignite.marshaller.optimized.OptimizedClassDescrip
>> tor.read(Lorg/apache/ignite/marshaller/optimized/Optimized
>> ObjectInputStream;)Ljava/lang/Object;
>> (130 bytes) @ 0x00007ff140804868 [0x00007ff1408044a0+0x3c8]
>> J 25716 C2
>> org.apache.ignite.marshaller.optimized.OptimizedObjectInputS
>> tream.readObjectOverride()Ljava/lang/Object;
>> (883 bytes) @ 0x00007ff1437bbe80 [0x00007ff1437ba2a0+0x1be0]
>> J 6523 C2 java.io.ObjectInputStream.readObject()Ljava/lang/Object; (124
>> bytes) @ 0x00007ff13f4e0d9c [0x00007ff13f4e0760+0x63c]
>> J 12272 C2
>> org.apache.ignite.marshaller.optimized.OptimizedObjectInputS
>> tream.readFields(Ljava/lang/Object;Lorg/apache/ignite/
>> marshaller/optimized/OptimizedClassDescriptor$ClassFields;)V
>> (336 bytes) @ 0x00007ff140833d78 [0x00007ff140833980+0x3f8]
>> J 12508 C2
>> org.apache.ignite.marshaller.optimized.OptimizedObjectInputS
>> tream.readSerializable(Ljava/lang/Class;Ljava/util/List;
>> Ljava/lang/reflect/Method;Lorg/apache/ignite/marshaller/
>> optimized/OptimizedClassDescriptor$Fields;)Ljava/lang/Object;
>> (176 bytes) @ 0x00007ff14178ee74 [0x00007ff14178e780+0x6f4]
>> J 16693 C2
>> org.apache.ignite.marshaller.optimized.OptimizedClassDescrip
>> tor.read(Lorg/apache/ignite/marshaller/optimized/Optimized
>> ObjectInputStream;)Ljava/lang/Object;
>> (130 bytes) @ 0x00007ff140804868 [0x00007ff1408044a0+0x3c8]
>> J 25716 C2
>> org.apache.ignite.marshaller.optimized.OptimizedObjectInputS
>> tream.readObjectOverride()Ljava/lang/Object;
>> (883 bytes) @ 0x00007ff1437bbe80 [0x00007ff1437ba2a0+0x1be0]
>> J 6523 C2 java.io.ObjectInputStream.readObject()Ljava/lang/Object; (124
>> bytes) @ 0x00007ff13f4e0d9c [0x00007ff13f4e0760+0x63c]
>> J 25841 C2
>> org.apache.ignite.marshaller.optimized.OptimizedMarshaller.u
>> nmarshal([BLjava/lang/ClassLoader;)Ljava/lang/Object;
>> (148 bytes) @ 0x00007ff140c27724 [0x00007ff140c27300+0x424]
>> J 25109 C1
>> org.apache.ignite.internal.processors.job.GridJobWorker.init
>> ialize(Lorg/apache/ignite/internal/managers/deployment/GridD
>> eployment;Ljava/lang/Class;)Z
>> (295 bytes) @ 0x00007ff141e6a83c [0x00007ff141e6a500+0x33c]
>> J 24444 C1
>> org.apache.ignite.internal.processors.job.GridJobProcessor.p
>> rocessJobExecuteRequest(Lorg/apache/ignite/cluster/ClusterNo
>> de;Lorg/apache/ignite/internal/GridJobExecuteRequest;)V
>> (1342 bytes) @ 0x00007ff1434879ac [0x00007ff143481280+0x672c]
>> J 24443 C1
>> org.apache.ignite.internal.processors.job.GridJobProcessor$J
>> obExecutionListener.onMessage(Ljava/util/UUID;Ljava/lang/Object;)V
>> (147 bytes) @ 0x00007ff140d08ab4 [0x00007ff140d075a0+0x1514]
>> J 14472 C2
>> org.apache.ignite.internal.managers.communication.GridIoMana
>> ger.processRegularMessage0(Lorg/apache/ignite/internal/ma
>> nagers/communication/GridIoMessage;Ljava/util/UUID;)V
>> (55 bytes) @ 0x00007ff141ddc8e4 [0x00007ff141ddc5e0+0x304]
>> J 11641 C2
>> org.apache.ignite.internal.managers.communication.GridIoManager$5.run()V
>> (52
>> bytes) @ 0x00007ff1414b02cc [0x00007ff1414b0240+0x8c]
>> j
>> java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/
>> util/concurrent/ThreadPoolExecutor$Worker;)V+95
>> j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+19
>> j  java.lang.Thread.run()V+11
>> v  ~StubRoutines::call_stub
>> V  [libjvm.so+0x68bbe6]  JavaCalls::call_helper(JavaValue*,
>> methodHandle*,
>> JavaCallArguments*, Thread*)+0x1056
>> V  [libjvm.so+0x68c0f1]  JavaCalls::call_virtual(JavaValue*, KlassHandle,
>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x321
>> V  [libjvm.so+0x68c597]  JavaCalls::call_virtual(JavaValue*, Handle,
>> KlassHandle, Symbol*, Symbol*, Thread*)+0x47
>> V  [libjvm.so+0x7232d0]  thread_entry(JavaThread*, Thread*)+0xa0
>> V  [libjvm.so+0xa68f3f]  JavaThread::thread_main_inner()+0xdf
>> V  [libjvm.so+0xa6906c]  JavaThread::run()+0x11c
>> V  [libjvm.so+0x91cb88]  java_start(Thread*)+0x108
>> C  [libpthread.so.0+0x7aa1]
>> hs_err_pid3410800.log
>> <http://apache-ignite-users.70518.x6.nabble.com/file/n8356/
>> hs_err_pid3410800.log>
>> hs_err_pid3410096.log
>> <http://apache-ignite-users.70518.x6.nabble.com/file/n8356/
>> hs_err_pid3410096.log>
>> hs_err_pid3408462.log
>> <http://apache-ignite-users.70518.x6.nabble.com/file/n8356/
>> hs_err_pid3408462.log>
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/JVM-Crash-SIGSEGV-on-GridUnsafe-copyMemory-tp8356.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>
>
> --
> Vladislav Pyatkov
>

Mime
View raw message