Return-Path: X-Original-To: apmail-flink-user-archive@minotaur.apache.org Delivered-To: apmail-flink-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3148517443 for ; Mon, 28 Sep 2015 09:24:42 +0000 (UTC) Received: (qmail 21163 invoked by uid 500); 28 Sep 2015 09:24:39 -0000 Delivered-To: apmail-flink-user-archive@flink.apache.org Received: (qmail 21084 invoked by uid 500); 28 Sep 2015 09:24:39 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 21074 invoked by uid 99); 28 Sep 2015 09:24:38 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Sep 2015 09:24:38 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 4F1151A1F50 for ; Mon, 28 Sep 2015 09:24:38 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.2 X-Spam-Level: ** X-Spam-Status: No, score=2.2 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id oCHfQjXCZSYc for ; Mon, 28 Sep 2015 09:24:31 +0000 (UTC) Received: from mail-wi0-f174.google.com (mail-wi0-f174.google.com [209.85.212.174]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTPS id E3906204DF for ; Mon, 28 Sep 2015 09:24:30 +0000 (UTC) Received: by wicfx3 with SMTP id fx3so91768582wic.0 for ; Mon, 28 Sep 2015 02:24:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=3/SDsrq4sYy+NFioS0RBGOIC/Nz9rUY9jj5V+F9eLzA=; b=aEfGtIt4eNJ3spooC+Vtxd8S59iXia5Uu6B0yjvKLeM4FHvOpr2JCeYBDQkfKeph4l aBrDd/dEBNQ4z0IxtC/B065w5aEgq9v7HbcvH+Bb/js6ttqiSIkwy7teX+qLth+trZKk k/AVaZpYbHluiYtHW2RwqTwZPIzj+ZYZKk86DwVLSMRMjbYkDON1mhSEjjXXx9Z31yqO 3SGtqshhk3bYmgJB2rfXf5h6aDVvD8gM54ADtxbnCF9hXCIkdJO3ZW1KPAn8yQkvf+w3 oCubem3Ui8BS/YVoHtadfedWdW4wQlmhKqV4V/ETzqQKSzKavxEhVpFJydyGjKp7qbO5 TgHA== MIME-Version: 1.0 X-Received: by 10.180.188.33 with SMTP id fx1mr17929642wic.31.1443432270641; Mon, 28 Sep 2015 02:24:30 -0700 (PDT) Received: by 10.28.139.69 with HTTP; Mon, 28 Sep 2015 02:24:30 -0700 (PDT) In-Reply-To: References: Date: Mon, 28 Sep 2015 11:24:30 +0200 Message-ID: Subject: Re: JVM core dump running Flink streaming From: Till Rohrmann To: user@flink.apache.org Content-Type: multipart/alternative; boundary=001a11c26956bfbf000520cb435a --001a11c26956bfbf000520cb435a Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable When the CMS garbage collector is used we additionally set -XX:+CMSClassUnloadingEnabled. Everything else are the default values. =E2=80=8B On Mon, Sep 28, 2015 at 11:22 AM, Jakob Ericsson wrote: > Yes. I just wanted to know if you have any preferred settings for CMS or > should I use fairly standard configs? > > On Mon, Sep 28, 2015 at 11:19 AM, Till Rohrmann > wrote: > >> Hi Jakob, >> >> you can control the used garbage collector by specifying the >> env.java.opts configuration parameter in your flink-conf.yaml. Here you >> can basically specify all the JVM options which will be given to the JVM= s. >> Thus, in your case you could try the following settings: env.java.opts: >> -XX:+UseConcMarkSweepGC. >> >> Cheers, >> Till >> =E2=80=8B >> >> On Mon, Sep 28, 2015 at 10:15 AM, Jakob Ericsson < >> jakob.ericsson@gmail.com> wrote: >> >>> Hi, >>> >>> I'm testing Flink streaming but seems to have some problems with jvm >>> core dumps. >>> I haven't really looked at the heap dump yet. >>> It seems to be related to the G1 GC. >>> >>> If I want to go back to CMS-GC, do you have any preferred settings? >>> >>> >>> >>> >>> # >>> # A fatal error has been detected by the Java Runtime Environment: >>> # >>> # SIGSEGV (0xb) at pc=3D0x00007fa281df900d, pid=3D36541, tid=3D1403356= 55700224 >>> # >>> # JRE version: Java(TM) SE Runtime Environment (8.0_45-b14) (build >>> 1.8.0_45-b14) >>> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.45-b02 mixed mode >>> linux-amd64 compressed oops) >>> # Problematic frame: >>> # V [libjvm.so+0x5b900d] >>> G1ParScanThreadState::copy_to_survivor_space(oopDesc*)+0x3d >>> # >>> # Failed to write core dump. Core dumps have been disabled. To enable >>> core dumping, try "ulimit -c unlimited" before starting Java again >>> # >>> # If you would like to submit a bug report, please visit: >>> # http://bugreport.java.com/bugreport/crash.jsp >>> # >>> >>> --------------- T H R E A D --------------- >>> >>> Current thread (0x00007fa26c00f800): GCTaskThread [stack: >>> 0x00007fa270d84000,0x00007fa270e85000] [id=3D36559] >>> >>> siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: >>> 0x0000000000000008 >>> >>> Registers: >>> RAX=3D0x00007fa2827b03b0, RBX=3D0x00007fa2827b03a0, RCX=3D0x00000000000= 00003, >>> RDX=3D0x0000000000000001 >>> RSP=3D0x00007fa270e83490, RBP=3D0x00007fa270e83520, RSI=3D0x00000000dc6= e5530, >>> RDI=3D0x0000000000000000 >>> R8 =3D0x00000000dc6e5530, R9 =3D0x00007fa282929ba7, R10=3D0x00000000c49= 742a8, >>> R11=3D0x00007fa2822b2e20 >>> R12=3D0x000000008c579484, R13=3D0x00000000dc6e5530, R14=3D0x00007fa270e= 83780, >>> R15=3D0x00007fa270e83aa0 >>> RIP=3D0x00007fa281df900d, EFLAGS=3D0x0000000000010246, >>> CSGSFS=3D0x0000000000000033, ERR=3D0x0000000000000004 >>> TRAPNO=3D0x000000000000000e >>> >>> Top of Stack: (sp=3D0x00007fa270e83490) >>> 0x00007fa270e83490: 00007fa270e834f0 00000000c49742c0 >>> 0x00007fa270e834a0: 00007fa270e83780 000000008210a22f >>> 0x00007fa270e834b0: 000000000000000b 00007fa26c333110 >>> 0x00007fa270e834c0: 0000000700000001 0000000000000001 >>> 0x00007fa270e834d0: 000000000257e760 000000000257e760 >>> 0x00007fa270e834e0: 00007fa270e83790 00007fa23c37da70 >>> 0x00007fa270e834f0: 00007fa270e83520 00007fa2827b03a0 >>> 0x00007fa270e83500: 000000008c579484 00007fa26c0f7e20 >>> 0x00007fa270e83510: 00007fa270e83780 00007fa270e83aa0 >>> ... >>> ... >>> ... >>> >>> >>> >> > --001a11c26956bfbf000520cb435a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

When the CMS garbage collector is used = we additionally set -XX:+CMSClassUnloadingEnabled. = Everything else are the default values.

=E2=80=8B
=

On Mon, Sep= 28, 2015 at 11:22 AM, Jakob Ericsson <jakob.ericsson@gmail.com= > wrote:
Yes. = I just wanted to know if you have any preferred settings for CMS or should = I use fairly standard configs?

On Mon, Sep 28, = 2015 at 11:19 AM, Till Rohrmann <trohrmann@apache.org> wr= ote:

Hi Jakob,

you can control the used garbag= e collector by specifying the env.java.opts configu= ration parameter in your flink-conf.yaml. Here you = can basically specify all the JVM options which will be given to the JVMs. = Thus, in your case you could try the following settings: e= nv.java.opts: -XX:+UseConcMarkSweepGC.

Cheers,
Till

=E2=80=8B

On Mon, Sep 28, 2015= at 10:15 AM, Jakob Ericsson <jakob.ericsson@gmail.com> wrote:
Hi,

I'm testing Flink streaming but seems to have some problems wi= th jvm core dumps.
I haven't really looked at the heap dump y= et.
It seems to be related to the G1 GC.

If I want to go back to CMS-GC, do you have any preferred settings?
<= div>



#
# A fatal error has been detected by the Java Runtime Environment:
#
# =C2=A0SIGSEGV (0xb) at pc=3D0x00007fa281df900d, pid= =3D36541, tid=3D140335655700224
#
# JRE version: Java(T= M) SE Runtime Environment (8.0_45-b14) (build 1.8.0_45-b14)
# Jav= a VM: Java HotSpot(TM) 64-Bit Server VM (25.45-b02 mixed mode linux-amd64 c= ompressed oops)
# Problematic frame:
# V =C2=A0[libjvm.= so+0x5b900d] =C2=A0G1ParScanThreadState::copy_to_survivor_space(oopDesc*)+0= x3d
#
# Failed to write core dump. Core dumps have been= disabled. To enable core dumping, try "ulimit -c unlimited" befo= re starting Java again
#
# If you would like to submit = a bug report, please visit:
#

---------------= =C2=A0T H R E A D =C2=A0---------------

Current t= hread (0x00007fa26c00f800): =C2=A0GCTaskThread [stack: 0x00007fa270d84000,0= x00007fa270e85000] [id=3D36559]

siginfo: si_signo:= 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x0000000000000008
<= div>
Registers:
RAX=3D0x00007fa2827b03b0, RBX=3D0x0= 0007fa2827b03a0, RCX=3D0x0000000000000003, RDX=3D0x0000000000000001
RSP=3D0x00007fa270e83490, RBP=3D0x00007fa270e83520, RSI=3D0x00000000dc6e= 5530, RDI=3D0x0000000000000000
R8 =3D0x00000000dc6e5530, R9 =3D0x= 00007fa282929ba7, R10=3D0x00000000c49742a8, R11=3D0x00007fa2822b2e20
<= div>R12=3D0x000000008c579484, R13=3D0x00000000dc6e5530, R14=3D0x00007fa270e= 83780, R15=3D0x00007fa270e83aa0
RIP=3D0x00007fa281df900d, EFLAGS= =3D0x0000000000010246, CSGSFS=3D0x0000000000000033, ERR=3D0x000000000000000= 4
=C2=A0 TRAPNO=3D0x000000000000000e

Top= of Stack: (sp=3D0x00007fa270e83490)
0x00007fa270e83490: =C2=A0 0= 0007fa270e834f0 00000000c49742c0
0x00007fa270e834a0: =C2=A0 00007= fa270e83780 000000008210a22f
0x00007fa270e834b0: =C2=A0 000000000= 000000b 00007fa26c333110
0x00007fa270e834c0: =C2=A0 0000000700000= 001 0000000000000001
0x00007fa270e834d0: =C2=A0 000000000257e760 = 000000000257e760
0x00007fa270e834e0: =C2=A0 00007fa270e83790 0000= 7fa23c37da70
0x00007fa270e834f0: =C2=A0 00007fa270e83520 00007fa2= 827b03a0
0x00007fa270e83500: =C2=A0 000000008c579484 00007fa26c0f= 7e20
0x00007fa270e83510: =C2=A0 00007fa270e83780 00007fa270e83aa0=
...
...
...

<= br>



--001a11c26956bfbf000520cb435a--