Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A59FB10FC5 for ; Fri, 1 Nov 2013 08:41:29 +0000 (UTC) Received: (qmail 17494 invoked by uid 500); 1 Nov 2013 08:41:25 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 17091 invoked by uid 500); 1 Nov 2013 08:41:25 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 17083 invoked by uid 99); 1 Nov 2013 08:41:24 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Nov 2013 08:41:24 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of asaf.mesika@gmail.com designates 209.85.219.51 as permitted sender) Received: from [209.85.219.51] (HELO mail-oa0-f51.google.com) (209.85.219.51) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Nov 2013 08:41:18 +0000 Received: by mail-oa0-f51.google.com with SMTP id h1so4200327oag.24 for ; Fri, 01 Nov 2013 01:40:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=MdLKiLQS0xWgmjlAykKlDFZlHVJ91LmF/7Sz7WVgzt0=; b=aPa0loaCEpE5URmHvdlfFhuljyLLnP29w+sVRUZwJ/lJ6RzMDukTv2ns1vVIWMEnks TCpTfTxMSAe89ldy1h47hBa04VMAxl+nhI6O8nT2HCOWGW5xSvfjQwAytg2geO832fHK tIeSrg4mQlhF6O9Vhx7IU84Y/GYpefW6pkChovZtolkWMnqxlqSCERQlrfV0zrWY3kf7 SBLxz+Abbg93g4qFRDn4iuxIWc/kZsENtBOULVxtco03BB9U7awAiyXnwLbnLDxr8wqL Mx9vX28XwRz9IP5YAzdrBHXMC5LIlGdCcL0RGNl435GWjbYeKD7KCIdKncfdMkXcJySw XO3w== MIME-Version: 1.0 X-Received: by 10.182.101.134 with SMTP id fg6mr1664114obb.30.1383295257038; Fri, 01 Nov 2013 01:40:57 -0700 (PDT) Received: by 10.76.159.166 with HTTP; Fri, 1 Nov 2013 01:40:56 -0700 (PDT) In-Reply-To: References: Date: Fri, 1 Nov 2013 10:40:56 +0200 Message-ID: Subject: Re: You Are Dead Exception due to promotion failure From: Asaf Mesika To: "user@hbase.apache.org" Content-Type: multipart/alternative; boundary=e89a8ff250e06a56a004ea198633 X-Virus-Checked: Checked by ClamAV on apache.org --e89a8ff250e06a56a004ea198633 Content-Type: text/plain; charset=UTF-8 Can you please explain why is this suspicious? On Monday, October 7, 2013, Jean-Daniel Cryans wrote: > This line: > > [CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 sys=3.74, > real=88.77 > secs] > > Is suspicious. Are you swapping? > > J-D > > > On Mon, Oct 7, 2013 at 8:34 AM, prakash kadel > >wrote: > > > Also, > > why is the CMS not kicking in early, i have set XX:+ > > UseCMSInitiatingOccupancyOnly??? > > > > Sincerely, > > Prakash > > > > > > On Tue, Oct 8, 2013 at 12:32 AM, prakash kadel > >wrote: > > > > > Hello, > > > I am getting this YADE all the time > > > > > > HBASE_HEAPSIZE=8000 > > > > > > Settings: -ea -XX:+UseConcMarkSweepGC -XX:MaxGCPauseMillis=200 > > > -XX:+HeapDumpOnOutOfMemoryError -XX:+CMSIncrementalMode > -XX:+UseParNewGC > > > -XX:CMSInitiatingOccupancyFraction=50 > -XX:+UseCMSInitiatingOccupancyOnly > > > -XX:NewSize=256m -XX:MaxNewSize=256m > > > > > > it seems there is promotion failure and the CMS take too long > > > > > > 2013-10-07T01:22:55.784+0900: [GC [ParNew: 235968K->26176K(235968K), > > > 0.3219980 secs] 7709485K->7538063K(8165824K) icms_dc=0 , 0.3221100 > secs] > > > [Times: user=0.27 sys=0.01, real=0.33 secs] > > > 2013-10-07T01:23:07.361+0900: [GC [ParNew: 235842K->26176K(235968K), > > > 0.1899680 secs] 7747729K->7578713K(8165824K) icms_dc=0 , 0.1900700 > secs] > > > [Times: user=0.26 sys=0.02, real=0.19 secs] > > > 2013-10-07T01:23:20.154+0900: [GC [ParNew: 235803K->26176K(235968K), > > > 0.2428200 secs] 7788341K->7615284K(8165824K) icms_dc=0 , 0.2429570 > secs] > > > [Times: user=0.25 sys=0.02, real=0.24 secs] > > > 2013-10-07T01:23:34.594+0900: [GC [ParNew: 235889K->26176K(235968K), > > > 0.2440980 secs] 7824998K->7651179K(8165824K) icms_dc=0 , 0.2442130 > secs] > > > [Times: user=0.20 sys=0.03, real=0.25 secs] > > > 2013-10-07T01:23:47.666+0900: [GC [ParNew: 235906K->26176K(235968K), > > > 0.2998100 secs] 7860909K->7686832K(8165824K) icms_dc=3 , 0.3020280 > secs] > > > [Times: user=0.23 sys=0.04, real=0.30 secs] > > > 2013-10-07T01:23:57.216+0900: [GC [1 CMS-initial-mark: > > 7660656K(7929856K)] > > > 7788778K(8165824K), 3.7665320 secs] [Times: user=0.07 sys=0.06, > real=3.77 > > > secs] > > > 2013-10-07T01:24:05.508+0900: [GC [ParNew: 235811K->26176K(235968K), > > > 0.4632860 secs] 7896468K->7721167K(8165824K) icms_dc=3 , 0.4634100 > secs] > > > [Times: user=0.21 sys=0.03, real=0.46 secs] > > > 2013-10-07T01:24:19.889+0900: [GC [ParNew: 235812K->26176K(235968K), > > > 0.3531980 secs] 7930804K->7755633K(8165824K) icms_dc=3 , 0.3533230 > secs] > > > [Times: user=0.24 sys=0.06, real=0.35 secs] > > > 2013-10-07T01:24:32.832+0900: [GC [ParNew: 235968K->26176K(235968K), > > > 0.6298370 secs] 7965425K->7790643K(8165824K) icms_dc=3 , 0.6299530 > secs] > > > [Times: user=0.23 sys=0.03, real=0.63 secs] > > > 2013-10-07T01:24:43.629+0900: [GC [ParNew: 235800K->26176K(235968K), > > > 0.3190580 secs] 8000268K->7825555K(8165824K) icms_dc=3 , 0.3191840 > secs] > > > [Times: user=0.24 sys=0.02, real=0.32 secs] > > > 2013-10-07T01:24:56.005+0900: [GC [ParNew: 235848K->26176K(235968K), > > > 0.4839400 secs] 8035228K->7860300K(8165824K) icms_dc=3 , 0.4840480 > secs] > > > [Times: user=0.31 sys=0.03, real=0.49 secs] > > > 2013-10-07T01:25:07.282+0900: [GC [ParNew: 235750K->26176K(235968K), > > > 0.3423250 secs] 8069875K->7895852K(8165824K) icms_dc=9 , 0.3424380 > secs] > > > [Times: user=0.21 sys=0.06, real=0.34 secs] > > > 2013-10-07T01:25:19.853+0900: [GC [ParNew (promotion failed): > > > 235745K->235745K(235968K), 0.3339710 > > secs][CMS2013-10-07T01:25:29.750+0900: > > > [CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 sys=3.74, > > > real=88.77 secs] > > > (concurrent mode failure): 7899125K->2882954K(7929856K), 42.8279810 > > secs] > > > 8105422K->2882954K(8165824K), [CMS Perm : 31956K->31861K(53340K)] > > icms_dc=9 > > > , 43.1621090 secs] [Times: user=10.40 sys=1.89, real=43.16 secs] > > > 2013-10-07T01:26:08.288+0900: [GC [1 CMS-initial-mark: > > 2882954K(7929856K)] > > > 2978434K(8165824K), 0.0965830 secs] [Times: user=0.04 sys=0.00, > real=0.09 > > > secs] > > > Heap > > > par new generation total 235968K, used 197697K [0x0000000606e00000, > > > 0x0000000616e00000, 0x0000000616e00000) > > > eden space 209792K, 94% used [0x0000000606e00000, > 0x0000000612f10718, > > > 0x0000000613ae0000) > > > from space 26176K, 0% used [0x0000000615470000, 0x0000000615470000, > > > --e89a8ff250e06a56a004ea198633--