Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DF04F10AC5 for ; Thu, 7 Nov 2013 16:04:22 +0000 (UTC) Received: (qmail 66533 invoked by uid 500); 7 Nov 2013 16:04:19 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 66256 invoked by uid 500); 7 Nov 2013 16:04:19 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 66248 invoked by uid 99); 7 Nov 2013 16:04:19 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Nov 2013 16:04:19 +0000 X-ASF-Spam-Status: No, hits=1.8 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of johnnyenglish739@gmail.com designates 209.85.217.178 as permitted sender) Received: from [209.85.217.178] (HELO mail-lb0-f178.google.com) (209.85.217.178) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Nov 2013 16:04:13 +0000 Received: by mail-lb0-f178.google.com with SMTP id l4so598516lbv.37 for ; Thu, 07 Nov 2013 08:03:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Mtx3HMsC/UpPHko1LRl/xXRVQavpoYMyXgfmFE4HaoA=; b=zal7+xWld5585U0jvWXNVtb2SmYcIIZW07V0aiZkmm9hi1dXt1RchmC35Hx1iokuNX ur9IbkJchDf7S5b8o08oj4G2g9Tuz0o6pF19G3gh6nYizvZW7a1hl/b+MR53iRmkKM6+ covQ5vN279DkOgSnOJTnRM5oMDzwhrTjLhGIZrtMFioSF4+yZbmKrq1eCEcQUQaQxr1l SjY0lUJRudkoiKioSxlvKp6a4rZqg1xfMUhOxY3hcs+DLkM7xIIw79qSQPFygOQNV/vt ajBqKFRG/UqOSd9BBnSeNyjfa6DYPWzkYiiOKiFejFJT0BboY1rHl22OaGOnHmBfMseH KRAQ== MIME-Version: 1.0 X-Received: by 10.112.59.134 with SMTP id z6mr6944643lbq.28.1383840233090; Thu, 07 Nov 2013 08:03:53 -0800 (PST) Received: by 10.114.181.39 with HTTP; Thu, 7 Nov 2013 08:03:53 -0800 (PST) In-Reply-To: <1383839993.77867.YahooMailNeo@web190103.mail.sg3.yahoo.com> References: <1383839993.77867.YahooMailNeo@web190103.mail.sg3.yahoo.com> Date: Thu, 7 Nov 2013 17:03:53 +0100 Message-ID: Subject: Re: RegionServer crash without any errors (compaction?) From: John To: user@hbase.apache.org, Dhaval Shah Content-Type: multipart/alternative; boundary=e89a8fb1f74884f55d04ea986946 X-Virus-Checked: Checked by ClamAV on apache.org --e89a8fb1f74884f55d04ea986946 Content-Type: text/plain; charset=ISO-8859-1 Hi, thanks for your fast answer. If I take a look at the cloudera manager at this time the %-time of using the GC increase at this time, so I think you are right. The max heap size is 1GB for this node. The hbase.hregion.max.filesize is also 1GB. regards 2013/11/7 Dhaval Shah > Did you look at your GC logs? Probably the compaction process is running > your region server out of memory. Can you provide more details on your > setup? Max heap size? Max Region HFile size? > > Regards, > Dhaval > > > ________________________________ > From: John > To: user@hbase.apache.org > Sent: Thursday, 7 November 2013 10:51 AM > Subject: RegionServer crash without any errors (compaction?) > > > Hi, > > I have a cluster with 7 regionserver. Some of them are crashing from time > to time wihtout any error message in the hbase log. If I take a look at the > log at the time I found this: > > 2013-11-07 15:29:02,511 INFO org.apache.hadoop.hbase.regionserver.Store: > Starting compaction of 2 file(s) in 1 of P_SO,< > http://xmlns.com/foaf/0.1/homepage > >,1383188177383.59d0259c87c07dc666a5600ba4d6c916. > i$ > 2013-11-07 15:29:10,471 INFO > org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter > type for hdfs:// > > pc08.pool.ifis.uni-luebeck.de:8020/hbase/P_SO/59d0259c87c07dc666a5600ba4d6c916/.tmp/f$ > 2013-11-07 15:31:05,944 INFO org.apache.hadoop.hbase.util.VersionInfo: > HBase 0.94.6-cdh4.4.0 > .... restart > > At this time 2 of the 7 RS crashed, both has this compaction message before > they crashed. I don't know exactly what compaction is, but it seems that > this compaction has to do with the crash. What can I do to avoid this > restart/crash? > > best regards > --e89a8fb1f74884f55d04ea986946--