Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 894EB200C44 for ; Mon, 27 Mar 2017 16:34:06 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 87D82160B85; Mon, 27 Mar 2017 14:34:06 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id D17EC160B5D for ; Mon, 27 Mar 2017 16:34:05 +0200 (CEST) Received: (qmail 39159 invoked by uid 500); 27 Mar 2017 14:34:04 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 39136 invoked by uid 99); 27 Mar 2017 14:34:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Mar 2017 14:34:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id C9E0BC6A0E for ; Mon, 27 Mar 2017 14:34:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.38 X-Spam-Level: ** X-Spam-Status: No, score=2.38 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, HTML_OBFUSCATE_05_10=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id rOqN702-xNfA for ; Mon, 27 Mar 2017 14:34:03 +0000 (UTC) Received: from mail-yw0-f170.google.com (mail-yw0-f170.google.com [209.85.161.170]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id DC83F5FACC for ; Mon, 27 Mar 2017 14:34:02 +0000 (UTC) Received: by mail-yw0-f170.google.com with SMTP id p77so32478280ywg.1 for ; Mon, 27 Mar 2017 07:34:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=hSVto9XtqaEmcCTmGesaNK7BgpMusdK+tFS41esaHlk=; b=GltbTEbKNhfcwm5vYBVkvwVvPNbzJ9BvNiEE+/6XBEA+0k2nikCll4eEfrWz4t+bfc UCaj5JDsze5Xd0cNnBb3yPWRrW1RohaAHXH53cwt/LNhC7y0hq3e6zMnLJ5b0x0Vb80s N/3jt1UZ/1DcotDtPPz1Il+8XlMCTVdIyEgPynzCCcAxs4CF5DaRV/vMvK5q/XYL1qDI 9G32eOU2psaegHUm8cgo/98pz/mXnCDGFSo6MIRVP7kzLa3EVhXIdOkzfJ1S3X6riLTW /Irtt06BzZI4I7fcpXCeSlp1eRKxyTHKgAKeUAVZ8ddzoe8IvyBxZKMPvQRw+gLVh9Bg S35g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=hSVto9XtqaEmcCTmGesaNK7BgpMusdK+tFS41esaHlk=; b=ltOFhCBdrp5iedruTfMvtWTBBlC+ZC6tBJNsyoBaLSKFwVMjPyYKVwTnEI9GOOYcc5 jcl+g3r61ObY/uhi5xK4hD/+0OsPRgqFNwKglXynQX9Q4loJLNrOTOYeBhU+yGLbkLot uwWzvhAnTVKahy5agoIx0yW26DlW3AiEVrShEievHRfPErwwRAWIpAT6lcZripdE2aaP B0N4s/wwjdkR15g2OQrbaNgNhyN+7sAhRjEX/xwWYLD/Iwqjd/S8nJSp7snVGguae/bd uTXlKXkp8jnerB8JBs6PA+ID15hM1cSBPzIID18pnwaaxJKvuPj1r4MSFdiRMNX+FAP0 m+0g== X-Gm-Message-State: AFeK/H3ALz03Migb49rjXcuC5SBIq7XimBnSS3B0TWAEECZTnapknBy+wxCM/8FgLZ9KbDlUuzBFxsunuYV6yQ== X-Received: by 10.129.95.132 with SMTP id t126mr18059383ywb.344.1490625242232; Mon, 27 Mar 2017 07:34:02 -0700 (PDT) MIME-Version: 1.0 Received: by 10.37.218.198 with HTTP; Mon, 27 Mar 2017 07:34:01 -0700 (PDT) In-Reply-To: References: From: Ted Yu Date: Mon, 27 Mar 2017 07:34:01 -0700 Message-ID: Subject: Re: All memstores flushed to be quite small files To: "user@hbase.apache.org" Content-Type: multipart/alternative; boundary=001a11473ac80e7e36054bb73c19 archived-at: Mon, 27 Mar 2017 14:34:06 -0000 --001a11473ac80e7e36054bb73c19 Content-Type: text/plain; charset=UTF-8 How many column families does your table have ? Which hbase release are you using ? Can you pastebin more of the server log around the time of flush ? Thanks On Mon, Mar 27, 2017 at 6:26 AM, Hef wrote: > Hi, > Does anyone have an idea why most of my 128MB memstore flushed files are > only several MBs? > > There are a lot of logs look as below: > > 2017-03-27 13:10:25,064 INFO > org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher: Flushed, > sequenceid=13042496, memsize=128.0 M, hasBloomFilter=true, into tmp file > hdfs://xxxxx > 2017-03-27 13:10:25,093 INFO org.apache.hadoop.hbase.regionserver.HStore: > Added > hdfs://nameservice1/hbase/data/default/mytable/ > 82f4e7d6931a57edf9654f32625f9fb7/t/0f59c07ef24644a38dce577d7fb6ac1b, > entries=545226, sequenceid=13042496, filesize=7.3 M > 2017-03-27 13:10:25,094 INFO org.apache.hadoop.hbase.regionserver.HRegion: > Finished memstore flush of ~128.02 MB/134235405, currentsize=352.34 > KB/360800 for region > mytable,3808555269964B038A7BC1EC6CC19B03\x00\x00\x00\x00\x00\x00\ > x00\xA7\x00\x06P\xC9])\x9A\xC9\x01\xA9Q<\xBDX\x88\x07\ > x82\xEAR\xD7,1490340797381.82f4e7d6931a57edf9654f32625f9fb7. > in 1052ms, sequenceid=13042496, compaction requested=false > > A 128MB memstore flushed to be only 7.3MB hfile, is that normal? > In what case could this happen? > > Thanks > Hef > --001a11473ac80e7e36054bb73c19--