Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1BEC69BAA for ; Wed, 22 Feb 2012 23:03:57 +0000 (UTC) Received: (qmail 26788 invoked by uid 500); 22 Feb 2012 23:03:55 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 26754 invoked by uid 500); 22 Feb 2012 23:03:55 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 26745 invoked by uid 99); 22 Feb 2012 23:03:55 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 22 Feb 2012 23:03:55 +0000 X-ASF-Spam-Status: No, hits=1.1 required=5.0 tests=FROM_12LTRDOM,FSL_RCVD_USER,RCVD_IN_DNSWL_LOW X-Spam-Check-By: apache.org Received-SPF: unknown (nike.apache.org: error in processing during lookup of alok@urbanairship.com) Received: from [209.85.210.169] (HELO mail-iy0-f169.google.com) (209.85.210.169) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 22 Feb 2012 23:03:47 +0000 Received: by iagz16 with SMTP id z16so871265iag.14 for ; Wed, 22 Feb 2012 15:03:26 -0800 (PST) Received-SPF: pass (google.com: domain of alok@urbanairship.com designates 10.50.153.198 as permitted sender) client-ip=10.50.153.198; Authentication-Results: mr.google.com; spf=pass (google.com: domain of alok@urbanairship.com designates 10.50.153.198 as permitted sender) smtp.mail=alok@urbanairship.com Received: from mr.google.com ([10.50.153.198]) by 10.50.153.198 with SMTP id vi6mr381800igb.30.1329951806475 (num_hops = 1); Wed, 22 Feb 2012 15:03:26 -0800 (PST) MIME-Version: 1.0 Received: by 10.50.153.198 with SMTP id vi6mr322809igb.30.1329951806413; Wed, 22 Feb 2012 15:03:26 -0800 (PST) Received: by 10.42.172.73 with HTTP; Wed, 22 Feb 2012 15:03:26 -0800 (PST) Date: Wed, 22 Feb 2012 15:03:26 -0800 Message-ID: Subject: .logs directory keeps growing From: Alok Singh To: user@hbase.apache.org Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQmMaSzp47nCZsIHA8ntzwPas2sWYeZcxgyHDKHtZCwfY+dpxF8qF351JaBkRNbo79ZBlVvV X-Virus-Checked: Checked by ClamAV on apache.org Environment Hbase: 0.92 Hadoop: hadoop-0.20.2-cdh3u3 I am testing hbase 0.92 for a new storage system we are building. In the tests, I insert around 2-3 billion rows and then run some scans/queries against it to test the performance. Once the tests are complete, I drop all of the tables and recreate them to clear out the test data. But, it doesn't look like the /hbase/root/.logs is getting cleared. It seems to just keep growing, and the only way to clear it out is to restart the hbase cluster. After the restarts, the regionserver logs are filled with this: --------- 12/02/22 22:57:58 INFO wal.HLogSplitter: This region's directory doesn't exist: hdfs://10.130.1.15:7080/foo/hbase/root/device/9605bdd29d02212f23499296542341d8. It is very likely that it was already split so it's safe to discard those edits. 12/02/22 22:57:58 INFO wal.HLogSplitter: This region's directory doesn't exist: hdfs://10.130.1.15:7080/foo/hbase/root/device.device/b3b96cd17e683e52f8c72e14229f5567. It is very likely that it was already split so it's safe to discard those edits. 12/02/22 22:57:58 INFO wal.HLogSplitter: This region's directory doesn't exist: hdfs://10.130.1.15:7080/foo/hbase/root/device/83832bcdc8a1bde1094d9039594cb067. It is very likely that it was already split so it's safe to discard those edits. 12/02/22 22:57:58 INFO wal.HLogSplitter: This region's directory doesn't exist: hdfs://10.130.1.15:7080/foo/hbase/root/device/140ded7a67c8cf2a0e7cbbf037033889. It is very likely that it was already split so it's safe to discard those edits. ..... ---------- Size of /hbase/root/device: 110GB Size of /hbase/root/.logs: 770GB Alok