Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8DD89DCDF for ; Wed, 12 Dec 2012 20:19:40 +0000 (UTC) Received: (qmail 67701 invoked by uid 500); 12 Dec 2012 20:19:38 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 67649 invoked by uid 500); 12 Dec 2012 20:19:38 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 67641 invoked by uid 99); 12 Dec 2012 20:19:38 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Dec 2012 20:19:38 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of kevin.odell@cloudera.com designates 209.85.220.169 as permitted sender) Received: from [209.85.220.169] (HELO mail-vc0-f169.google.com) (209.85.220.169) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Dec 2012 20:19:31 +0000 Received: by mail-vc0-f169.google.com with SMTP id gb23so1321931vcb.14 for ; Wed, 12 Dec 2012 12:19:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:x-gm-message-state; bh=8VRK6gfLOJApB/vx4FGjgpjZD/g+I6xPw5UXSRlYYbk=; b=O8TNwr17P90UQkDNWQUSrbs8si27T8GWSzfktJZqcvr+hDbWrxJ7a+3q70s8k4fh3Q mMYhmwpN5kV/pcZiBwXWzMYFNw5fHPlcTVI2xGOljJ3DUi8//+I2Iqy0ZGw9Lk6vR8UO mzuhUXEKkEX1r+K6VkAn29oHbqy58HwVdWUDfOo1y/8Y2K+sURA40Kuo+E2+/DNM6Z6p g068Parbe0FKtgroul4bAIqCZaxSYEv6kYEHb2cAb586nveqJrEWJUd7EnRWTXVHNHqR cHx7Ru3Csczs7NQ5u0Ux1euuRAG0H53yNV78gEMGeHcZVL7y9IJW89atQRL0FXnZUL7a v7rw== MIME-Version: 1.0 Received: by 10.220.151.138 with SMTP id c10mr1214898vcw.56.1355343550339; Wed, 12 Dec 2012 12:19:10 -0800 (PST) Received: by 10.58.154.40 with HTTP; Wed, 12 Dec 2012 12:19:10 -0800 (PST) In-Reply-To: References: Date: Wed, 12 Dec 2012 15:19:10 -0500 Message-ID: Subject: Re: How to know it's time for a major compaction? From: "Kevin O'dell" To: user@hbase.apache.org Content-Type: multipart/alternative; boundary=f46d043893b3dddadf04d0ad8248 X-Gm-Message-State: ALoCoQmyJTDxBbyLY4ww0WOz4oMGqak78Mfd2i02tU4Zqt9bfgU01lNhovU+YbqT5q4Ss7OpuMbU X-Virus-Checked: Checked by ClamAV on apache.org --f46d043893b3dddadf04d0ad8248 Content-Type: text/plain; charset=ISO-8859-1 Otis, I think it depends on your use case. Some use cases are extremely dependent on locality and latency, so they will major compact any region the balancer has moved. Some people follow the HFile count and will trigger a major when there are x amount of files. Others will just follow best practices and do it daily. Some do very little writes and only compact once a week. On Wed, Dec 12, 2012 at 3:15 PM, Otis Gospodnetic < otis.gospodnetic@gmail.com> wrote: > Hello, > > What are the main metrics/signals people use/look at to say "OK, I really > need to do a major compaction now", other than "hm, performance is not > good, let's try major compaction tonight"? > > Thanks, > Otis > -- > HBASE Performance Monitoring - http://sematext.com/spm/index.html > Search Analytics - http://sematext.com/search-analytics/index.html > -- Kevin O'Dell Customer Operations Engineer, Cloudera --f46d043893b3dddadf04d0ad8248--