Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 8F800200C75 for ; Sun, 21 May 2017 10:46:08 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 8E158160BC5; Sun, 21 May 2017 08:46:08 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id D628C160BA4 for ; Sun, 21 May 2017 10:46:07 +0200 (CEST) Received: (qmail 46167 invoked by uid 500); 21 May 2017 08:46:07 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 46156 invoked by uid 99); 21 May 2017 08:46:07 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 May 2017 08:46:07 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 99EBDC3AE8 for ; Sun, 21 May 2017 08:46:06 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.202 X-Spam-Level: X-Spam-Status: No, score=-99.202 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id QMcGn-vgZ5MG for ; Sun, 21 May 2017 08:46:06 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id D8A055F249 for ; Sun, 21 May 2017 08:46:05 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 548E4E0D77 for ; Sun, 21 May 2017 08:46:05 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 1EFDF21B59 for ; Sun, 21 May 2017 08:46:04 +0000 (UTC) Date: Sun, 21 May 2017 08:46:04 +0000 (UTC) From: "Ted Yu (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-18084) Improve CleanerChore to clean from directory which consumes more disk space MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Sun, 21 May 2017 08:46:08 -0000 [ https://issues.apache.org/jira/browse/HBASE-18084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16018759#comment-16018759 ] Ted Yu commented on HBASE-18084: -------------------------------- bq. the cleaning time is way more long than getting space and sorting. If the initial batch contains large directory, cleaning such directory already provides gain compared with current patch since the cleaning would be deemed important considering the outcome of the sorting. > Improve CleanerChore to clean from directory which consumes more disk space > --------------------------------------------------------------------------- > > Key: HBASE-18084 > URL: https://issues.apache.org/jira/browse/HBASE-18084 > Project: HBase > Issue Type: Bug > Reporter: Yu Li > Assignee: Yu Li > Attachments: HBASE-18084.patch, HBASE-18084.v2.patch > > > Currently CleanerChore cleans the directory in dictionary order, rather than from the directory with largest space usage. And when data abnormally accumulated to some huge volume in archive directory, the cleaning speed might not be enough. > This proposal is another improvement working together with HBASE-18083 to resolve our online issue (archive dir consumed more than 1.8PB SSD space) -- This message was sent by Atlassian JIRA (v6.3.15#6346)