Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id D4CC6200BC0 for ; Tue, 15 Nov 2016 15:36:00 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id D3768160B19; Tue, 15 Nov 2016 14:36:00 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 516C4160AD8 for ; Tue, 15 Nov 2016 15:36:00 +0100 (CET) Received: (qmail 2545 invoked by uid 500); 15 Nov 2016 14:35:59 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 2506 invoked by uid 99); 15 Nov 2016 14:35:59 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 15 Nov 2016 14:35:59 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 4C6A42C4C74 for ; Tue, 15 Nov 2016 14:35:59 +0000 (UTC) Date: Tue, 15 Nov 2016 14:35:59 +0000 (UTC) From: "huaxiang sun (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-16981) Expand Mob Compaction Partition policy from daily to weekly, monthly and beyond MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 15 Nov 2016 14:36:01 -0000 [ https://issues.apache.org/jira/browse/HBASE-16981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15667298#comment-15667298 ] huaxiang sun commented on HBASE-16981: -------------------------------------- Thanks [~jingcheng.du@intel.com] and [~anoop.hbase] for review. > Expand Mob Compaction Partition policy from daily to weekly, monthly and beyond > ------------------------------------------------------------------------------- > > Key: HBASE-16981 > URL: https://issues.apache.org/jira/browse/HBASE-16981 > Project: HBase > Issue Type: New Feature > Components: mob > Affects Versions: 2.0.0 > Reporter: huaxiang sun > Assignee: huaxiang sun > Attachments: HBASE-16981.master.001.patch, HBASE-16981.master.002.patch, Supportingweeklyandmonthlymobcompactionpartitionpolicyinhbase.pdf > > > Today the mob region holds all mob files for all regions. With daily partition mob compaction policy, after major mob compaction, there is still one file per region daily. Given there is 365 days in one year, at least 365 files per region. Since HDFS has limitation for number of files under one folder, this is not going to scale if there are lots of regions. To reduce mob file number, we want to introduce other partition policies such as weekly, monthly to compact mob files within one week or month into one file. This jira is create to track this effort. -- This message was sent by Atlassian JIRA (v6.3.4#6332)