Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 17 Nov 2016 20:14:58 +0000 (UTC)
From: "huaxiang sun (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.13016781.1477962955000.306734.1479413698948@Atlassian.JIRA>
In-Reply-To: <JIRA.13016781.1477962955000@Atlassian.JIRA>
References: <JIRA.13016781.1477962955000@Atlassian.JIRA> <JIRA.13016781.1477962955285@arcas>
Subject: [jira] [Commented] (HBASE-16981) Expand Mob Compaction Partition
 policy from daily to weekly, monthly and beyond
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 17 Nov 2016 20:15:01 -0000


    [ https://issues.apache.org/jira/browse/HBASE-16981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15674688#comment-15674688 ] 

huaxiang sun commented on HBASE-16981:
--------------------------------------

Thanks [~jingcheng.du@intel.com] and [~anoop.hbase], very good points. I will create the first draft based on your comments. The one point I want to add is the major mob compact. Major compact will do something I implement today. Some use cases I am aware of is that mob compaction chore is disabled, the major mob compaction is manually scheduled to reduce the number of the mob files.

> Expand Mob Compaction Partition policy from daily to weekly, monthly and beyond
> -------------------------------------------------------------------------------
>
>                 Key: HBASE-16981
>                 URL: https://issues.apache.org/jira/browse/HBASE-16981
>             Project: HBase
>          Issue Type: New Feature
>          Components: mob
>    Affects Versions: 2.0.0
>            Reporter: huaxiang sun
>            Assignee: huaxiang sun
>         Attachments: HBASE-16981.master.001.patch, HBASE-16981.master.002.patch, Supportingweeklyandmonthlymobcompactionpartitionpolicyinhbase.pdf
>
>
> Today the mob region holds all mob files for all regions. With daily partition mob compaction policy, after major mob compaction, there is still one file per region daily. Given there is 365 days in one year, at least 365 files per region. Since HDFS has limitation for number of files under one folder, this is not going to scale if there are lots of regions. To reduce mob file number,  we want to introduce other partition policies such as weekly, monthly to compact mob files within one week or month into one file. This jira is create to track this effort.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)