Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 1 Feb 2017 19:35:52 +0000 (UTC)
From: "Ajay Jadhav (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.13026849.1481247188000.10326.1485977752088@Atlassian.JIRA>
In-Reply-To: <JIRA.13026849.1481247188000@Atlassian.JIRA>
References: <JIRA.13026849.1481247188000@Atlassian.JIRA> <JIRA.13026849.1481247188674@jira-lw-us.apache.org>
Subject: [jira] [Updated] (HBASE-17280) Add mechanism to control hbase
 cleaner behavior
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 01 Feb 2017 19:35:57 -0000


     [ https://issues.apache.org/jira/browse/HBASE-17280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ajay Jadhav updated HBASE-17280:
--------------------------------
    Release Note: 
The HBase cleaner chore process cleans up old WAL files and archived HFiles. Cleaner operation can affect query performance when running heavy workloads, so disable the cleaner during peak hours. The cleaner has the following HBase shell commands:

- cleaner_chore_enabled: Queries whether cleaner chore is enabled/ disabled. 
- cleaner_chore_run: Manually runs the cleaner to remove files.
- cleaner_chore_switch: enables or disables the cleaner and returns the previous state of the cleaner. For example, cleaner-switch true enables the cleaner.

Following APIs are added in Admin:
- setCleanerChoreRunning(boolean on): Enable/Disable the cleaner chore
- runCleanerChore(): Ask for cleaner chore to run
- isCleanerChoreEnabled(): Query whether cleaner chore is enabled/ disabled. 

  was:
The HBase cleaner chore process cleans up old WAL files and archived HFiles. Cleaner operation can affect query performance when running heavy workloads, so disable the cleaner during peak hours. The cleaner has the following HBase shell commands:

- cleaner_chore_state: Queries the state of cleaner chore. 
Possible responses are as follows: 
HFILE_CLEANER_ENABLED, LOG_CLEANER_ENABLED, BOTH_HFILE_LOG_CLEANER_ENABLED, BOTH_HFILE_LOG_CLEANER_DISABLED.

- cleaner_chore_run: Manually runs the cleaner to remove files.

- cleaner_chore_switch: enables or disables the cleaner and returns the previous state of the cleaner. For example, cleaner-switch true enables the cleaner.

Following APIs are added in Admin:
- setCleanerChoreRunning(boolean on): Enable/Disable the cleaner chore
- runCleanerChore(): Ask for cleaner chore to run
- getCleanerChoreState(): Query the cleaner chore state


> Add mechanism to control hbase cleaner behavior
> -----------------------------------------------
>
>                 Key: HBASE-17280
>                 URL: https://issues.apache.org/jira/browse/HBASE-17280
>             Project: HBase
>          Issue Type: Improvement
>          Components: Client, hbase, shell
>    Affects Versions: 2.0.0, 1.2.0
>            Reporter: Ajay Jadhav
>            Priority: Minor
>             Fix For: 2.0.0, 1.2.0
>
>         Attachments: HBASE-17280.branch-1.2.patch, HBASE-17280.branch-2.0.patch, HBASE-17280.master.003.patch, HBASE-17280.master.004.patch, HBASE-17280.v1-branch-1.2.patch, HBASE-17280.v2-branch-1.2.patch, HBASE-17280.v2-branch-2.patch
>
>
> Cleaner is used to get rid of archived HFiles and old WALs in HBase.
> In the case of heavy workload, cleaner can affect query performance by creating a lot of connections to perform costly reads/ writes against underlying filesystem.
> This patch allows the user to control HBase cleaner behavior by providing shell commands to enable/ disable and manually run it.
> Our main intention with this patch was to avoid running the expensive cleaner chore during peak times. During our experimentation, we saw a lot of HFiles and WAL log related files getting created inside archive dir (didn't see ZKlock related files). Since we were replacing hdfs with S3, these delete calls will take forever to complete.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)