hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yu Li (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-14061) Support CF-level Storage Policy
Date Mon, 09 Jan 2017 03:34:58 GMT

     [ https://issues.apache.org/jira/browse/HBASE-14061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Yu Li updated HBASE-14061:
       Resolution: Fixed
     Hadoop Flags: Reviewed
    Fix Version/s: 2.0.0
     Release Note: 
After HBASE-14061 we support to set storage policy for HFile through "hbase.hstore.block.storage.policy"
configuration, and we support CF-level setting to override the settings from configuration
file. Currently supported storage policies include ALL_SSD/ONE_SSD/HOT/WARM/COLD, refer to
for more details

For example, to create a table with two families: "cf1" with "ALL_SSD" storage policy and
"cf2" with "ONE_SSD", we could use below command in hbase shell:
create 'table',{NAME=>'f1',STORAGE_POLICY=>'ALL_SSD'},{NAME=>'f2',STORAGE_POLICY=>'ONE_SSD'}

We could also set the configuration in table attribute like all other configurations:
create 'table',{NAME=>'f1',CONFIGURATION=>{'hbase.hstore.block.storage.policy'=>'ONE_SSD'}}
           Status: Resolved  (was: Patch Available)

Pushed to master branch (and master only since branch-1 still supports hadoop 2.5.1), add
release note and resolving issue.

Thanks all for review.

> Support CF-level Storage Policy
> -------------------------------
>                 Key: HBASE-14061
>                 URL: https://issues.apache.org/jira/browse/HBASE-14061
>             Project: HBase
>          Issue Type: Sub-task
>          Components: HFile, regionserver
>         Environment: hadoop-2.6.0
>            Reporter: Victor Xu
>            Assignee: Yu Li
>             Fix For: 2.0.0
>         Attachments: HBASE-14061-master-v1.patch, HBASE-14061.v2.patch, HBASE-14061.v3.patch,
> After reading [HBASE-12848|https://issues.apache.org/jira/browse/HBASE-12848] and [HBASE-12934|https://issues.apache.org/jira/browse/HBASE-12934],
I wrote a patch to implement cf-level storage policy. 
> My main purpose is to improve random-read performance for some really hot data, which
usually locates in certain column family of a big table.
> Usage:
> $ hbase shell
> > alter 'TABLE_NAME', METADATA => {'hbase.hstore.block.storage.policy' => 'POLICY_NAME'}
> > alter 'TABLE_NAME', {NAME=>'CF_NAME', METADATA => {'hbase.hstore.block.storage.policy'
> HDFS's setStoragePolicy can only take effect when new hfile is created in a configured
directory, so I had to make sub directories(for each cf) in region's .tmp directory and set
storage policy for them.
> Besides, I had to upgrade hadoop version to 2.6.0 because dfs.getStoragePolicy cannot
be easily written in reflection, and I needed this api to finish my unit test.

This message was sent by Atlassian JIRA

View raw message