hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14061) Support CF-level Storage Policy
Date Wed, 29 Jul 2015 02:18:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14645371#comment-14645371
] 

Andrew Purtell commented on HBASE-14061:
----------------------------------------

bq. 'hbase.hstore.block.storage.policy'

Consider making this a first class CF schema attribute. 'STORAGE_POLICY' perhaps?

bq. fs.getStoragePolicy cannot be easily written in reflection

Agreed, this is going to be a bit messy but I had a look at the code in the patch and it seems
doable at first glance.

We could only commit this to a branch where we're willing to set the minimum supported Hadoop
version at 2.6.0 or if the storage policy API is called through reflection we can port the
result to more places.

> Support CF-level Storage Policy
> -------------------------------
>
>                 Key: HBASE-14061
>                 URL: https://issues.apache.org/jira/browse/HBASE-14061
>             Project: HBase
>          Issue Type: Sub-task
>          Components: HFile, regionserver
>         Environment: hadoop-2.6.0
>            Reporter: Victor Xu
>            Assignee: Victor Xu
>         Attachments: HBASE-14061-master-v1.patch
>
>
> After reading [HBASE-12848|https://issues.apache.org/jira/browse/HBASE-12848] and [HBASE-12934|https://issues.apache.org/jira/browse/HBASE-12934],
I wrote a patch to implement cf-level storage policy. 
> My main purpose is to improve random-read performance for some really hot data, which
usually locates in certain column family of a big table.
> Usage:
> $ hbase shell
> > alter 'TABLE_NAME', METADATA => {'hbase.hstore.block.storage.policy' => 'POLICY_NAME'}
> > alter 'TABLE_NAME', {NAME=>'CF_NAME', METADATA => {'hbase.hstore.block.storage.policy'
=> 'POLICY_NAME'}}
> HDFS's setStoragePolicy can only take effect when new hfile is created in a configured
directory, so I had to make sub directories(for each cf) in region's .tmp directory and set
storage policy for them.
> Besides, I had to upgrade hadoop version to 2.6.0 because dfs.getStoragePolicy cannot
be easily written in reflection, and I needed this api to finish my unit test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message