hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11263) Share the open/close store file thread pool for all store in a region
Date Fri, 30 May 2014 17:15:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14013960#comment-14013960
] 

stack commented on HBASE-11263:
-------------------------------

Naming the thread pool for the first region that passes by seems a bit odd:

+        String threadNamePrefix =
+            "StoreFileOpenerThread-" + getRegionNameAsString();



> Share the open/close store file thread pool for all store in a region
> ---------------------------------------------------------------------
>
>                 Key: HBASE-11263
>                 URL: https://issues.apache.org/jira/browse/HBASE-11263
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>    Affects Versions: 0.99.0
>            Reporter: Liu Shaohui
>            Assignee: Liu Shaohui
>            Priority: Minor
>         Attachments: HBASE-11263-trunk-v1.diff
>
>
> Currently, the open/close store file thread pool is divided equally to all stores of
a region. 
> {code}
>   protected ThreadPoolExecutor getStoreFileOpenAndCloseThreadPool(
>       final String threadNamePrefix) {
>     int numStores = Math.max(1, this.htableDescriptor.getFamilies().size());
>     int maxThreads = Math.max(1,
>         conf.getInt(HConstants.HSTORE_OPEN_AND_CLOSE_THREADS_MAX,
>             HConstants.DEFAULT_HSTORE_OPEN_AND_CLOSE_THREADS_MAX)
>             / numStores);
>     return getOpenAndCloseThreadPool(maxThreads, threadNamePrefix);
>   }
> {code}
> This is not very optimal in following scenarios:
> # The data of some column families are very large and there are many hfiles in those
stores, and others may be very small and in-memory column families. 
> # Usually we preserve some column families for later needs. The thread pool for these
column families are wasted。
> The simple way is to share a big thread pool for all stores to open/close hfiles.  
> Suggestions are welcomed. 
> Thanks. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message