hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5033) Opening/Closing store in parallel to reduce region open/close time
Date Tue, 20 Dec 2011 19:32:31 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173457#comment-13173457
] 

Phabricator commented on HBASE-5033:
------------------------------------

Liyin has commented on the revision "[jira][HBASE-5033][[89-fb]]Opening/Closing store in parallel
to reduce region open/close time".

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:294 As you suggested, it has
already used 2 separate parameters to control the number of stores and store files,  which
will be opened or closed in parallel.

  For example:
  hbase.hregion.storeCloser.threads.max is to control the number of parallel closing stores.
  While hbase.hregion.storeFileCloser.threads.max is to control the number of parallel closing
store files.
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:814 It would be easy to
control the life cycle of thread pool and decouple the dependency of open and close operation
if using separate thread pool. There are some details explanation in the previous comments
:)

  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:536 I am not quite sure
whether we need to set these threads as daemons since the thread pool will be shutdown in
the "finally" block anyway.
  The main thread shall never leave any tasks running in these thread pools after the finally
block.

  Is there any specific reason ?
  or it will always be safe to set these threads as daemons ?


REVISION DETAIL
  https://reviews.facebook.net/D933

                
> Opening/Closing store in parallel to reduce region open/close time
> ------------------------------------------------------------------
>
>                 Key: HBASE-5033
>                 URL: https://issues.apache.org/jira/browse/HBASE-5033
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Liyin Tang
>            Assignee: Liyin Tang
>         Attachments: D933.1.patch, D933.2.patch, D933.3.patch
>
>
> Region servers are opening/closing each store and each store file for every store in
sequential fashion, which may cause inefficiency to open/close regions. 
> So this diff is to open/close each store in parallel in order to reduce region open/close
time. Also it would help to reduce the cluster restart time.
> 1) Opening each store in parallel
> 2) Loading each store file for every store in parallel
> 3) Closing each store in parallel
> 4) Closing each store file for every store in parallel.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message