Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 20 Dec 2011 19:32:31 +0000 (UTC)
From: "Phabricator (Commented) (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: 
 <1178004059.32177.1324409551576.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1709918120.13317.1323905971152.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HBASE-5033) Opening/Closing store in parallel
 to reduce region open/close time
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-5033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173457#comment-13173457 ] 

Phabricator commented on HBASE-5033:
------------------------------------

Liyin has commented on the revision "[jira][HBASE-5033][[89-fb]]Opening/Closing store in parallel to reduce region open/close time".

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:294 As you suggested, it has already used 2 separate parameters to control the number of stores and store files,  which will be opened or closed in parallel.

  For example:
  hbase.hregion.storeCloser.threads.max is to control the number of parallel closing stores.
  While hbase.hregion.storeFileCloser.threads.max is to control the number of parallel closing store files.
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:814 It would be easy to control the life cycle of thread pool and decouple the dependency of open and close operation if using separate thread pool. There are some details explanation in the previous comments :)

  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:536 I am not quite sure whether we need to set these threads as daemons since the thread pool will be shutdown in the "finally" block anyway.
  The main thread shall never leave any tasks running in these thread pools after the finally block.

  Is there any specific reason ?
  or it will always be safe to set these threads as daemons ?


REVISION DETAIL
  https://reviews.facebook.net/D933

                
> Opening/Closing store in parallel to reduce region open/close time
> ------------------------------------------------------------------
>
>                 Key: HBASE-5033
>                 URL: https://issues.apache.org/jira/browse/HBASE-5033
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Liyin Tang
>            Assignee: Liyin Tang
>         Attachments: D933.1.patch, D933.2.patch, D933.3.patch
>
>
> Region servers are opening/closing each store and each store file for every store in sequential fashion, which may cause inefficiency to open/close regions. 
> So this diff is to open/close each store in parallel in order to reduce region open/close time. Also it would help to reduce the cluster restart time.
> 1) Opening each store in parallel
> 2) Loading each store file for every store in parallel
> 3) Closing each store in parallel
> 4) Closing each store file for every store in parallel.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira