hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2405) Close, split, open of regions in RegionServer are run by a single thread only.
Date Tue, 06 Apr 2010 16:33:33 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854048#action_12854048
] 

Jonathan Gray commented on HBASE-2405:
--------------------------------------

The issue is that an OPEN and CLOSE can/will trigger compactions and flushes.  These should
not be done inline, they should be queued like user triggered events are.  That's where the
new complexity comes in.

On OPEN, if the region contains references, will be compacted.  Does this currently happen
inline in the worker?  I think that might be queued.  If not, it should be.

On CLOSE, we flush the region.  This flush should be turned into a double-flush and done through
a separate thread pool that can be configured w/ a max concurrent flushes.

I don't want to run opens/closes in their own threads because then you don't have easy ways
to configure how much you want to use HDFS at once.  maxConcurrentCompactions and maxConcurrentFlushes
should (exist and) be strictly followed.

> Close, split, open of regions in RegionServer are run by a single thread only.
> ------------------------------------------------------------------------------
>
>                 Key: HBASE-2405
>                 URL: https://issues.apache.org/jira/browse/HBASE-2405
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: stack
>             Fix For: 0.20.4, 0.21.0
>
>         Attachments: nsre.txt
>
>
> JGray and Karthik observed yesterday that a regoin open message arrived at the regionserver
but that the regionserver worker thread did not get around to the actually opening until 45
seconds later (region offline for 45 seconds).  We only run a single Worker thread in a regoinserver
processing open, close, and splits.  In this case, a long running close (or two) held up the
worker thread.  We need to run more than a single worker.  A pool of workers?  Should opens
be prioritized?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message