hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "HBase Review Board (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2832) Priorities and multi-threading for MemStore flushing
Date Tue, 13 Jul 2010 05:47:50 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887663#action_12887663
] 

HBase Review Board commented on HBASE-2832:
-------------------------------------------

Message from: "Jonathan Gray" <jgray@apache.org>

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/301/
-----------------------------------------------------------

(Updated 2010-07-12 22:43:42.753269)


Review request for hbase, stack, Karthik Ranganathan, and Kannan Muthukkaruppan.


Changes
-------

Incorporates changes I acknowledged from stack review.


Summary
-------

Adds support for priorities and concurrency to regionserver flushing.
- Adds support for RS-side events/handlers/executors
- Adds support for prioritized HBaseEventHandlers
- Flushing now happens through FlushHandler, a new HBaseEventHandler.  There is an RS_FLUSHER
executor pool that defaults to two threads right now but is also checking a conf value.  There
is a good bit of documentation in FlushHandler.
- Adds unit test TestFlushHandler.  There is a nicer way to detect when flushes finish now
for other tests.
- Handling of FS errors is pushed into FlushHandler now.  The changes happening with the master
rewrite introduce a ServerStatus interface (probably a RegionStatus for rs side) that will
contain the necessary methods rather than using HRegionServer directly as is required for
now.
- Something weird not passing in tests with multiple masters and regionservers, still working
that out.


This addresses bug HBASE-2832.
    http://issues.apache.org/jira/browse/HBASE-2832


Diffs (updated)
-----

  trunk/src/main/java/org/apache/hadoop/hbase/executor/HBaseEventHandler.java 963507 
  trunk/src/main/java/org/apache/hadoop/hbase/executor/HBaseExecutorService.java 963507 
  trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 963507 
  trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 963507

  trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 963507 
  trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java 963507 
  trunk/src/main/java/org/apache/hadoop/hbase/regionserver/handler/FlushHandler.java PRE-CREATION

  trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestFlushHandler.java PRE-CREATION


Diff: http://review.hbase.org/r/301/diff


Testing
-------

Adds TestFlushHandler which passes.  Working on getting unit tests passing now, something
related to the ExecutorService.


Thanks,

Jonathan




> Priorities and multi-threading for MemStore flushing
> ----------------------------------------------------
>
>                 Key: HBASE-2832
>                 URL: https://issues.apache.org/jira/browse/HBASE-2832
>             Project: HBase
>          Issue Type: New Feature
>          Components: regionserver
>            Reporter: Jonathan Gray
>            Assignee: Jonathan Gray
>            Priority: Critical
>             Fix For: 0.90.0
>
>
> Similar to HBASE-1476 and HBASE-2646 which are for compactions, but do this for flushes.
> Flushing when we hit the normal flush size is a low priority flush.  Other types of flushes
(heap pressure, blocking client requests, etc) are high priority.
> Should have a tunable number of concurrent flushes.
> Will use the {{HBaseExecutorService}} and {{HBaseEventHandler}} introduced from master/zk
changes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message