hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray" <jg...@apache.org>
Subject Re: Review Request: HBASE-2832: Priorities and multi-threading for MemStore flushing
Date Tue, 13 Jul 2010 04:41:44 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated 2010-07-12 21:41:44.943871)

Review request for hbase, stack, Karthik Ranganathan, and Kannan Muthukkaruppan.


Fixes some issues introduced by also having executor services running on the RS-side.  Specifically,
shutdown hooks to the HBaseExecutorService would shutdown and clear all executor pools, rather
than just those of the shutting down server instance.

Now the map of services is always prefixed with the serverName and during shutdown we only
actually shutdown those prefixed with the serverName.

More unit tests are now passing, still finish the suite.


Adds support for priorities and concurrency to regionserver flushing.
- Adds support for RS-side events/handlers/executors
- Adds support for prioritized HBaseEventHandlers
- Flushing now happens through FlushHandler, a new HBaseEventHandler.  There is an RS_FLUSHER
executor pool that defaults to two threads right now but is also checking a conf value.  There
is a good bit of documentation in FlushHandler.
- Adds unit test TestFlushHandler.  There is a nicer way to detect when flushes finish now
for other tests.
- Handling of FS errors is pushed into FlushHandler now.  The changes happening with the master
rewrite introduce a ServerStatus interface (probably a RegionStatus for rs side) that will
contain the necessary methods rather than using HRegionServer directly as is required for
- Something weird not passing in tests with multiple masters and regionservers, still working
that out.

This addresses bug HBASE-2832.

Diffs (updated)

  trunk/src/main/java/org/apache/hadoop/hbase/executor/HBaseEventHandler.java 963507 
  trunk/src/main/java/org/apache/hadoop/hbase/executor/HBaseExecutorService.java 963507 
  trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 963507 
  trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 963507

  trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 963507 
  trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java 963507 
  trunk/src/main/java/org/apache/hadoop/hbase/regionserver/handler/FlushHandler.java PRE-CREATION

  trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestFlushHandler.java PRE-CREATION

Diff: http://review.hbase.org/r/301/diff


Adds TestFlushHandler which passes.  Working on getting unit tests passing now, something
related to the ExecutorService.



View raw message