hbase-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nspiegelb...@apache.org
Subject svn commit: r1181970 - /hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/client/HTable.java
Date Tue, 11 Oct 2011 17:44:52 GMT
Author: nspiegelberg
Date: Tue Oct 11 17:44:52 2011
New Revision: 1181970

URL: http://svn.apache.org/viewvc?rev=1181970&view=rev
Optimize the htable thread pool

The current setting is corepool=0 and using linkedblockingqueue.
This setting is equal to singleThreadPool, only 1 thread is active at all

Here is how open source trunk works:
It uses SynchronousQueue and Integer.MAX_VALUE as maximum thread number. It
will have better performance than our current setting. But the only problem is
it didn't bound the max thread size.

 int maxThreads = conf.getInt("hbase.htable.threads.max", Integer.MAX_VALUE);
    if (maxThreads == 0) {
      maxThreads = 1; // is there a better default?

    // Using the "direct handoff" approach, new threads will only be created
    // if it is necessary and will grow unbounded. This could be bad but in HCM
    // we only create as many Runnables as there are region servers. It means
    // it also scales when new region servers are added.
    this.pool = new ThreadPoolExecutor(1, maxThreads,
        60, TimeUnit.SECONDS,
        new SynchronousQueue<Runnable>(),
        new DaemonThreadFactory());

Test Plan: will run all the unit tests and test on dev cluster

Reviewers: kannan, cgthayer, vinodv, mbautin, nspiegelberg, dhruba, jgray,
srash, pkhemani

Reviewed By: kannan

CC: dist-storage@lists, hbase@lists, dhruba, liyintang, kannan

Differential Revision: 320506


Modified: hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/client/HTable.java
URL: http://svn.apache.org/viewvc/hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/client/HTable.java?rev=1181970&r1=1181969&r2=1181970&view=diff
--- hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/client/HTable.java (original)
+++ hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/client/HTable.java Tue Oct 11
17:44:52 2011
@@ -152,13 +152,15 @@ public class HTable implements HTableInt
       nrHRS = 10;
     int nrThreads = conf.getInt("hbase.htable.threads.max", nrHRS);
-    int corePools = conf.getInt("hbase.htable.threads.corePool", 0);
     // Unfortunately Executors.newCachedThreadPool does not allow us to
     // set the maximum size of the pool, so we have to do it ourselves.
-    this.pool = new ThreadPoolExecutor(corePools, nrThreads,
+    this.pool = new ThreadPoolExecutor(nrThreads, nrThreads,
         60, TimeUnit.SECONDS,
         new LinkedBlockingQueue<Runnable>(),
         new DaemonThreadFactory());
+    // allow the core pool threads timeout and terminate
+    ((ThreadPoolExecutor)this.pool).allowCoreThreadTimeOut(true);
   public Configuration getConfiguration() {

View raw message