hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4863) Make HBase Thrift server more configurable and add a command-line UI test
Date Thu, 24 Nov 2011 22:52:40 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156923#comment-13156923
] 

Hadoop QA commented on HBASE-4863:
----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12505038/0001-Fix-thread-leaks-in-the-HBase-thread-pool-server.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 15 new or modified tests.

    -1 javadoc.  The javadoc tool appears to have generated -162 warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    -1 findbugs.  The patch appears to introduce 67 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit
warnings.

     -1 core tests.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.client.TestAdmin
                  org.apache.hadoop.hbase.util.TestThreads

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/364//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/364//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/364//console

This message is automatically generated.
                
> Make HBase Thrift server more configurable and add a command-line UI test
> -------------------------------------------------------------------------
>
>                 Key: HBASE-4863
>                 URL: https://issues.apache.org/jira/browse/HBASE-4863
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Mikhail Bautin
>            Assignee: Mikhail Bautin
>         Attachments: 0001-Fix-thread-leaks-in-the-HBase-thread-pool-server.patch, D531.1.patch,
D531.2.patch, D531.3.patch
>
>
> This started as an internal hotfix where we found out that the Thrift server spawned
15000 threads. To bound the thread pool size I added a custom thread pool server implementation
called HBaseThreadPoolServer into HBase codebase, and made the following parameters configurable
from both command line and as config settings: minWorkerThreads, maxWorkerThreads, and maxQueuedRequests.
Under an increasing load, the server creates new threads for every connection before the pool
size reaches minWorkerThreads. After that, the server puts new connections into the queue
and only creates a new thread when the queue is full. If an attempt to create a new thread
fails, the server drops connection. The default TThreadPoolServer would crash in that case,
but it never happened because the thread pool was unbounded, so the server would hang indefinitely,
consume a lot of memory, and cause huge latency spikes on the client side.
> Another part of this fix is refactoring and unit testing of the command-line part of
the Thrift server. The logic there is sufficiently complicated, and the existing ThriftServer
class does not test that part at all. The new TestThriftServerCmdLine test starts the Thrift
server on a random port with various combinations of options and talks to it through the client
API from another thread.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message