Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 300F8D595 for ; Tue, 7 Aug 2012 20:02:12 +0000 (UTC) Received: (qmail 54040 invoked by uid 500); 7 Aug 2012 20:02:12 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 54004 invoked by uid 500); 7 Aug 2012 20:02:12 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 53994 invoked by uid 99); 7 Aug 2012 20:02:11 -0000 Received: from issues-vm.apache.org (HELO issues-vm) (140.211.11.160) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 07 Aug 2012 20:02:11 +0000 Received: from isssues-vm.apache.org (localhost [127.0.0.1]) by issues-vm (Postfix) with ESMTP id BA390142855 for ; Tue, 7 Aug 2012 20:02:11 +0000 (UTC) Date: Tue, 7 Aug 2012 20:02:11 +0000 (UTC) From: "Zhihong Ted Yu (JIRA)" To: issues@hbase.apache.org Message-ID: <613789167.1111.1344369731768.JavaMail.jiratomcat@issues-vm> In-Reply-To: <839112812.109262.1343358875725.JavaMail.jiratomcat@issues-vm> Subject: [jira] [Commented] (HBASE-6466) Enable multi-thread for memstore flush MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-6466?page=3Dcom.atlassian= .jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1343= 0582#comment-13430582 ]=20 Zhihong Ted Yu commented on HBASE-6466: --------------------------------------- @Elliot: The flush taking 60s normally would take how many seconds to complete ? Thanks =20 > Enable multi-thread for memstore flush > -------------------------------------- > > Key: HBASE-6466 > URL: https://issues.apache.org/jira/browse/HBASE-6466 > Project: HBase > Issue Type: Improvement > Reporter: chunhui shen > Assignee: chunhui shen > Attachments: HBASE-6466.patch, HBASE-6466v2.patch, HBASE-6466v3.p= atch > > > If the KV is large or Hlog is closed with high-pressure putting, we found= memstore is often above the high water mark and block the putting. > So should we enable multi-thread for Memstore Flush? > Some performance test data for reference, > 1.test environment =EF=BC=9A=20 > random writting=EF=BC=9Bupper memstore limit 5.6GB;lower memstore limit 4= .8GB;400 regions per regionserver=EF=BC=9Brow len=3D50 bytes, value len=3D1= 024 bytes;5 regionserver, 300 ipc handler per regionserver;5 client, 50 thr= ead handler per client for writing > 2.test results: > one cacheFlush handler, tps: 7.8k/s per regionserver, Flush:10.1MB/s per = regionserver, appears many aboveGlobalMemstoreLimit blocking > two cacheFlush handlers, tps: 10.7k/s per regionserver, Flush:12.46MB/s p= er regionserver, > 200 thread handler per client & two cacheFlush handlers, tps:16.1k/s per = regionserver, Flush:18.6MB/s per regionserver -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrato= rs: https://issues.apache.org/jira/secure/ContactAdministrators!default.jsp= a For more information on JIRA, see: http://www.atlassian.com/software/jira