Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 23 Jan 2013 02:30:14 +0000 (UTC)
From: "Ted Yu (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12600418.1343358796657.177032.1358908214842@arcas>
In-Reply-To: <JIRA.12600418.1343358796657@arcas>
References: <JIRA.12600418.1343358796657@arcas>
Subject: [jira] [Commented] (HBASE-6466) Enable multi-thread for memstore
 flush
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/HBASE-6466?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1356=
0325#comment-13560325 ]=20

Ted Yu commented on HBASE-6466:
-------------------------------

Integrated to trunk.

Thanks for the patch, Chunhui.

Thanks for your work, Sergey.
               =20
> Enable multi-thread for memstore flush
> --------------------------------------
>
>                 Key: HBASE-6466
>                 URL: https://issues.apache.org/jira/browse/HBASE-6466
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>    Affects Versions: 0.96.0
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>            Priority: Critical
>             Fix For: 0.96.0
>
>         Attachments: 6466-v6.patch, HBASE-6466.patch, HBASE-6466v2.patch,=
 HBASE-6466v3.1.patch, HBASE-6466v3.patch, HBASE-6466-v4.patch, HBASE-6466-=
v4.patch, HBASE-6466-v5.patch
>
>
> If the KV is large or Hlog is closed with high-pressure putting, we found=
 memstore is often above the high water mark and block the putting.
> So should we enable multi-thread for Memstore Flush?
> Some performance test data for reference,
> 1.test environment =EF=BC=9A=20
> random writting=EF=BC=9Bupper memstore limit 5.6GB;lower memstore limit 4=
.8GB;400 regions per regionserver=EF=BC=9Brow len=3D50 bytes, value len=3D1=
024 bytes;5 regionserver, 300 ipc handler per regionserver;5 client, 50 thr=
ead handler per client for writing
> 2.test results:
> one cacheFlush handler, tps: 7.8k/s per regionserver, Flush:10.1MB/s per =
regionserver, appears many aboveGlobalMemstoreLimit blocking
> two cacheFlush handlers, tps: 10.7k/s per regionserver, Flush:12.46MB/s p=
er regionserver,
> 200 thread handler per client & two cacheFlush handlers, tps:16.1k/s per =
regionserver, Flush:18.6MB/s per regionserver

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrato=
rs
For more information on JIRA, see: http://www.atlassian.com/software/jira