hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Virag Kothari (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11610) Enhance remote meta updates
Date Mon, 25 Aug 2014 15:45:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109223#comment-14109223
] 

Virag Kothari commented on HBASE-11610:
---------------------------------------

bq. It's great that v3 is both simpler and more efficient!

Its ~5% faster than v2 which creates a seperate HTable for each put. Compared to the threadLocal
HTable (original patch), it is ~5% slower. (this experiments are done over series of 5 runs).


bq. is MultiHConnection that useful? I see its size is defaulted to 1? have you compare the
performances with greater values? 

All the above experiments are done with 10 HConnections. It improves the write throughput
on META server by ~15% compared to single HConnection when doing assignment for 1M regions.
But most users will have significantly lower than 1M regions, so by default we only will have
one HConnection.
I had tried with 15 connections but didn't see any increase in throughput. 

bq. Each Connection will come with a pool of 256 threads (there is HBASE-11590 to improve
this a little), plus the one of MultiHConnection

I believe that pool is only created when you do a getTable() on HConnection. As we are not
going the HTable route, all threads accessing regionStateStore will use the single pool managed
by MultiHConnection. We can get rid of this pool when we have a new processBatchCallBack API
which maintains a pool (though that will be per connection).

bq. we should have some thing like hbase.regionstatestore.meta.threads.max instead of hbase.hconnection.threads.max)

how about hbase.multihconnection.threads.max as the pool is part of MultiHconnection?

bq. I like the patch

Thanks!

Will address comments shortly.




 


> Enhance remote meta updates
> ---------------------------
>
>                 Key: HBASE-11610
>                 URL: https://issues.apache.org/jira/browse/HBASE-11610
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Jimmy Xiang
>            Assignee: Virag Kothari
>         Attachments: HBASE-11610.patch, HBASE-11610_2.patch, HBASE-11610_v3.patch
>
>
> Currently, if the meta region is on a regionserver instead of the master, meta update
is synchronized on one HTable instance. We should be able to do better.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message