hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: MATA load balance
Date Wed, 03 Feb 2010 08:32:54 GMT
A little birdy told me that META performance can potentially degrade
with a high # of store files, so try to major_compact '.META.' first.

Secondly, yes META can be a bottleneck, but it should serve out of ram
nearly constantly. Combined with longer lived clients, this should
mitigate things somewhat.

One option is to use a long lived gateway process, eg: thrift, which
will amortize the cost of the META lookup over many small client
connections.  This is what I do with PHP, and it works well.


2010/2/3  <y_823910@tsmc.com>:
> Hi,
> Our cluster with 3 zookeepers, 10 region servers, 19 data nodes.
> Each machine has 4 core cpu, 12G ram.
> There are 1322 regions in our cluster now.
> We fired up to 3000 hbase client in parallel to fetch hbase data for
> distributed computing.
> Despite of Htable just one time visit to MATA table; there is only one
> server with the MATA information,
> it seems a bottleneck while I fired so many clients at the same time.
> Any suggestions?
> Fleming Chiu(邱宏明)
> 707-6128
> y_823910@tsmc.com
> 週一無肉日吃素救地球(Meat Free Monday Taiwan)
>  ---------------------------------------------------------------------------
>                                                         TSMC PROPERTY
>  This email communication (and any attachments) is proprietary information
>  for the sole use of its
>  intended recipient. Any unauthorized review, use or distribution by anyone
>  other than the intended
>  recipient is strictly prohibited.  If you are not the intended recipient,
>  please notify the sender by
>  replying to this email, and then delete this email and any copies of it
>  immediately. Thank you.
>  ---------------------------------------------------------------------------

View raw message