hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Brown <...@siftscience.com>
Subject Re: High CPU Utilization by meta region
Date Mon, 28 Nov 2016 14:34:44 GMT
Hi Ted,

The region server hosting hbase:meta only has the meta region on it so it
has 1 region while other region servers can have more than 100 regions on
them.
I didn't notice anything interesting in the logs in my opinion. Is there
anything in particular I should watch out for?
The hbase:meta table was major compacted yesterday and we're still
experiencing the issue.

Thanks for the quick response,
Tim


On Mon, Nov 28, 2016 at 5:45 AM, Ted Yu <ted_yu@yahoo.com.invalid> wrote:

> Does the region server hosting hbase:meta have roughly the same number of
> regions as the other servers ?
> Did you find anything interesting in the server log (where hbase:meta is
> hosted) ?
> Have you tried major compacting the hbase:meta table ?
> In 1.2, DEFAULT_HBASE_META_VERSIONS is still 10. See HBASE-16832
>
>
>     On Sunday, November 27, 2016 6:53 PM, Timothy Brown <
> tim@siftscience.com> wrote:
>
>
>  Hi Everyone,
>
> I apologize for starting an additional thread about this but I wasn't
> subscribed to the users mailing list when I sent the original and can't
> figure out how to respond to the original :(
>
> Original Message:
>
> We are seeing about 80% CPU utilization on the Region Server that solely
> serves the meta table while other region servers typically have under 50%
> CPU utilization. Is this expected?
>
> Here's some more info about our cluster:
> HBase version 1.2
> Number of regions: 72
> Number of tables: 97
> Approx. requests per second to meta region server: 3k
> Approx. requests per second to entire HBase cluster: 90k
>
> Additional info:
>
>
> From Storefile Metrics:
> Stores Num: 1
> Storefiles: 1
> Storefile Size: 30m
> Uncompressed Storefile Size: 30m
> Index Size: 459k
>
>
> I/O for the region server with only meta on it:
> 48M bytes in
> 5.9B bytes out
>
> I used the debug dump on the region server's UI but it was too large
> for paste bin so here's a portion of it: http://pastebin.com/nkYhEceE
>
>
> Thanks for the help,
>
> Tim
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message