hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: trimming RegionLoad fields
Date Thu, 17 Mar 2011 21:52:43 GMT
Without solid evidence of "we'll be saving X megabytes" I don't see a
compelling reason to hacking that stuff out yet.

We sort of need a better out-of-the-box monitoring system. One idea I
had was to embed OpenTSDB inside the HMaster. This way OpenTSDB would
store info about a HBase cluster back in the same cluster it monitors.
While this may sound "weird" I think it makes sense because every
great database system provides strong self monitoring tools. Eg:
Oracle, etc.  Due to the LGPL, this is not currently viable.  Perhaps
there is an alternative floating out there we can ship with? And not
ganglia :-)

On Thu, Mar 17, 2011 at 2:47 PM, Andrew Purtell <apurtell@apache.org> wrote:
> memstoreSizeMB is part of the output printed by the shell when you do status 'detailed'.
>
> I use that.
>
> Isn't that information useful to others?
>
>  - Andy
>
> --- On Thu, 3/17/11, Ryan Rawson <ryanobjc@gmail.com> wrote:
>
>> From: Ryan Rawson <ryanobjc@gmail.com>
>> Subject: Re: trimming RegionLoad fields
>> To: dev@hbase.apache.org
>> Cc: "Ted Yu" <yuzhihong@gmail.com>
>> Date: Thursday, March 17, 2011, 2:37 PM
>> How much memory does profiling
>> indicating these objects use?  How much
>> are you expecting to save?
>>
>> Saving 4-8 bytes even on a 10k region cluster is still only
>> 80k of ram, not really significant.
>>
>>
>> On Thu, Mar 17, 2011 at 2:32 PM, Ted Yu <yuzhihong@gmail.com>
>> wrote:
>> > Hi,
>> > See email thread 'One of the regionserver aborted,
>> then the master shut down
>> > itself' for background.
>> > I am evaluating various ways of trimming the memory
>> footprint of RegionLoad
>> > because there would be so many regions in production
>> cluster.
>> >
>> > Looking at field memstoreSizeMB of RegionLoad, I only
>> found this reference -
>> > AvroUtil.hslToASL()
>> > Load balancer currently isn't checking this metric.
>> And HRegion has
>> > memstoreSize field.
>> >
>> > I wonder whether we can trim field memstoreSizeMB off
>> RegionLoad.
>> >
>> > Please comment.
>> >
>>
>
>
>
>

Mime
View raw message