hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Kolesky <j...@opower.com>
Subject Re: Extending HBase Metrics Framework
Date Thu, 06 Jun 2013 19:31:09 GMT
I have not written any code against the latest metrics, so I'm just
searching around.  Take the advice with a grain of salt until a more
experienced developer pipes up.

BaseSourceImpl (implemented for hadoop1 and hadoop2) looks like the entry
point:

https://github.com/apache/hbase/blob/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/metrics/BaseSourceImpl.java

Many implementations exist (also for both hadoop1 and hadoop2).  One such
example is

https://github.com/apache/hbase/blob/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/wal/MetricsWALSourceImpl.java

Jeff


On Thu, Jun 6, 2013 at 11:53 AM, Joarder KAMAL <joarderm@gmail.com> wrote:

> Thanks for pointing out the way. I've looked at the Git code as well
> however looking back at the apidocs <http://hbase.apache.org/apidocs/> for
> HBase-0.97.0-Snapshot I can only see org.apache.hadoop.metrics2 package
> there.
>
> I am using HBase 0.94.7, could you kindly help directing to the right code
> repo and package?
>
> Thanks again...
>
>
> Regards,
> Joarder Kamal
>
>
> On 7 June 2013 03:50, Jeff Kolesky <jeff@opower.com> wrote:
>
> > If you want to add metrics, you can implement a
> > org.apache.hadoop.metrics.Updater and register it with a MetricsContext.
> >  There is an example at:
> >
> >
> >
> https://github.com/jeffkole/odoop/blob/master/hbase-filters/src/main/java/com/opower/hadoop/hbase/filter/DeployedFilterMetrics.java
> >
> > This exposes metrics for a filter that I wrote, but if you want to pull
> out
> > metrics for HBase internals, you likely need to do modify HBase source
> code
> > to get at the data or hook up to your metric updater.
> >
> > Jeff
> >
> >
> > On Thu, Jun 6, 2013 at 10:35 AM, Joarder KAMAL <joarderm@gmail.com>
> wrote:
> >
> > > Hi Bing,
> > >
> > > Thanks for answering.
> > >
> > > 1. Probably my question was unclear. I was asking whether it is
> possible
> > to
> > > generate custom metrics from HBase by modifying the source code.
> > > 2. Thanks for letting me know that it is possible to log the metrics in
> > any
> > > kind of file. But I guess it would be still in XML format !! Can I save
> > the
> > > logs in raw text format?? I'll try to do this tomorrow.
> > >
> > > Thanks again.
> > >
> > >
> > > Regards,
> > > Joarder Kamal
> > >
> > >
> > >
> > > On 6 June 2013 17:42, Bing Jiang <jiangbinglover@gmail.com> wrote:
> > >
> > > > 1) Metrics infos have been collected from HMaster\RS, and covering
> rpc,
> > > > jvm, hbase functional module (like compaction\flush\request
> > > Count\DataBlock
> > > > read &&write etc).
> > > >
> > > > 2) You can set metrics info handler in conf/hadoop-metrics.properties
> > > > for example, you can set hbase.class =
> > > > org.apache.hadoop.metrics.ganglia.GangliaContext31, it will tell
> > Metrics
> > > > System where the metrics will be sent to. Of course, this is Ganglia.
> > > > if set
> > > >
> > hbase.class=org.apache.hadoop.hbase.metrics.file.TimeStampingFileContext,
> > > > it will write the metrics to file.
> > > >
> > > > # hbase.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> > > > # hbase.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
> > > > # hbase.period=10
> > > > # hbase.servers=10.100.4.151:8650
> > > >
> > #hbase.class=org.apache.hadoop.hbase.metrics.file.TimeStampingFileContext
> > > > #hbase.period=10
> > > > #hbase.fileName=/opt/hbase/logs/metrics_hbase.log
> > > >
> > > >
> > > >
> > > >
> > > > 2013/6/6 Joarder KAMAL <joarderm@gmail.com>
> > > >
> > > > > Two Questions:
> > > > >
> > > > > 1. Is it possible to extend HBase metrics/reporting framework to
> > > generate
> > > > > custom metrics? I was looking at the api docs
> > > > > http://hbase.apache.org/apidocs/
> > > > > 2. Is there any way to write my own client (instead of using
> Ganglia
> > or
> > > > > else) to retrieve the currently available metrics?
> > > > >
> > > > > HBase version: 0.94.7
> > > > >
> > > > > Any suggestions will be very helpful.
> > > > >
> > > > > Many thanks ...
> > > > >
> > > > > Regards,
> > > > > Joarder Kamal
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Bing Jiang
> > > > Tel:(86)134-2619-1361
> > > > weibo: http://weibo.com/jiangbinglover
> > > > BLOG: http://blog.sina.com.cn/jiangbinglover
> > > > National Research Center for Intelligent Computing Systems
> > > > Institute of Computing technology
> > > > Graduate University of Chinese Academy of Science
> > > >
> > >
> >
> >
> >
> > --
> > *Jeff Kolesky*
> > Chief Software Architect
> > *Opower*
> >
>



-- 
*Jeff Kolesky*
Chief Software Architect
*Opower*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message