hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Schäfer <syrious3...@hotmail.de>
Subject Re: Capturing RegionServerMetrics during inserts
Date Tue, 10 Jan 2012 23:51:39 GMT

Hi Mikael,

shame on me I did that.

You're totally right that it is  by chance if it the round second.

Now I use Thread.sleep ...benchmark have nearly doubled.

Thanks for the simple hint..


--------
Christian hi

In short this is because you cannot assume that your loop will get exactly
there by a whole sec. There are a lot of other internal process running in
JVM they will make your "if" irrelevant.

Generally using busy wait loop is not the best way to do periodic task like
you want to. You can use Thread.sleep(1000) approach (it will be better
than your current situation), again it is not 100% accurate but more than
you did.

Cheers

Mikael.S
On Jan 10, 2012 2:23 AM, "Christian Schäfer" <syrious3000@hotmail.de> wrote:

>
> I just wonder why it isn't able to print the server load info every second
> to the console with the following code.
> Instead it just prints irregularly in time whats very disadvantageous
> because I want to make a simple requests/second diagram where there has to
> be a value each second.
>
> (Think using Ganglia is overkill for my simple test here)
>
>  public void getRegionServerMetrics() throws Exception{
>        HBaseAdmin admin = new HBaseAdmin(config);
>        Collection<HServerInfo> serverInfos = new ArrayList<HServerInfo>();
>        HMasterInterface masterIF =
>  admin.getMaster();
>
>        long timeStamp = 0L;
>
>        while (true) {
>            timeStamp = System.currentTimeMillis();
>            if(( timeStamp % 1000) == 0){
>                serverInfos = masterIF.getClusterStatus().getServerInfo();
>
>                System.out.println(timeStamp +" - "+serverInfos);
>            }
>        }
>    }
>
> Still tried it on two different clusters from different networks.
> ---------
>
> Perfect...thats exactly what I was looking for.
> Thanks a lot for the working code :-)
>       Von: Mikael Sitruk <mikael.sitruk@gmail.com>
>  An: user@hbase.apache.org
>  Gesendet: 18:01 Sonntag, 8.Januar 2012
>  Betreff: Re: Capturing RegionServerMetrics during inserts
>
> If you just want the load you can do the
>  below
> HBaseAdmin admin;
>
> try {
>
> admin = new HBaseAdmin(m_hbConfig);
>
> for (HServerInfo info :
> admin.getMaster().getClusterStatus().getServerInfo()){
> System.out.println(info.getServerName() + "-->" + info.getLoad());
> }
>
> } catch (Exception e) {
> throw new Exception("Failed to get load", e);
> }
>
>
> Output
> server1,60020,1326017646195-->requests=0, regions=100, usedHeap=9077,
> maxHeap=12261
> server2,60020,1326017643782-->requests=0, regions=100, usedHeap=2755,
> maxHeap=12261
> server3,60020,1326017643727-->requests=0, regions=100, usedHeap=6256,
> maxHeap=12261
> server4,60020,1326017643722-->requests=0, regions=101, usedHeap=7913,
> maxHeap=12261
>
>
> But if you want the metrics you will need to go with Alex suggestion
>
> Mikael.S
>
> On Sun, Jan 8, 2012 at 5:04 PM, Alex Baranau <alex.baranov.v@gmail.com
> >wrote:
>
> > Hi,
> >
> > According to your code:
> >
> > > RegionServerMetrics metrics = new RegionServerMetrics();
> >
> > I think that's why it is always empty.
> >
> >
> > I believe that the right way to get those stats is to use data exposed
> via
> > JMX by the RegionServer.
> > Alternatively you can poll RegionServer web ui periodically and parse
> html
> > to fetch data you want.
> >
> > Alex Baranau
> > ----
> > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Hadoop -
> HBase
> >
> > On Sat, Jan 7, 2012 at 10:51 AM, Christian Schäfer
> > <syrious3000@hotmail.de>wrote:
> >
> > >
> > > Hello,
> > >
> > > I want to
>  measure requests per second for each Region Server during
> > > inserts on a table that is spread over this Region Servers.
> > >
> > > So during inserts I run local java app to grab the Region Server
> metrics:
> > >
> > >  public void getRegionServerMetrics() throws Exception{
> > >        HBaseAdmin admin = new HBaseAdmin(config);
> > >        HTable table = new HTable(config, TABLE_NAME);
> > >
> > >        Map<HRegionInfo, HServerAddress> regionInfo =
> > > table.getRegionsInfo();
> > >
> > >        RegionServerMetrics metrics = new RegionServerMetrics();
> > >        MetricsIntValue value =
> > >  metrics.requests;
> > >
> > >        while(true){
> > >
>  System.out.println(metrics.toString());
> > >            Thread.sleep(1000);
> > >        }
> > >    }
> > >
> > > But it prints just:
> > >
> > > requests=0,
> > >  regions=0, stores=0, storefiles=0, storefileIndexSize=0,
> > > memstoreSize=0, compactionQueueSize=0, flushQueueSize=0, usedHeap=14,
> > > maxHeap=865, blockCacheSize=0, blockCacheFree=0, blockCacheCount=0,
> > > blockCacheHitCount=0, blockCacheMissCount=0, blockCacheEvictedCount=0,
> > > blockCacheHitRatio=0, blockCacheHitCachingRatio=0
> > >
> > > Although the WebGUI of HMaster shows that there are requests for all
> > > Region Servers.
> > >
> > > The TABLE_NAME is equal for the inserting app and the metrics app.
> > >
> > > Any suggestions?
> >
>
>
>
> --
> Mikael.S 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message