hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yun peng <pengyunm...@gmail.com>
Subject Re: How to check if a major_compact is done?
Date Thu, 08 Nov 2012 20:58:38 GMT
Hi, Mikael and Jeremy, thanks for your detailed answers. I have tried the
MBean code as an attempt. I get certain information out from the printout,
like follows,
...
javax.management.MBeanAttributeInfo[description=NoDescription,
name=compactionTimeNumOps, type=java.lang.Integer, read-only, descriptor={}]
...

I have a minor question, (sorry about the extra questions, but I am sort of
newbie to web service interfaces). The descriptor (or the content) of
MBeanAttributeInfo here is empty, which does not match with the data I can
see through jconsole, which show compactionTimeNumOps=4 (after issuing two
major compaction requests). Should I pause the data/content from somewhere
other than MBeanAttributeInfo.getDescriptor().

Regards,
Yun


On Thu, Nov 8, 2012 at 2:36 PM, Mikael Sitruk <mikael.sitruk@gmail.com>wrote:

> Yun hi
>
> I did something like this in the past (in 0.92.0 version). That trigger
> major compaction and monitor their completion. I did not go through the JMX
> (which is not an indicator of major compaction), but via
> the org.apache.hadoop.hbase.HServerLoad.RegionLoad
>  class.
> When an major compaction ends the number of store file will be the number
> of store which can be checked via: regionLoad.getStorefiles() ==
> task.getStores()
>
> I will try to find the code back and post it on github.
>
>
> As for MBeans you can do somehting like:
>
> url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://" + hostName + ":" +
> portNum + "/jmxrmi");
> HashMap<String, String> env = new HashMap<String, String>();
> JMXConnector jmxc = JMXConnectorFactory.connect(url, env);
> MBeanServerConnection mbsc = jmxc.getMBeanServerConnection();
> ObjectName rsStat = new ObjectName(
> "hadoop:service=RegionServer,name=RegionServerStatistics");   //put the
> mbean you need here.
> MBeanInfo mbeanInfo = mbsc.getMBeanInfo(rsStat);
>
> for (MBeanAttributeInfo mbAttrInfo : mbeanInfo.getAttributes()) {
> System.out.println(mbAttrInfo);
> }
>
>
> Regards,
> Mikael.S
>
> On Thu, Nov 8, 2012 at 8:31 PM, yun peng <pengyunmomo@gmail.com> wrote:
>
> > Yes, JMX exposes compaction time. My cluster has JMX enabled, and I can
> > view certain statistics from remote jconsole.
> >
> > However, I dont find any reference talking about how to write a Java
> > program to collect a specific statistics, like compaction time available
> in
> > region server status. Does anyone here have experience in writing
> low-level
> > code to collect JMX data, instead of using jconsole or other high-level
> > tools...
> >
> > PS: I have found some sample code
> > here<
> >
> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/metrics/package-summary.html
> > >,
> > yet the API is deprecated.
> > Regards,
> > Yun
> > On Thu, Nov 8, 2012 at 6:43 AM, Jean-Marc Spaggiari <
> > jean-marc@spaggiari.org
> > > wrote:
> >
> > > Please someone correct me if I'm wrong, but I think there is some
> > > information exposed to JMX which give you the duration (and size) of
> > > the last compaction.
> > >
> > > JM
> > >
> > > 2012/11/8, PG <pengyunmomo@gmail.com>:
> > > > Hi, thanks for the comments. One thing is,,,, shouldn't web UI comes
> > from
> > > > the hbase API, or can I issue function call to get the progress of
> > > > compaction?.
> > > > Hun
> > > >
> > > > On Nov 8, 2012, at 1:33 AM, ramkrishna vasudevan
> > > > <ramkrishna.s.vasudevan@gmail.com> wrote:
> > > >
> > > >> There is no interface which says that the major compaction is
> > completed.
> > > >> But you can see that major compaction is in progress from the web
> UI.
> > > >> Sorry if am wrong here.
> > > >>
> > > >> Regards
> > > >> Ram
> > > >>
> > > >> On Thu, Nov 8, 2012 at 11:38 AM, yun peng <pengyunmomo@gmail.com>
> > > wrote:
> > > >>
> > > >>> Hi, All,
> > > >>> I want to measure the duration of a major compaction in HBase.
> Since
> > > the
> > > >>> function call majorCompact is asynchronous, I may need to manually
> > > check
> > > >>> when the major compaction is done. Does Hbase (as of version
> 0.92.4)
> > > >>> provide an interface to determine completion of major compaction?
> > > >>> Thanks.
> > > >>> Yun
> > > >>>
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message