hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikael Sitruk <mikael.sit...@gmail.com>
Subject Re: How to check if a major_compact is done?
Date Thu, 08 Nov 2012 19:36:37 GMT
Yun hi

I did something like this in the past (in 0.92.0 version). That trigger
major compaction and monitor their completion. I did not go through the JMX
(which is not an indicator of major compaction), but via
the org.apache.hadoop.hbase.HServerLoad.RegionLoad
 class.
When an major compaction ends the number of store file will be the number
of store which can be checked via: regionLoad.getStorefiles() ==
task.getStores()

I will try to find the code back and post it on github.


As for MBeans you can do somehting like:

url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://" + hostName + ":" +
portNum + "/jmxrmi");
HashMap<String, String> env = new HashMap<String, String>();
JMXConnector jmxc = JMXConnectorFactory.connect(url, env);
MBeanServerConnection mbsc = jmxc.getMBeanServerConnection();
ObjectName rsStat = new ObjectName(
"hadoop:service=RegionServer,name=RegionServerStatistics");   //put the
mbean you need here.
MBeanInfo mbeanInfo = mbsc.getMBeanInfo(rsStat);

for (MBeanAttributeInfo mbAttrInfo : mbeanInfo.getAttributes()) {
System.out.println(mbAttrInfo);
}


Regards,
Mikael.S

On Thu, Nov 8, 2012 at 8:31 PM, yun peng <pengyunmomo@gmail.com> wrote:

> Yes, JMX exposes compaction time. My cluster has JMX enabled, and I can
> view certain statistics from remote jconsole.
>
> However, I dont find any reference talking about how to write a Java
> program to collect a specific statistics, like compaction time available in
> region server status. Does anyone here have experience in writing low-level
> code to collect JMX data, instead of using jconsole or other high-level
> tools...
>
> PS: I have found some sample code
> here<
> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/metrics/package-summary.html
> >,
> yet the API is deprecated.
> Regards,
> Yun
> On Thu, Nov 8, 2012 at 6:43 AM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org
> > wrote:
>
> > Please someone correct me if I'm wrong, but I think there is some
> > information exposed to JMX which give you the duration (and size) of
> > the last compaction.
> >
> > JM
> >
> > 2012/11/8, PG <pengyunmomo@gmail.com>:
> > > Hi, thanks for the comments. One thing is,,,, shouldn't web UI comes
> from
> > > the hbase API, or can I issue function call to get the progress of
> > > compaction?.
> > > Hun
> > >
> > > On Nov 8, 2012, at 1:33 AM, ramkrishna vasudevan
> > > <ramkrishna.s.vasudevan@gmail.com> wrote:
> > >
> > >> There is no interface which says that the major compaction is
> completed.
> > >> But you can see that major compaction is in progress from the web UI.
> > >> Sorry if am wrong here.
> > >>
> > >> Regards
> > >> Ram
> > >>
> > >> On Thu, Nov 8, 2012 at 11:38 AM, yun peng <pengyunmomo@gmail.com>
> > wrote:
> > >>
> > >>> Hi, All,
> > >>> I want to measure the duration of a major compaction in HBase. Since
> > the
> > >>> function call majorCompact is asynchronous, I may need to manually
> > check
> > >>> when the major compaction is done. Does Hbase (as of version 0.92.4)
> > >>> provide an interface to determine completion of major compaction?
> > >>> Thanks.
> > >>> Yun
> > >>>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message