hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "takeshi.miao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7525) A canary monitoring program specifically for regionserver
Date Thu, 10 Jan 2013 02:42:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13549310#comment-13549310

takeshi.miao commented on HBASE-7525:

This is for Jonathan Hsieh's question
There are 4 differences compared with #HBASE-4393
1. this tool will take any one region from each region server to monitor, not every region
in whole HBase cluster
2. this tool was implemented with multi-threaded feature, so it will not be blocked if any
region server being hung
3. this tool is taking one or more region server FQDN as options, then will monitor the given
region servers
3.1 monitor all region servers if no option given
4. this tool can also take one or more regular expression patterns for region server FQDN
for user easily use

I use this tool on our internal HBase operation, so I think that other people may have the
identical requirements

> A canary monitoring program specifically for regionserver
> ---------------------------------------------------------
>                 Key: HBASE-7525
>                 URL: https://issues.apache.org/jira/browse/HBASE-7525
>             Project: HBase
>          Issue Type: New Feature
>          Components: monitoring
>    Affects Versions: 0.94.0
>            Reporter: takeshi.miao
>            Priority: Minor
>             Fix For: 0.94.0
>         Attachments: HBASE-7525-v0.patch, RegionServerCanary.java
> *Motivation*
> This ticket is to provide a canary monitoring tool specifically for HRegionserver, details
as follows
> 1. This tool is required by operation team due to they thought that the canary for each
region of a HBase is too many for them, so I implemented this coarse-granular one based on
the original o.a.h.h.tool.Canary for them
> 2. And this tool is implemented by multi-threading, which means the each Get request
sent by a thread. the reason I use this way is due to we suffered the region server hung issue
by now the root cause is still not clear. so this tool can help operation team to detect hung
region server if any.
> *example*
> 1. the tool docs
> ./bin/hbase org.apache.hadoop.hbase.tool.RegionServerCanary -help
> Usage: [opts] [regionServerName 1 [regionServrName 2...]]
>  regionServerName - FQDN serverName, can use linux command:hostname -f to check your
>  where [-opts] are:
>    -help Show this help and exit.
>    -e    Use regionServerName as regular expression
>       which means the regionServerName is regular expression pattern
>    -f <B>         stop whole program if first error occurs, default is true
>    -t <N>         timeout for a check, default is 600000 (milisecs)
>    -daemon        Continuous check at defined intervals.
>    -interval <N>  Interval between checks (sec)
> 2. Will send a request to each regionserver in a HBase cluster
> ./bin/hbase org.apache.hadoop.hbase.tool.RegionServerCanary
> 3. Will send a request to a regionserver by given name
> ./bin/hbase org.apache.hadoop.hbase.tool.RegionServerCanary rs1.domainname
> 4. Will send a request to regionserver(s) by given regular-expression
> /opt/trend/circus-opstool/bin/hbase-canary-monitor-each-regionserver.sh -e rs1.domainname.pattern
> // another example
> ./bin/hbase org.apache.hadoop.hbase.tool.RegionServerCanary -e tw-poc-tm-puppet-hdn[0-9]\{1,2\}.client.tw.trendnet.org
> 5. Will send a request to a regionserver and also set a timeout limit for this test
> // query regionserver:rs1.domainname with timeout limit 10sec
> // -f false, means that will not exit this program even test failed
> ./bin/hbase org.apache.hadoop.hbase.tool.RegionServerCanary -f false -t 10000 rs1.domainname
> // echo "1" if timeout
> echo "$?"
> 6. Will run as daemon mode, which means it will send request to each regionserver periodically
> ./bin/hbase org.apache.hadoop.hbase.tool.RegionServerCanary -daemon

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message