hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13996) Add write sniffing in canary
Date Tue, 18 Aug 2015 17:07:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701605#comment-14701605

Andrew Purtell commented on HBASE-13996:

No problem at all [~liushaohui]. 

This is fine for when the canary is using the default canary table name "hbase:canary". It's
the explanation to the user that Stack asked for.
+        } else if (tableName.equals(Canary.CANARY_TABLE_NAME)){
+            description = "The hbase:canary table is used to sniff the write availbility
of each regionserver.";

However, I think the tool should allow for the user to give a different name for the canary
table. The default can be "hbase:canary" of course and nobody would need to change it today.
However I can see a future time when we could have different resource allocation and admission
control policies by namespace and in that case a user might want to run a separate canary
instance for the different namespaces that are in use. 

Since we are publishing write timing, should the size of the value being written be configurable?
This way a user can also watch for latency blips when processing their use case's average
or p95 or whatever value size. Should be fine, we will hold down the total amount of data
written because each iteration of the canary will rewrite the same locations. Minor detail,
feel free to ignore.

Should this interval be configurable?
+            // check canary distribution for every 10 minutes

Should the slop factor be configurable?
+      if (numberOfRegions < numberOfServers * regionsPerServer * 0.7
+          || numberOfRegions > numberOfServers * regionsPerServer * 1.5) {
Why 30% less but 50% more?

Ah, even better, we are setting TTLs on the canary family data. Great. Should this be adjustable?
Perhaps with a command line option.
+      family.setTimeToLive(24 * 60 * 60 * 1000);

Should be good feedback for now (smile)

> Add write sniffing in canary
> ----------------------------
>                 Key: HBASE-13996
>                 URL: https://issues.apache.org/jira/browse/HBASE-13996
>             Project: HBase
>          Issue Type: Improvement
>          Components: canary
>    Affects Versions: 0.98.13,
>            Reporter: Liu Shaohui
>            Assignee: Liu Shaohui
>             Fix For: 2.0.0, 1.3.0, 0.98.15
>         Attachments: HBASE-13996-v001.diff, HBASE-13996-v002.diff
> Currently the canary tool only sniff the read operations, it's hard to finding the problem
in write path. 
> To support the write sniffing, we create a system table named '_canary_'  in the canary
tool. And the tool will make sure that the region number is large than the number of the regionserver
and the regions will be distributed onto all regionservers.
> Periodically, the tool will put data to these regions to calculate the write availability
of HBase and send alerts if needed.

This message was sent by Atlassian JIRA

View raw message