hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9531) a command line (hbase shell) interface to retreive the replication metrics and show replication lag
Date Wed, 25 Jun 2014 01:34:24 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042936#comment-14042936
] 

Andrew Purtell commented on HBASE-9531:
---------------------------------------

This looks plausible to me and I like the clean extension of the existing 'status' command.


{code}
+  // A default Load for the case of no replication
+  private static final HashMap<String, String> noReplicationLoadMap = new HashMap<String,
String>();
+  static {
+    noReplicationLoadMap.put(REPLICATIONLOADSOURCE, "Replication disabled");
+    noReplicationLoadMap.put(REPLICATIONLOADSINK, "Replication disabled");
+  }
+
{code}

Can we just not set the new fields in ClusterStatus if replication is not active?

In ReplicationLoad.java, please don't start method names with capital letters.

The default status command is 'summary', so we shouldn't dump all of the source and sink information
as default, that's not a summary by definition.

{code}
diff --git hbase-shell/src/main/ruby/shell/commands/status.rb hbase-shell/src/main/ruby/shell/commands/status.rb
index f72c13c..4654b4a 100644
--- hbase-shell/src/main/ruby/shell/commands/status.rb
+++ hbase-shell/src/main/ruby/shell/commands/status.rb
@@ -22,18 +22,22 @@ module Shell
     class Status < Command
       def help
         return <<-EOF
-Show cluster status. Can be 'summary', 'simple', or 'detailed'. The
+Show cluster status. Can be 'summary', 'simple', 'detailed', or 'replication'. The
 default is 'summary'. Examples:
 
   hbase> status
   hbase> status 'simple'
   hbase> status 'summary'
   hbase> status 'detailed'
+  hbase> status 'replication'
+  hbase> status 'replication', 'source'
+  hbase> status 'replication', 'sink'
+
 EOF
       end
 
-      def command(format = 'summary')
-        admin.status(format)
+      def command(format = 'summary',type = 'both')
+        admin.status(format,type)
       end
     end
   end
{code}


> a command line (hbase shell) interface to retreive the replication metrics and show replication
lag
> ---------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-9531
>                 URL: https://issues.apache.org/jira/browse/HBASE-9531
>             Project: HBase
>          Issue Type: New Feature
>          Components: Replication
>    Affects Versions: 0.99.0
>            Reporter: Demai Ni
>            Assignee: Demai Ni
>             Fix For: 0.99.0, 0.98.4
>
>         Attachments: HBASE-9531-trunk-v0.patch, HBASE-9531-trunk-v0.patch
>
>
> This jira is to provide a command line (hbase shell) interface to retreive the replication
metrics info such as:ageOfLastShippedOp, timeStampsOfLastShippedOp, sizeOfLogQueue ageOfLastAppliedOp,
and timeStampsOfLastAppliedOp. And also to provide a point of time info of the lag of replication(source
only)
> Understand that hbase is using Hadoop metrics(http://hbase.apache.org/metrics.html),
which is a common way to monitor metric info. This Jira is to serve as a light-weight client
interface, comparing to a completed(certainly better, but heavier)GUI monitoring package.
I made the code works on 0.94.9 now, and like to use this jira to get opinions about whether
the feature is valuable to other users/workshop. If so, I will build a trunk patch. 
> All inputs are greatly appreciated. Thank you!
> The overall design is to reuse the existing logic which supports hbase shell command
'status', and invent a new module, called ReplicationLoad.  In HRegionServer.buildServerLoad()
, use the local replication service objects to get their loads  which could be wrapped in
a ReplicationLoad object and then simply pass it to the ServerLoad. In ReplicationSourceMetrics
and ReplicationSinkMetrics, a few getters and setters will be created, and ask Replication
to build a "ReplicationLoad".  (many thanks to Jean-Daniel for his kindly suggestions through
dev email list)
> the replication lag will be calculated for source only, and use this formula: 
> {code:title=Replication lag|borderStyle=solid}
> 	if sizeOfLogQueue != 0 then max(ageOfLastShippedOp, (current time - timeStampsOfLastShippedOp))
//err on the large side
> 	else if (current time - timeStampsOfLastShippedOp) < 2* ageOfLastShippedOp then lag
= ageOfLastShippedOp // last shipped happen recently 
>         else lag = 0 // last shipped may happens last night, so NO real lag although
ageOfLastShippedOp is non-zero
> {code}
> External will look something like:
> {code:title=status 'replication'|borderStyle=solid}
> hbase(main):001:0> status 'replication'
> version 0.94.9
> 3 live servers
>     hdtest017.svl.ibm.com:
>         SOURCE:PeerID=1, ageOfLastShippedOp=14, sizeOfLogQueue=0, timeStampsOfLastShippedOp=Wed
Sep 04 14:49:48 PDT 2013
>         SINK  :AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Wed Sep 04 14:48:48
PDT 2013
>     hdtest018.svl.ibm.com:
>         SOURCE:PeerID=1, ageOfLastShippedOp=0, sizeOfLogQueue=0, timeStampsOfLastShippedOp=Wed
Sep 04 14:48:48 PDT 2013
>         SINK  :AgeOfLastAppliedOp=14, TimeStampsOfLastAppliedOp=Wed Sep 04 14:50:59
PDT 2013
>     hdtest015.svl.ibm.com:
>         SOURCE:PeerID=1, ageOfLastShippedOp=0, sizeOfLogQueue=0, timeStampsOfLastShippedOp=Wed
Sep 04 14:48:48 PDT 2013
>         SINK  :AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Wed Sep 04 14:48:48
PDT 2013
> hbase(main):002:0> status 'replication','source'
> version 0.94.9
> 3 live servers
>     hdtest017.svl.ibm.com:
>         SOURCE:PeerID=1, ageOfLastShippedOp=14, sizeOfLogQueue=0, timeStampsOfLastShippedOp=Wed
Sep 04 14:49:48 PDT 2013
>     hdtest018.svl.ibm.com:
>         SOURCE:PeerID=1, ageOfLastShippedOp=0, sizeOfLogQueue=0, timeStampsOfLastShippedOp=Wed
Sep 04 14:48:48 PDT 2013
>     hdtest015.svl.ibm.com:
>         SOURCE:PeerID=1, ageOfLastShippedOp=0, sizeOfLogQueue=0, timeStampsOfLastShippedOp=Wed
Sep 04 14:48:48 PDT 2013
> hbase(main):003:0> status 'replication','sink'
> version 0.94.9
> 3 live servers
>     hdtest017.svl.ibm.com:
>         SINK  :AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Wed Sep 04 14:48:48
PDT 2013
>     hdtest018.svl.ibm.com:
>         SINK  :AgeOfLastAppliedOp=14, TimeStampsOfLastAppliedOp=Wed Sep 04 14:50:59
PDT 2013
>     hdtest015.svl.ibm.com:
>         SINK  :AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Wed Sep 04 14:48:48
PDT 2013
> hbase(main):003:0> status 'replication','lag' 
> version 0.94.9
> 3 live servers
>     hdtest017.svl.ibm.com: lag = 0
>     hdtest018.svl.ibm.com: lag = 14
>     hdtest015.svl.ibm.com: lag = 0
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message