hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12665) When aborting, dump metrics
Date Wed, 10 Dec 2014 00:58:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240438#comment-14240438
] 

stack commented on HBASE-12665:
-------------------------------

Since it small again, putting into a String and logging it like we used to.  Looks like this
now:

{code}
776 2014-12-09 16:55:21,337 FATAL [main] regionserver.HRegionServer(1921): ABORTING region
server 192.168.1.241,63193,1418172920801: I'm dead
 777 2014-12-09 16:55:21,338 FATAL [main] regionserver.HRegionServer(1927): RegionServer abort:
loaded coprocessors are: [org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint]
 778 2014-12-09 16:55:21,358 INFO  [main] regionserver.HRegionServer(1931): Dump of metrics
as JSON on abort: {
 779   "beans" : [ {
 780     "name" : "java.lang:type=Memory",
 781     "modelerType" : "sun.management.MemoryImpl",
 782     "HeapMemoryUsage" : {
 783       "committed" : 317718528,
 784       "init" : 268435456,
 785       "max" : 1771044864,
 786       "used" : 192265360
 787     },
 788     "Verbose" : false,
 789     "ObjectPendingFinalizationCount" : 0,
 790     "NonHeapMemoryUsage" : {
 791       "committed" : 58130432,
 792       "init" : 24576000,
 793       "max" : 318767104,
 794       "used" : 57784928
 795     },
 796     "ObjectName" : "java.lang:type=Memory"
 797   } ],
 798   "beans" : [ {
 799     "name" : "Hadoop:service=HBase,name=RegionServer,sub=IPC",
 800     "modelerType" : "RegionServer,sub=IPC",
 801     "tag.Context" : "regionserver",
 802     "tag.Context" : "regionserver",
 803     "tag.Hostname" : "kalashnikov-20.local",
 804     "queueSize" : 0,
 805     "numCallsInGeneralQueue" : 0,
 806     "numCallsInReplicationQueue" : 0,
 807     "numCallsInPriorityQueue" : 0,
 808     "numOpenConnections" : 1,
 809     "numActiveHandler" : 0,
 810     "QueueCallTime_num_ops" : 5,
 811     "QueueCallTime_min" : 0,
 812     "QueueCallTime_max" : 0,
 813     "QueueCallTime_mean" : 0.0,
 814     "QueueCallTime_median" : 0.0,
 815     "QueueCallTime_75th_percentile" : 0.0,
 816     "QueueCallTime_95th_percentile" : 0.0,
 817     "QueueCallTime_99th_percentile" : 0.0,
 818     "authenticationFailures" : 0,
 819     "authorizationFailures" : 0,
 820     "authenticationSuccesses" : 0,
 821     "authorizationSuccesses" : 4,
 822     "ProcessCallTime_num_ops" : 5,
 823     "ProcessCallTime_min" : 2,
 824     "ProcessCallTime_max" : 7,
 825     "ProcessCallTime_mean" : 4.0,
 826     "ProcessCallTime_median" : 3.0,
 827     "ProcessCallTime_75th_percentile" : 6.0,
 828     "ProcessCallTime_95th_percentile" : 7.0,
 829     "ProcessCallTime_99th_percentile" : 7.0,
 830     "sentBytes" : 103,
 831     "receivedBytes" : 780
 832   } ],
 833   "beans" : [ {
 834     "name" : "Hadoop:service=HBase,name=RegionServer,sub=Replication",
 835     "modelerType" : "RegionServer,sub=Replication",
 836     "tag.Context" : "regionserver",
 837     "tag.Hostname" : "kalashnikov-20.local",
 838     "sink.appliedOps" : 0,
 839     "sink.appliedBatches" : 0,
 840     "sink.ageOfLastAppliedOp" : 0
 841   } ],
 842   "beans" : [ {
 843     "name" : "Hadoop:service=HBase,name=RegionServer,sub=Server",
 844     "modelerType" : "RegionServer,sub=Server",
 845     "tag.Context" : "regionserver",
 846     "tag.zookeeperQuorum" : "localhost:63768",
 847     "tag.serverName" : "192.168.1.241,63135,1418172857470",
 848     "tag.clusterId" : "58b09b0b-207c-4df3-8e43-9bb98a2ff37b",
 849     "tag.Context" : "regionserver",
 850     "tag.Hostname" : "kalashnikov-20.local",
 851     "regionCount" : 2,
 852     "storeCount" : 0,
 853     "hlogFileCount" : 0,
 854     "hlogFileSize" : 0,
 855     "storeFileCount" : 0,
 856     "memStoreSize" : 0,
 857     "storeFileSize" : 0,
 858     "regionServerStartTime" : 1418172857470,
 859     "totalRequestCount" : 49,
 860     "readRequestCount" : 0,
 861     "writeRequestCount" : 0,
 862     "checkMutateFailedCount" : 0,
 863     "checkMutatePassedCount" : 0,
 864     "storeFileIndexSize" : 0,
 865     "staticIndexSize" : 0,
 866     "staticBloomSize" : 0,
 867     "mutationsWithoutWALCount" : 0,
 868     "mutationsWithoutWALSize" : 0,
 869     "percentFilesLocal" : 0,
 870     "compactionQueueLength" : 0,
 871     "flushQueueLength" : 0,
 872     "blockCacheFreeSize" : 707690288,
 873     "blockCacheCount" : 0,
....

{code}

> When aborting, dump metrics
> ---------------------------
>
>                 Key: HBASE-12665
>                 URL: https://issues.apache.org/jira/browse/HBASE-12665
>             Project: HBase
>          Issue Type: Bug
>          Components: Operability
>            Reporter: stack
>            Assignee: stack
>             Fix For: 1.0.0, 2.0.0, 0.98.9
>
>         Attachments: 0001-First-cut.patch, 0001-HBASE-12665-When-aborting-dump-metrics.patch,
12665v3.txt, dump.txt
>
>
> We used to dump out all metrics when we were exiting on abort. Was of use debugging why
the abort.  We used to have this.  [~jdcryans] noticed it was dropped by his brother [~eclark]
over in HBASE-6410 "Move RegionServer Metrics to metrics2" To stop the two brothers fighting
I intervened with this patch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message