hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jeff saremi <jeffsar...@hotmail.com>
Subject Re: Certain metric groups are missing when using a custom sink in place of FileSink
Date Thu, 16 Feb 2017 16:42:11 GMT
Just to answer my own concern, this seemed to be a result of lack of metrics for some of those
Contexts.

Nothing to be alarmed about.


________________________________
From: jeff saremi <jeffsaremi@hotmail.com>
Sent: Wednesday, February 15, 2017 10:21:51 AM
To: user@hadoop.apache.org
Subject: Re: Certain metric groups are missing when using a custom sink in place of FileSink


Here is some more debug information:

2017-02-15 10:19:08,266 DEBUG [Timer for 'HBase' metrics system] impl.MetricsSinkAdapter:
enqueue, logicalTime=80
2017-02-15 10:19:18,267 DEBUG [Timer for 'HBase' metrics system] impl.MetricsSystemImpl: Snapshotted
source RegionServer,sub=WAL
2017-02-15 10:19:18,282 DEBUG [Timer for 'HBase' metrics system] impl.MetricsSystemImpl: Snapshotted
source RegionServer,sub=IPC
2017-02-15 10:19:18,282 DEBUG [Timer for 'HBase' metrics system] impl.MetricsSystemImpl: Snapshotted
source UgiMetrics
2017-02-15 10:19:18,282 DEBUG [Timer for 'HBase' metrics system] impl.MetricsSystemImpl: Snapshotted
source RegionServer,sub=Replication
2017-02-15 10:19:18,282 DEBUG [Timer for 'HBase' metrics system] impl.MetricsSystemImpl: Snapshotted
source JvmMetrics
2017-02-15 10:19:18,282 DEBUG [Timer for 'HBase' metrics system] impl.MetricsSystemImpl: Snapshotted
source RegionServer,sub=Server
2017-02-15 10:19:18,282 DEBUG [Timer for 'HBase' metrics system] impl.MetricsSystemImpl: Snapshotted
source RegionServer,sub=Regions
2017-02-15 10:19:18,282 DEBUG [Timer for 'HBase' metrics system] impl.MetricsSystemImpl: Snapshotted
source MetricsSystem,sub=Stats


This shows that the metrics subsystem has subscribed to all "subs".
However only 2 of those make it:

2017-02-15 10:17:58,262 DEBUG [mdmsink] impl.MetricsSinkAdapter: Pushing record RegionServer,sub=WAL.regionserver.WAL
to mdmsink
2017-02-15 10:17:58,293 TRACE [mdmsink] metrics.MdmServiceClient: {"timestamp":1487182678262,"namespace":"regionserver.WAL","dimensions":...
2017-02-15 10:17:58,356 ERROR [mdmsink] metrics.MdmServiceClient: Getting bad status code
from the service: 204
2017-02-15 10:17:58,356 DEBUG [mdmsink] impl.MetricsSinkAdapter: Pushing record RegionServer,sub=IPC.regionserver.RegionServer
to mdmsink
2017-02-15 10:17:58,356 TRACE [mdmsink] metrics.MdmServiceClient: {"timestamp":1487182678262,"namespace":"regionserver.RegionServer","dimensions":...
2017-02-15 10:17:58,481 ERROR [mdmsink] metrics.MdmServiceClient: Getting bad status code
from the service: 204
2017-02-15 10:17:58,481 DEBUG [mdmsink] impl.MetricsSinkAdapter: Pushing record UgiMetrics.ugi.UgiMetrics
to mdmsink
2017-02-15 10:17:58,481 TRACE [mdmsink] metrics.MdmServiceClient: {"timestamp":1487182678262,"namespace":"ugi.UgiMetrics","dimensions":[{"key":"Context","value":"ugi"},...


________________________________
From: jeff saremi <jeffsaremi@hotmail.com>
Sent: Wednesday, February 15, 2017 8:33:25 AM
To: user@hadoop.apache.org
Subject: Fw: Certain metric groups are missing when using a custom sink in place of FileSink


forwarding here since this seems to be a generic metric-related issue and not necessarily
HBase.


________________________________
From: jeff saremi <jeffsaremi@hotmail.com>
Sent: Tuesday, February 14, 2017 9:29 PM
To: user@hbase.apache.org
Subject: Certain metric groups are missing when using a custom sink in place of FileSink


When we use a FileSink to log metrics from HBase we can see more names than when we use a
custom metric sink. Is there something undocumented that we're missing?
For instance when using FileSink, we can see WAL, RegionServer, Replication,Server , and Regions
in the log.
However if we use a custom sink (which happens to be writing whatever it gets to the log as
well) we can only see WAL and RegionServer.

These are the config lines we're using:

hbase.sink.mdmsink.class=mycomp.metrics.MdmSink
hbase.sink.mdmsink.server=myserver:8880
*.period=10


Mime
View raw message