flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [flink] AHeise commented on a change in pull request #17067: [FLINK-23817][docs] Add metric description to Kafka source and File sink.
Date Tue, 31 Aug 2021 10:13:15 GMT

AHeise commented on a change in pull request #17067:
URL: https://github.com/apache/flink/pull/17067#discussion_r699181895



##########
File path: docs/content/docs/connectors/datastream/kafka.md
##########
@@ -213,26 +213,115 @@ Note that Kafka source does **NOT** rely on committed offsets for fault
toleranc
 is only for exposing the progress of consumer and consuming group for monitoring.
 
 ### Monitoring
-Kafka source exposes metrics in Flink's metric group for monitoring and diagnosing.
-#### Scope of Metric
-All metrics of Kafka source reader are registered under group ```KafkaSourceReader```, which
is a 
-child group of operator metric group. Metrics related to a specific topic partition will
be registered
-in the group ```KafkaSourceReader.topic.<topic_name>.partition.<partition_id>```.
 
-For example, current consuming offset of topic "my-topic" and partition 1 will be reported
in metric: 
-```<some_parent_groups>.operator.KafkaSourceReader.topic.my-topic.partition.1.currentOffset```
,
+Kafka source exposes the following metrics in the respective [scope]({{< ref "docs/ops/metrics"
>}}/#scope).
 
-and number of successful commits will be reported in metric:
-```<some_parent_groups>.operator.KafkaSourceReader.commitsSucceeded``` .
+#### Scope of Metric
 
-#### List of Metrics
+All metrics of Kafka source reader are registered under group ```<job metric group>.<operator
name>.<subtask>.KafkaSourceReader```, which is a 
+child group of operator metric group. Metrics related to a specific topic partition will
be registered
+in the group ```<job metric group>.<operator name>.<subtask>.KafkaSourceReader.topic.<topic_name>.partition.<partition_id>```.
+
+For example, current consuming offset of topic "my-topic" and partition 1 will be reported
in metric 
+```<job metric group>.<operator name>.<subtask>.KafkaSourceReader.topic.my-topic.partition.1.currentOffset```,
and number of successful commits will be reported in metric
+```<job metric group>.<operator name>.<subtask>.KafkaSourceReader.commitsSucceeded```.
+
+<table class="table table-bordered">
+  <thead>
+    <tr>
+      <th class="text-left" style="width: 15%">Scope</th>
+      <th class="text-left" style="width: 18%">Metrics</th>
+      <th class="text-left" style="width: 18%">User Variables</th>
+      <th class="text-left" style="width: 39%">Description</th>
+      <th class="text-left" style="width: 10%">Type</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+        <th rowspan="2">Task</th>
+        <td>numBytesIn</td>
+        <td>n/a</td>
+        <td>The total number of input bytes since the source started. Count towards
the numBytesIn in TaskIOMetricsGroup.</td>
+        <td>Counter</td>
+    </tr>
+    <tr>
+        <td>numBytesInPerSecond</td>
+        <td>n/a</td>
+        <td>The input bytes per second.</td>
+        <td>Meter</td>
+    </tr>
+    <tr>
+        <th rowspan="10">Operator</th>
+        <td>numRecordsIn</td>

Review comment:
       This probably also means that we can complete scratch the default file sink metrics
(recordOut and the meter) right?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message