flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: ResultPartitionMetrics
Date Mon, 23 Oct 2017 08:43:29 GMT
The metrics registered in initializeBufferMetrics aggregate across all 
InputGates, whereas the metrics registered in the InputGateMetrics are 
separate for each InputGate.

As an example, let's say a task has 2 input gates, with each having 2 
input buffers queued:

// IOMetricGroup#initializeBufferMetrics
task.buffers.inputQueueLength= 4

// detailed metrics
task.input.0.totalQueueLen = 2
task.input.1.totalQueueLen = 2

On 23.10.2017 09:31, Timo Walther wrote:
> Hi Aitozi,
>
> I will loop in people that are more familar with the network stack and 
> metrics. Maybe this is a bug?
>
> Regards,
> Timo
>
>
>
> Am 10/22/17 um 4:36 PM schrieb aitozi:
>> Hi,
>>
>> i see in version 1.3, it add the ResultPartitionMetrics with
>> issue:https://issues.apache.org/jira/browse/FLINK-5090
>>
>> but i am doubt what is the difference between totalQueueLen and
>> inputQueueLength in
>> https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/metrics.html#network

>>
>>
>> i read the code where register the both metric in Task.java:
>>
>> this.metrics.getIOMetricGroup().initializeBufferMetrics(this);
>>
>> // register detailed network metrics, if configured
>> if
>> (taskManagerConfig.getConfiguration().getBoolean(TaskManagerOptions.NETWORK_DETAILED_METRICS))

>>
>> {
>>                 // similar to MetricUtils.instantiateNetworkMetrics() 
>> but inside this
>> IOMetricGroup
>>                 MetricGroup networkGroup =
>> this.metrics.getIOMetricGroup().addGroup("Network");
>>                 MetricGroup outputGroup = 
>> networkGroup.addGroup("Output");
>>                 MetricGroup inputGroup = networkGroup.addGroup("Input");
>>
>>                 // output metrics
>>                 for (int i = 0; i < producedPartitions.length; i++) {
>> ResultPartitionMetrics.registerQueueLengthMetrics(
>>                         outputGroup.addGroup(i), producedPartitions[i]);
>>                 }
>>
>>                 for (int i = 0; i < inputGates.length; i++) {
>>                     InputGateMetrics.registerQueueLengthMetrics(
>>                         inputGroup.addGroup(i), inputGates[i]);
>>                 }
>>             }
>>
>> i think the first :initializeBufferMetrics#InputBuffersGauge will get 
>> all
>> the buffers in AllInputGate of the Task and i think the method in
>> InputGateMetric
>> "group.gauge("totalQueueLen",metrics.getTotalQueueLenGauge());" does the
>> same thing , if i understand wrong , please tell me,.
>>
>> thanks,
>> Aitozi
>>
>>
>>
>> -- 
>> Sent from: 
>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>
>
>


Mime
View raw message