Return-Path: <issues-return-70072-archive-asf-public=cust-asf.ponee.io@flink.apache.org>
X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io
Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io
Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183])
	by cust-asf2.ponee.io (Postfix) with ESMTP id B84E0200B8D
	for <archive-asf-public-internal@cust-asf2.ponee.io>; Thu,  8 Sep 2016 14:33:22 +0200 (CEST)
Received: by cust-asf.ponee.io (Postfix)
	id B740A160AA5; Thu,  8 Sep 2016 12:33:22 +0000 (UTC)
Delivered-To: archive-asf-public@cust-asf.ponee.io
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by cust-asf.ponee.io (Postfix) with SMTP id D56D0160AD0
	for <archive-asf-public@cust-asf.ponee.io>; Thu,  8 Sep 2016 14:33:21 +0200 (CEST)
Received: (qmail 18569 invoked by uid 500); 8 Sep 2016 12:33:21 -0000
Mailing-List: contact issues-help@flink.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:issues-help@flink.apache.org>
List-Unsubscribe: <mailto:issues-unsubscribe@flink.apache.org>
List-Post: <mailto:issues@flink.apache.org>
List-Id: <issues.flink.apache.org>
Reply-To: dev@flink.apache.org
Delivered-To: mailing list issues@flink.apache.org
Received: (qmail 18553 invoked by uid 99); 8 Sep 2016 12:33:21 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 08 Sep 2016 12:33:21 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id BC43A2C1B81
	for <issues@flink.apache.org>; Thu,  8 Sep 2016 12:33:20 +0000 (UTC)
Date: Thu, 8 Sep 2016 12:33:20 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@flink.apache.org
Message-ID: <JIRA.12952773.1458728282000.519450.1473338000768@Atlassian.JIRA>
In-Reply-To: <JIRA.12952773.1458728282000@Atlassian.JIRA>
References: <JIRA.12952773.1458728282000@Atlassian.JIRA> <JIRA.12952773.1458728282538@arcas>
Subject: [jira] [Commented] (FLINK-3660) Measure latency of elements and
 expose it through web interface
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
archived-at: Thu, 08 Sep 2016 12:33:22 -0000


    [ https://issues.apache.org/jira/browse/FLINK-3660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15473745#comment-15473745 ] 

ASF GitHub Bot commented on FLINK-3660:
---------------------------------------

Github user zentol commented on a diff in the pull request:

    https://github.com/apache/flink/pull/2386#discussion_r77996152
  
    --- Diff: docs/monitoring/metrics.md ---
    @@ -475,52 +474,76 @@ Flink exposes the following system metrics:
           <td></td>
         </tr>
         <tr>
    -      <tr>
    -        <th rowspan="7"><strong>Task</strong></t>
    -        <td>currentLowWatermark</td>
    -        <td>The lowest watermark a task has received.</td>
    -      </tr>
    -      <tr>
    -        <td>lastCheckpointDuration</td>
    -        <td>The time it took to complete the last checkpoint.</td>
    -      </tr>
    -      <tr>
    -        <td>lastCheckpointSize</td>
    -        <td>The total size of the last checkpoint.</td>
    -      </tr>
    -      <tr>
    -        <td>restartingTime</td>
    -        <td>The time it took to restart the job.</td>
    -      </tr>
    -      <tr>
    -        <td>numBytesInLocal</td>
    -        <td>The total number of bytes this task has read from a local source.</td>
    -      </tr>
    -      <tr>
    -        <td>numBytesInRemote</td>
    -        <td>The total number of bytes this task has read from a remote source.</td>
    -      </tr>
    -      <tr>
    -        <td>numBytesOut</td>
    -        <td>The total number of bytes this task has emitted.</td>
    -      </tr>
    -    </tr>
    -    <tr>
    -      <tr>
    -        <th rowspan="3"><strong>Operator</strong></th>
    -        <td>numRecordsIn</td>
    -        <td>The total number of records this operator has received.</td>
    -      </tr>
    -      <tr>
    -        <td>numRecordsOut</td>
    -        <td>The total number of records this operator has emitted.</td>
    -      </tr>
    -      <tr>
    -        <td>numSplitsProcessed</td>
    -        <td>The total number of InputSplits this data source has processed.</td>
    -      </tr>
    +      <th rowspan="7"><strong>Task</strong></th>
    +      <td>currentLowWatermark</td>
    +      <td>The lowest watermark a task has received.</td>
    +    </tr>
    +    <tr>
    +      <td>lastCheckpointDuration</td>
    +      <td>The time it took to complete the last checkpoint.</td>
    +    </tr>
    +    <tr>
    +      <td>lastCheckpointSize</td>
    +      <td>The total size of the last checkpoint.</td>
    +    </tr>
    +    <tr>
    +      <td>restartingTime</td>
    +      <td>The time it took to restart the job.</td>
    +    </tr>
    +    <tr>
    +      <td>numBytesInLocal</td>
    +      <td>The total number of bytes this task has read from a local source.</td>
    +    </tr>
    +    <tr>
    +      <td>numBytesInRemote</td>
    +      <td>The total number of bytes this task has read from a remote source.</td>
    +    </tr>
    +    <tr>
    +      <td>numBytesOut</td>
    +      <td>The total number of bytes this task has emitted.</td>
    +    </tr>
    +    <tr>
    +      <th rowspan="4"><strong>Operator</strong></th>
    +      <td>numRecordsIn</td>
    +      <td>The total number of records this operator has received.</td>
    +    </tr>
    +    <tr>
    +      <td>numRecordsOut</td>
    +      <td>The total number of records this operator has emitted.</td>
    +    </tr>
    +    <tr>
    +      <td>numSplitsProcessed</td>
    +      <td>The total number of InputSplits this data source has processed (if the operator is a data source).</td>
    +    </tr>
    +    <tr>
    +      <td>latency</td>
    +      <td>A latency gauge reporting the latency distribution from the different sources.</td>
         </tr>
       </tbody>
     </table>
     
    +
    +### Latency tracking
    +
    +Flink allows to track the latency of records traveling through the system. To enable the latency tracking
    +a `latencyTrackingInterval` (in milliseconds) has to be set to a positive value in the `ExecutionConfig`.
    +
    +At the `latencyTrackingInterval`, the sources will periodically emit a special record, called a `LatencyMaker`.
    --- End diff --
    
    LatencyMaker -> LatencyMarker


> Measure latency of elements and expose it through web interface
> ---------------------------------------------------------------
>
>                 Key: FLINK-3660
>                 URL: https://issues.apache.org/jira/browse/FLINK-3660
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Streaming
>            Reporter: Robert Metzger
>            Assignee: Robert Metzger
>             Fix For: pre-apache
>
>
> It would be nice to expose the end-to-end latency of a streaming job in the webinterface.
> To achieve this, my initial thought was to attach an ingestion-time timestamp at the sources to each record.
> However, this introduces overhead for a monitoring feature users might not even use (8 bytes for each element + System.currentTimeMilis() on each element).
> Therefore, I suggest to implement this feature by periodically sending special events, similar to watermarks through the topology. 
> Those {{LatencyMarks}} are emitted at a configurable interval at the sources and forwarded by the tasks. The sinks will compare the timestamp of the latency marks with their current system time to determine the latency.
> The latency marks will not add to the latency of a job, but the marks will be delayed similarly than regular records, so their latency will approximate the record latency.
> Above suggestion expects the clocks on all taskmanagers to be in sync. Otherwise, the measured latencies would also include the offsets between the taskmanager's clocks.
> In a second step, we can try to mitigate the issue by using the JobManager as a central timing service. The TaskManagers will periodically query the JM for the current time in order to determine the offset with their clock.
> This offset would still include the network latency between TM and JM but it would still lead to reasonably good estimations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)