Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id B704C200D48 for ; Wed, 15 Nov 2017 03:55:27 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id B567C160C07; Wed, 15 Nov 2017 02:55:27 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 06CB6160BF4 for ; Wed, 15 Nov 2017 03:55:26 +0100 (CET) Received: (qmail 32370 invoked by uid 500); 15 Nov 2017 02:55:25 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 32360 invoked by uid 99); 15 Nov 2017 02:55:25 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Nov 2017 02:55:25 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id E8A40180781 for ; Wed, 15 Nov 2017 02:55:24 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.129 X-Spam-Level: ** X-Spam-Status: No, score=2.129 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id UZKQxaGStpoN for ; Wed, 15 Nov 2017 02:55:24 +0000 (UTC) Received: from mail-lf0-f54.google.com (mail-lf0-f54.google.com [209.85.215.54]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id A7D9F5F56D for ; Wed, 15 Nov 2017 02:55:23 +0000 (UTC) Received: by mail-lf0-f54.google.com with SMTP id w21so24535194lfc.6 for ; Tue, 14 Nov 2017 18:55:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=aFhh7dmh4uAmJK9P3Q+NcX4Ne9M+TDtA86wX/PPRH+4=; b=ScgFC9+wPwSdEAGWPMxGY9d2LMLkpkNlEizdpxcfolbXWQmV6APDZgXCEguq9JPPf1 TahqTw3IX0U99yXr8wwXWoHEOkB0kNyUb5fhHxIBe4oHDKZlGcm6YlMs3hfWZZoO2ld4 PJf1nIeOKUSsM8HSkyFr0sZhjLbMbtcm5S9we0uowQrCxjMmwFpoCh50ihMYnjr88/nH fKGWJ5LVOusx3/kAAXoflnvnC2ktkFHPqo1xd4/4Z0qS6QmZvy5d6qbFDdsRTme1tRPi FpZDiYlmU3No8TZ6dTmfE53JzFY44+eD1X79ny8GzVuO4QN5cMQu2a1M32yb9MH51Gho 3PWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=aFhh7dmh4uAmJK9P3Q+NcX4Ne9M+TDtA86wX/PPRH+4=; b=cWPxiP1uBvQx/FCIeR/jL64YRg4ZhBIN7u7sZfxTQ8hEpKARDDWkRpNqvwDZbNg+OK xIM3NxfAW9ne3hSXDPPYhHw/yp9H6fSKt1Mn+8fKocc1lr0I8JO7dXfnnUoEIhBXDUnb bQXZZz7dseU0CHhY706vdzOfxiglQgCMHQ+IOkpoQrUCGKKonIG2plersjrJ5s3xkq8d q1JJpxk8txYMS0zf3v3tHNuogJeE08YWDhGLo8TaU86ykDHSRhA+QZqxW+EouymyKWJ/ wYwpLWYME3fLYnG4mlSTQ2unM5QVfpixGc6n/IyOEAFfFl5KX9R4hnRHRJPhNDHvrAKA bDqA== X-Gm-Message-State: AJaThX5PB1bHVwBppapbw0HC9gO0A9dWccwYoA0Xmw7T9mxE0A9kW2Qk KL7/UEAC8ysTV/llKQ52POTIT9+1Mw4+i329nnh5vg== X-Google-Smtp-Source: AGs4zMY8gHzuf84jM9VBoQSbrM+liZFO6Gvv+cPGPmoaIT7uEKcHYMaT7cpdZx/QBq/ID93ZiJ9oslBf4hYFd5015JA= X-Received: by 10.46.20.87 with SMTP id 23mr1409978lju.125.1510714521622; Tue, 14 Nov 2017 18:55:21 -0800 (PST) MIME-Version: 1.0 Received: by 10.179.7.129 with HTTP; Tue, 14 Nov 2017 18:55:01 -0800 (PST) From: Nomchin Banga Date: Tue, 14 Nov 2017 21:55:01 -0500 Message-ID: Subject: Capturing statistics for more than 5 minutes To: user@flink.apache.org Content-Type: multipart/alternative; boundary="f403045fb2d86b1fbe055dfca232" archived-at: Wed, 15 Nov 2017 02:55:27 -0000 --f403045fb2d86b1fbe055dfca232 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi We are a group of graduate students from Purdue University who are doing an experimental study to compare different data ingestion engines. For this purpose, we are trying to collect statistics of running jobs over a few days. However, Flink=E2=80=99s UI captures the statistics for the las= t 5 minutes. We tried to hit the Rest API but that does not contain the data of statistics other than bytes read/written. It would be very helpful if you could guide us regarding a way in which we can capture the entire history of metrics. Looking forward to hearing from you Regards Nomchin Banga --f403045fb2d86b1fbe055dfca232 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi

We are a group of graduate students from Purdue University who are doin= g an experimental study to compare different data ingestion engines.=

For this purpose, we are trying to collect statistics= of running jobs over a few days. However, Flink=E2=80=99s UI captures the = statistics for the last 5 minutes.
We tried to hit the Rest API but that does no= t contain the data of statistics other than bytes read/written.

It would be very helpful if you could guide us regarding a w= ay in which we can capture the entire history of metrics.

Looking forward to hearing from you

Re= gards
Nomchin Banga
--f403045fb2d86b1fbe055dfca232--