Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 47926200D2D for ; Fri, 13 Oct 2017 04:31:21 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 45EC4160BE4; Fri, 13 Oct 2017 02:31:21 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 644291609E8 for ; Fri, 13 Oct 2017 04:31:20 +0200 (CEST) Received: (qmail 53954 invoked by uid 500); 13 Oct 2017 02:31:19 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 53944 invoked by uid 99); 13 Oct 2017 02:31:19 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 13 Oct 2017 02:31:19 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 5F421C744D for ; Fri, 13 Oct 2017 02:31:18 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.379 X-Spam-Level: ** X-Spam-Status: No, score=2.379 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id 3XYeTFTR4dM3 for ; Fri, 13 Oct 2017 02:31:17 +0000 (UTC) Received: from mail-oi0-f50.google.com (mail-oi0-f50.google.com [209.85.218.50]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 82EAD5F39D for ; Fri, 13 Oct 2017 02:31:16 +0000 (UTC) Received: by mail-oi0-f50.google.com with SMTP id j126so12186465oib.8 for ; Thu, 12 Oct 2017 19:31:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=T+55ivTE8b/R+34BwCKJrm5c/xuVJvCaVEup040hrAk=; b=lLpM1v+YwhI//PaER3JErKxDzo0LQvCAI65XBzvVQvbGsOC8Z3L2BXslGEzkeU6/rG wq2jHxhyRttUBlZNi92ODj/Q5Ii6Kg14LAt5R8cZHOPL2JezyHRp4KTPjGVn/leIyRPm //8VVULJaqhJYLv/dAz/QUj3l3x4hJAa7/yZihoVF1KFUfDkkSRYlV0tTF9A9AlFQDIH OnJKgKFzaE0BuPD4GiXoC99X3dx0M6iSxuAY0uQPDaLa4gPFRoMSVHMmBYT0LcHgG6++ iwAfZpTbr13eHxGEMwjV5W2ZhK5SAiEKt5w+FTxsXsR6HNNhwhT8OQ8Rd63jw24gApQN tU/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=T+55ivTE8b/R+34BwCKJrm5c/xuVJvCaVEup040hrAk=; b=bifWXOI7dRwfEH7xaR2JQxxpj7mJj6VNcdyloCwOf2PVRYXkz0jHNgDYQmJOeaIOU0 aIGTF+UxMEFN+RULTm8cMxSjWERJWpOVFhW2zjIbNkA8bN+MY6Feoh1XjAhLTlfk2KcS FCMQB6aAl1gIONeb3JxEBZdyzr+PB+CZA5Dg0RP0pPixr8fdxVASYTIgRJfVvXzFfRPG 6R3BPPSnX3ALZP/0sP5xG4iJMy4cprFo+VPMmajLwysZovSsnxUC6EdHADYUlnYRcvdl F+0DIhPITwrAGMPRKha++IFeeJ/uS5sXCZI4OAKi5sY12eS005bRFtWL9xmfEIFzPdwH rtSw== X-Gm-Message-State: AMCzsaV+bU2lfSAAqjms50IAvb0Jzd654mSpo4Pj7/eSfCxLr5SOYMsZ CZq1SP2ktI8bcuWugensvxCpi0eRLFz1MrOvrxposQ== X-Google-Smtp-Source: AOwi7QCtWFUh8nWjLbYIUGxov8EQ7poBqwNw6q7yQ5G+vMZKKPyEiXjY+g4dSpBmyoG96pu9Wn5UAvHh+mMXJ0x9iN8= X-Received: by 10.202.8.79 with SMTP id 76mr2106016oii.199.1507861875059; Thu, 12 Oct 2017 19:31:15 -0700 (PDT) MIME-Version: 1.0 Received: by 10.74.181.134 with HTTP; Thu, 12 Oct 2017 19:31:14 -0700 (PDT) In-Reply-To: References: <464fbce2-c872-6f81-8531-e001d8c793e2@apache.org> From: Colin Williams Date: Thu, 12 Oct 2017 19:31:14 -0700 Message-ID: Subject: Re: Writing an Integration test for flink-metrics To: Chesnay Schepler Cc: user Content-Type: multipart/alternative; boundary="94eb2c1303706eeea9055b64738b" archived-at: Fri, 13 Oct 2017 02:31:21 -0000 --94eb2c1303706eeea9055b64738b Content-Type: text/plain; charset="UTF-8" Team wants an integration test, I'm not sure what unit test you had in mind. Actually feel that I've been trying to avoid the reporter method but that would be more end to end. The documentation for metrics and Scala are missing with the exception of Gauge: https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/metrics.html . Should I file a issue against that? Then it leaves you guessing a little bit how to implement Counters. One approach tried was using objects object PointFilter extends RichMapFunction[... @transient lazy val someCounter = getRuntimeContext.getMetricGroup.counter(...) This allowed access to the counter before and after execution . However between the unit tests the Counter kept its value also and that's a no for the test. Think that might be an issue with ScalaTest. I've tried to get at the counter from some other directions like trying to find a way to inject a reporter to get it's state. But don't see a way to do it. So probably the best thing to do is fire up something to collect the metrics from the reporter. On Thu, Oct 12, 2017 at 5:29 AM, Chesnay Schepler wrote: > Well damn, i should've read the second part of the initial mail. > > I'm wondering though, could you not unit-test this behavior? > > > On 12.10.2017 14:25, Chesnay Schepler wrote: > >> You could also write a custom reporter that opens a socket or similar for >> communication purposes. >> >> You can then either query it for the metrics, or even just trigger the >> verification in the reporter, >> and fail with an error if the reporter returns an error. >> >> On 12.10.2017 14:02, Piotr Nowojski wrote: >> >>> Hi, >>> >>> Doing as you proposed using JMXReporter (or custom reporter) should >>> work. I think there is no easier way to do this at the moment. >>> >>> Piotrek >>> >>> On 12 Oct 2017, at 04:58, Colin Williams >>> com> wrote: >>>> >>>> I have a RichMapFunction and I'd like to ensure Meter fields are >>>> properly incremented. I've been trying to think of the best way to do this. >>>> Currently I think that I'd need to either implement my own reporter (or use >>>> JMX) and write to a socket, create a listener and wait for the reporter to >>>> send the message. >>>> >>>> Is this a good approach for writing the test, or should I be >>>> considering something else? >>>> >>> >>> >> >> > --94eb2c1303706eeea9055b64738b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Team wants an integration test, I'= m not sure what unit test you had in mind. Actually feel that I've been= trying to avoid the reporter method but that would be more end to end.
=
The documentation for metrics and Scala are missing with the exce= ption of Gauge: https://ci.apache.org/projects/flink/f= link-docs-release-1.3/monitoring/metrics.html . Should I file a issue a= gainst that?

Then it leaves you guessing a little bit how to i= mplement Counters. One approach tried was using objects

object PointFilter extends RichMapFunction[...
  @transient lazy val someCounter =3D ge=
tRuntimeContext.getMetricGroup.counter(...)

This allowed= access to the counter before and after execution . However between the uni= t tests the Counter kept its value also and that's a no for the test. T= hink that might be an issue with ScalaTest.

I= 've tried to get at the counter from some other directions like trying = to find a way to inject a reporter to get it's state. But don't see= a way to do it. So probably the best thing to do is fire up something to c= ollect the metrics from the reporter.

On Thu, Oct 12, 2017 at 5:29 AM, Chesna= y Schepler <chesnay@apache.org> wrote:
Well damn, i should've read the second part of the initia= l mail.

I'm wondering though, could you not unit-test this behavior?


On 12.10.2017 14:25, Chesnay Schepler wrote:
You could also write a custom reporter that opens a socket or similar for c= ommunication purposes.

You can then either query it for the metrics, or even just trigger the veri= fication in the reporter,
and fail with an error if the reporter returns an error.

On 12.10.2017 14:02, Piotr Nowojski wrote:
Hi,

Doing as you proposed using JMXReporter (or custom reporter) should work. I= think there is no easier way to do this at the moment.

Piotrek

On 12 Oct 2017, at 04:58, Colin Williams <colin.williams.seattle@gmail.c= om> wrote:

I have a RichMapFunction and I'd like to ensure Meter fields are proper= ly incremented. I've been trying to think of the best way to do this. C= urrently I think that I'd need to either implement my own reporter (or = use JMX) and write to a socket, create a listener and wait for the reporter= to send the message.

Is this a good approach for writing the test, or should I be considering so= mething else?





--94eb2c1303706eeea9055b64738b--