Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 94BEB200D11 for ; Mon, 18 Sep 2017 04:40:02 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 933CD1609D9; Mon, 18 Sep 2017 02:40:02 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 8AE6E1609D8 for ; Mon, 18 Sep 2017 04:40:01 +0200 (CEST) Received: (qmail 87620 invoked by uid 500); 18 Sep 2017 02:40:00 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 87610 invoked by uid 99); 18 Sep 2017 02:40:00 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 18 Sep 2017 02:40:00 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id E964D1A12AE for ; Mon, 18 Sep 2017 02:39:59 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.679 X-Spam-Level: * X-Spam-Status: No, score=1.679 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id lbCL0FXg7DwA for ; Mon, 18 Sep 2017 02:39:58 +0000 (UTC) Received: from mail-qt0-f182.google.com (mail-qt0-f182.google.com [209.85.216.182]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 2A6485FE64 for ; Mon, 18 Sep 2017 02:39:58 +0000 (UTC) Received: by mail-qt0-f182.google.com with SMTP id i13so6292986qtc.11 for ; Sun, 17 Sep 2017 19:39:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=QAw3vO3YOCc2kdGMKQ24r4t6CyPrUSpMyG1/kt+D0Qk=; b=XSEAdS7UdYgUxrqrnuQfyiW5vS+kmFgK07mWAlWiU9Jf7lMAPIsjZIAgCGiWO6MLmF iSTdOsMmP1Ld2SrA8Qtqm9y4en1mgqUtwdG/wZEa29XEbsgjMPPjwBp0fCeK6rcOaSSK FznOOYNeM9qp+2oAoOiZMEQEIUOcAkDqafycPTJ+JgiWIPMwc6uohsCvnwwksBjc0PLn DUtWh+PjU/P5iet2UqCwiYFj932y2oOX08FrjH/uJwVYq34mhq8h0YeBAwa+IgHiBaXO SVdfaFCvgKXE7plFZyVBRB6ew9U2cpg1Vbs7Oq1YrI0QtGyrNzET5b76mu7TC3Kma/lH /2Fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=QAw3vO3YOCc2kdGMKQ24r4t6CyPrUSpMyG1/kt+D0Qk=; b=icxzxGGk0xOfwnbDjgsCS27XhvA4Y1zF3x0/1R7lPyixRHlahtO5F09yvVShvnzDo9 3BbN7v5o2E5m9Cd9D6UN3eAB/Z22VflRUuQtoME4qPYePF1RABCKagowCg2groHub6NI EAk26yFRKs+I84ozqmOv+oimLNgPruIc261b5vuNd3P4su1aLpR8rjXzEoehMPHvqG97 aGO5bp0HS/qs+2oajZTibiiJh5CROn28S7dOoUwQQeEtJ+rkzPVitJZH8n7FSRDtGj3D 3utCDfDJL1q2eWpfDZxoehS8DrQMaOdOaZtmFQzknIOnFVYFTpQmZOYc5DbpnCzTDtx3 9U1Q== X-Gm-Message-State: AHPjjUhSBfbj4d0BEkLGtMVkxmK53b0WyBJsyY85GEsNlzf5ZnF7hE13 lFD7EMtedsKMzZRaz0C7XAz4cOLcv5/lW2/sEcs= X-Google-Smtp-Source: AOwi7QD2Szf2KZ7L++J4wN6MFaCUdPH/upBihsh8PprIV2co6xaDggzHgFRez/WOD/8vp6Wu0wayiwf71Izs+bfEEJA= X-Received: by 10.200.23.153 with SMTP id o25mr48019238qtj.206.1505702397163; Sun, 17 Sep 2017 19:39:57 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.31.225 with HTTP; Sun, 17 Sep 2017 19:39:56 -0700 (PDT) In-Reply-To: References: From: Michael Fong Date: Mon, 18 Sep 2017 10:39:56 +0800 Message-ID: Subject: Re: need instruction on how the Flink metric works To: Jiewen Shao Cc: user@flink.apache.org Content-Type: multipart/alternative; boundary="f403045d62ba853b2505596da8a8" archived-at: Mon, 18 Sep 2017 02:40:02 -0000 --f403045d62ba853b2505596da8a8 Content-Type: text/plain; charset="UTF-8" Hi, You may enable metrics reporter to see the output of your metrics; counter in your example. There is a brief documentation regarding to metrics and reporter setup at link . The easiest approach, in my opinion, is to set up a JMX reporter so that you may see your metrics via JConsole. Hope this helps. Regrads, On Mon, Sep 18, 2017 at 10:27 AM, Jiewen Shao wrote: > I'm new to flink and I have read https://ci.apache.org/ > projects/flink/flink-docs-release-1.3/monitoring/metrics.html, I am still > unclear where do I read the metrics I added. > > for example, > > public static void main(String[] args) throws Exception { > > > StreamExecutionEnvironment env = StreamExecutionEnvironment. > getExecutionEnvironment(); > > env.setParallelism(2); > > > > List wordList = Arrays.asList("Hive", "Presto", "Impala", > "Parquet","ORC","Hadoop", "Flink", "Spark", "Storm", "Tez", "Flink"); > > > DataStreamSource source = env.fromCollection(wordList); > > DataStream> dataStream = env > .fromCollection(wordList).map(new WordLengthCounter()); > > dataStream.print(); > > env.execute(); > > } > > > and > > > public class WordLengthCounter extends RichMapFunction Tuple2> { > > > private static final long serialVersionUID = 1L; > > private Counter counter; > > > @Override > > public void open(Configuration config) { > > this.counter = getRuntimeContext() > > .getMetricGroup() > > .counter("myCounter"); > > } > > > @Override > > public Tuple2 map(String value) throws Exception { > > this.counter.inc(); > > return new Tuple2(value, value.length()); > > } > > } > > > Now, where do I see the counter? Sorry for the naive question > > can anyone point me to any good end-to-end "hello world" example for flink > metrics. > --f403045d62ba853b2505596da8a8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,=C2=A0

You may enable metrics report= er to see the output of your metrics; counter in your example.=C2=A0
<= div>
There is a brief documentation regarding to metrics and = reporter setup at link. The easiest approach, in m= y opinion, is to set up a JMX reporter so that you may see your metrics via= JConsole.=C2=A0

Hope this helps.

Regrads,


=
On Mon, Sep 18, 2017 at 10:27 AM, Jiewen Shao <jiewenshao@gmail.com> wrote:
I'm new to flink and I have read=C2=A0https://ci.apache.org/projects/flink/flink= -docs-release-1.3/monitoring/metrics.html, I am still unclear= where do I read the metrics I added.=C2=A0

for example,= =C2=A0

public st= atic void main(String[]= args) throws Exception {


=C2=A0 =C2=A0 =C2=A0 =C2=A0 StreamExecutionEnvironment <= span style=3D"color:rgb(126,80,79)">env =3D StreamExecutionEnvironme= nt.getExecutionEnvironment();

=C2=A0 =C2= =A0 =C2=A0 =C2=A0 env.setParall= elism(2);

=C2=A0 =C2=A0 =C2=A0= =C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 List<String> wordList =3D Arrays.asList("Hive", "Presto", &q= uot;Impala", "Parquet&quo= t;,"ORC","Hadoop"= , "Flink", "S= park", "Storm", "Tez", "Flink"); =C2=A0 =C2=A0 =C2=A0 =C2=A0

= =C2=A0 =C2=A0 =C2=A0 =C2=A0 DataStreamSource<String> source =3D env.fromCollection(wordList);

=C2=A0 = =C2=A0 =C2=A0 =C2=A0 DataStream<Tuple2<String, Integer>> dataStream =3D env.fromCollection(wordList).map(new WordLengthCounter());

=C2=A0 =C2=A0 =C2=A0 = =C2=A0 dataStream.print();

<= p style=3D"margin:0px;font-size:13px;line-height:normal;font-family:Monaco"= > =C2=A0 =C2=A0 =C2=A0 =C2=A0 env.execute();

=C2=A0 =C2=A0 }


and=C2=A0

=

public class WordLengthCounter extends RichMapFunction<String, Tuple2<St= ring, Integer>> {


private static final long serialVersion= UID =3D 1L;

private Counter counter;


=

@Override

=C2=A0 public void open(Configuration con= fig) {

=C2=A0 =C2=A0 this.counter<= /span> =3D getRuntimeContext()

=C2=A0 =C2=A0 = =C2=A0 .getMetricGroup()

=C2=A0 =C2=A0 =C2=A0 .= counter("myCounter");=

=C2=A0 }


@Override

public Tuple2= <String, Integer> map(String val= ue) throws Exception {<= /p>

this.counter.inc();

return new Tuple2<String, Integer>(<= span style=3D"color:rgb(126,80,79)">value, value.length());

}

}



Now, where do I see the counter? Sorr= y for the naive question


can anyone point= me to any good end-to-end "hello world" example for flink metric= s.

--f403045d62ba853b2505596da8a8--