Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id DCF45200B6B for ; Thu, 25 Aug 2016 20:41:41 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id DB4A6160AA5; Thu, 25 Aug 2016 18:41:41 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id AEB73160AA4 for ; Thu, 25 Aug 2016 20:41:40 +0200 (CEST) Received: (qmail 80992 invoked by uid 500); 25 Aug 2016 18:41:39 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 80983 invoked by uid 99); 25 Aug 2016 18:41:39 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Aug 2016 18:41:39 +0000 Received: from mail-wm0-f51.google.com (mail-wm0-f51.google.com [74.125.82.51]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 2500F1A003E for ; Thu, 25 Aug 2016 18:41:39 +0000 (UTC) Received: by mail-wm0-f51.google.com with SMTP id q128so250651932wma.1 for ; Thu, 25 Aug 2016 11:41:38 -0700 (PDT) X-Gm-Message-State: AE9vXwPd8gmlQ9m6UbMZY8H05++H5P1vPJEUNnXDlXRm4vaU0SmfR1LEaSl2p8xXiyGHXIlOhv2uE9tPRiY80w== X-Received: by 10.28.125.80 with SMTP id y77mr9588595wmc.25.1472150497691; Thu, 25 Aug 2016 11:41:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.75.36 with HTTP; Thu, 25 Aug 2016 11:41:17 -0700 (PDT) In-Reply-To: References: <1472050550167-8664.post@n4.nabble.com> From: Robert Metzger Date: Thu, 25 Aug 2016 20:41:17 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Dealing with Multiple sinks in Flink To: "user@flink.apache.org" Content-Type: multipart/alternative; boundary=001a1141921878e658053ae9bff5 archived-at: Thu, 25 Aug 2016 18:41:42 -0000 --001a1141921878e658053ae9bff5 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I would first try to understand the metrics system locally, for example using the VisualVM tool, that also allows you to access JMX-exported metrics. Once you've seen how it works, you can look into remote JMX access. There's a page in the Flink documentation about the metrics. On Thu, Aug 25, 2016 at 8:36 PM, vinay patil wrote: > Thanks Robert will try that, can you provide more details on how to > integrate that. > > I had sysouts to check if the watermarks are getting generated, I am > getting the values, but as I said the window is not getting triggered for > parallelism greater than 1, I have tried using AscendingTimeStampExtracto= r > but facing the same issue. > > > > Regards, > Vinay Patil > > On Thu, Aug 25, 2016 at 1:32 PM, rmetzger0 [via Apache Flink User Mailing > List archive.] <[hidden email] > > wrote: > >> Flink 1.1.1 has a metric for exposing the low watermark of each operator= . >> Maybe you can access a TaskManager via JMX to see the value of the WM. >> >> Watermarks are sometimes a bit tricky. >> >> On Thu, Aug 25, 2016 at 4:29 PM, vinay patil <[hidden email] >> > wrote: >> >>> Hi Max, >>> >>> Here is the code for Timestamp assigner and watermark generation. >>> PFA >>> >>> Regards, >>> Vinay Patil >>> >>> On Thu, Aug 25, 2016 at 7:39 AM, Maximilian Michels [via Apache Flink >>> User Mailing List archive.] <[hidden email] >>> > wrote: >>> >>>> I'm assuming there is something wrong with your Watermark/Timestamp >>>> assigner. Could you share some of the code? >>>> >>>> On Wed, Aug 24, 2016 at 9:54 PM, vinay patil <[hidden email] >>>> > wrote: >>>> >>>> > Hi, >>>> > >>>> > Just an update, the window is not getting triggered when I change th= e >>>> > parallelism to more than 1. >>>> > >>>> > Can you please explain why this is happening ? >>>> > >>>> > Regards, >>>> > Vinay Patil >>>> > >>>> > On Wed, Aug 24, 2016 at 9:55 AM, vinay patil [via Apache Flink User >>>> Mailing >>>> > List archive.] <[hidden email]> wrote: >>>> >>>> >> >>>> >> Hi Max, >>>> >> >>>> >> I tried writing to local file as well, its giving me the same issue= , >>>> I >>>> >> have attached the logs and dummy pipeline >>>> code.logs.txtdummy_pipeline.txt >>>> >> >>>> >> ________________________________ >>>> >> If you reply to this email, your message will be added to the >>>> discussion >>>> >> below: >>>> >> >>>> >> http://apache-flink-user-mailing-list-archive.2336050.n4.nab >>>> ble.com/Dealing-with-Multiple-sinks-in-Flink-tp8643p8664.html >>>> >> To start a new topic under Apache Flink User Mailing List archive., >>>> email >>>> >> [hidden email] >>>> >> To unsubscribe from Apache Flink User Mailing List archive., click >>>> here. >>>> >> NAML >>>> > >>>> > >>>> > >>>> > ________________________________ >>>> > View this message in context: Re: Dealing with Multiple sinks in >>>> Flink >>>> > >>>> > Sent from the Apache Flink User Mailing List archive. mailing list >>>> archive >>>> > at Nabble.com. >>>> >>>> >>>> ------------------------------ >>>> >>> If you reply to this email, your message will be added to the discussio= n >> below: >> http://apache-flink-user-mailing-list-archive.2336050.n4.nab >> ble.com/Dealing-with-Multiple-sinks-in-Flink-tp8643p8685.html >> To start a new topic under Apache Flink User Mailing List archive., emai= l [hidden >> email] >> To unsubscribe from Apache Flink User Mailing List archive., click here. >> NAML >> >> > > > *MyTimestampExtractor.java* (4K) Download Attachment > > > ------------------------------ > View this message in context: Re: Dealing with Multiple sinks in Flink > > Sent from the Apache Flink User Mailing List archive. mailing list archiv= e > at > Nabble.com. > > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > http://apache-flink-user-mailing-list-archive.2336050.n4. > nabble.com/Dealing-with-Multiple-sinks-in-Flink-tp8643p8703.html > To start a new topic under Apache Flink User Mailing List archive., email= [hidden > email] > To unsubscribe from Apache Flink User Mailing List archive., click here. > NAML > > > > ------------------------------ > View this message in context: Re: Dealing with Multiple sinks in Flink > > Sent from the Apache Flink User Mailing List archive. mailing list archiv= e > at > Nabble.com. > --001a1141921878e658053ae9bff5 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I would first try to understand the metrics system locally= , for example using the VisualVM tool, that also allows you to access JMX-e= xported metrics.
Once you've seen how it works, you can look into r= emote JMX access.

There's a page in the Flink = documentation about the metrics.

=
On Thu, Aug 25, 2016 at 8:36 PM, vinay patil <vinay18.patil@gmail.com> wrote:
Thanks Robert will try that, can you provide mor= e details on how to integrate that.

=C2=A0I had sysouts = to check if the watermarks are getting generated, I am getting the values, = but as I said the window is not getting triggered for parallelism greater t= han 1, I have tried using AscendingTimeStampExtractor but facing the same i= ssue.



Regards,
Vinay Patil

On Thu, Aug 25, 2016 at 1:3= 2 PM, rmetzger0 [via Apache Flink User Mailing List archive.] <[hidden ema= il]> wrote:
Flink 1.1.1 has a metric for exposing the low watermark o= f each operator.
Maybe you can access a TaskManager via JMX to see the = value of the WM.

Watermarks are sometimes a bit tr= icky.
<= br>
On Thu, Aug 25, 2016 at 4:29 PM, vinay = patil <[hidden email]> wrote:
<= div dir=3D"ltr">Hi Max,

Here is the code for Timestamp a= ssigner and watermark generation.
PFA

Regard= s,
Vinay Patil
<= /div>

On Thu, Aug 25, 2016 at 7:39 AM, Maxim= ilian Michels [via Apache Flink User Mailing List archive.] <[hidden email= ]> wrote:
I'm assuming there is something wrong with your Watermark/Timestamp
assigner. Could you share some of the code?

On Wed, Aug 24, 2016 at 9:54 PM, vinay patil= <[hidden email]<= /a>> wrote:

> Hi,
>
> Just an update, the window is not getting triggered when I change = the
> parallelism to more than 1.
>
> Can you please explain why this is happening ?
>
> Regards,
> Vinay Patil
>
> On Wed, Aug 24, 2016 at 9:55 AM, vinay patil [via Apache Flink Use= r Mailing
> List archive.] <[hidden email]> w= rote: >> [hidden email]
>> To unsubscribe from Apache Flink User Mailing List arc= hive., click here.
>> NAML
>
>
>
> ________________________________
> View this message in context: Re: Dealing with Multiple sinks in F= link
>
> Sent from the Apache Flink User Mailing List archive. mailing list= archive
> at Nabble.com.
=09 =09 =09

If you reply to this email, your message = will be added to the discussion below:
http://apache-fli= nk-user-mailing-list-archive.2336050.n4.nabble.com/Dealing-with-M= ultiple-sinks-in-Flink-tp8643p8685.html
To start a new topic under Apache Flink User Mailing List archive., email= [hidden email] =
To unsubscribe from Apache Flink User Mailing List archive., click here.
NAML<= /a>
If you reply to this email, your message = will be added to the discussion below:
http://apache-fli= nk-user-mailing-list-archive.2336050.n4.nabble.com/Dealing-with-M= ultiple-sinks-in-Flink-tp8643p8703.html
To start a new topic under Apache Flink User Mailing List archive., email= [hidden email] =
To unsubscribe from Apache Flink User Mailing List archive., click here.
NAML<= /a>

=09 =09 =09
View this message in context: Re: Dealing with Multiple sinks in Flink
Sent from the Apache Flink User Mailing List archiv= e. mailing list archive at Nabble.com.

--001a1141921878e658053ae9bff5--