flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Dealing with Multiple sinks in Flink
Date Thu, 25 Aug 2016 18:41:17 GMT
I would first try to understand the metrics system locally, for example
using the VisualVM tool, that also allows you to access JMX-exported
metrics.
Once you've seen how it works, you can look into remote JMX access.

There's a page in the Flink documentation about the metrics.

On Thu, Aug 25, 2016 at 8:36 PM, vinay patil <vinay18.patil@gmail.com>
wrote:

> Thanks Robert will try that, can you provide more details on how to
> integrate that.
>
>  I had sysouts to check if the watermarks are getting generated, I am
> getting the values, but as I said the window is not getting triggered for
> parallelism greater than 1, I have tried using AscendingTimeStampExtractor
> but facing the same issue.
>
>
>
> Regards,
> Vinay Patil
>
> On Thu, Aug 25, 2016 at 1:32 PM, rmetzger0 [via Apache Flink User Mailing
> List archive.] <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=8704&i=0>> wrote:
>
>> Flink 1.1.1 has a metric for exposing the low watermark of each operator.
>> Maybe you can access a TaskManager via JMX to see the value of the WM.
>>
>> Watermarks are sometimes a bit tricky.
>>
>> On Thu, Aug 25, 2016 at 4:29 PM, vinay patil <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=8703&i=0>> wrote:
>>
>>> Hi Max,
>>>
>>> Here is the code for Timestamp assigner and watermark generation.
>>> PFA
>>>
>>> Regards,
>>> Vinay Patil
>>>
>>> On Thu, Aug 25, 2016 at 7:39 AM, Maximilian Michels [via Apache Flink
>>> User Mailing List archive.] <[hidden email]
>>> <http:///user/SendEmail.jtp?type=node&node=8692&i=0>> wrote:
>>>
>>>> I'm assuming there is something wrong with your Watermark/Timestamp
>>>> assigner. Could you share some of the code?
>>>>
>>>> On Wed, Aug 24, 2016 at 9:54 PM, vinay patil <[hidden email]
>>>> <http:///user/SendEmail.jtp?type=node&node=8685&i=0>> wrote:
>>>>
>>>> > Hi,
>>>> >
>>>> > Just an update, the window is not getting triggered when I change the
>>>> > parallelism to more than 1.
>>>> >
>>>> > Can you please explain why this is happening ?
>>>> >
>>>> > Regards,
>>>> > Vinay Patil
>>>> >
>>>> > On Wed, Aug 24, 2016 at 9:55 AM, vinay patil [via Apache Flink User
>>>> Mailing
>>>> > List archive.] <[hidden email]> wrote:
>>>>
>>>> >>
>>>> >> Hi Max,
>>>> >>
>>>> >> I tried writing to local file as well, its giving me the same issue,
>>>> I
>>>> >> have attached the logs and dummy pipeline
>>>> code.logs.txtdummy_pipeline.txt
>>>> >>
>>>> >> ________________________________
>>>> >> If you reply to this email, your message will be added to the
>>>> discussion
>>>> >> below:
>>>> >>
>>>> >> http://apache-flink-user-mailing-list-archive.2336050.n4.nab
>>>> ble.com/Dealing-with-Multiple-sinks-in-Flink-tp8643p8664.html
>>>> >> To start a new topic under Apache Flink User Mailing List archive.,
>>>> email
>>>> >> [hidden email]
>>>> >> To unsubscribe from Apache Flink User Mailing List archive., click
>>>> here.
>>>> >> NAML
>>>> >
>>>> >
>>>> >
>>>> > ________________________________
>>>> > View this message in context: Re: Dealing with Multiple sinks in
>>>> Flink
>>>> >
>>>> > Sent from the Apache Flink User Mailing List archive. mailing list
>>>> archive
>>>> > at Nabble.com.
>>>>
>>>>
>>>> ------------------------------
>>>>
>>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://apache-flink-user-mailing-list-archive.2336050.n4.nab
>> ble.com/Dealing-with-Multiple-sinks-in-Flink-tp8643p8685.html
>> To start a new topic under Apache Flink User Mailing List archive., email [hidden
>> email] <http:///user/SendEmail.jtp?type=node&node=8692&i=1>
>> To unsubscribe from Apache Flink User Mailing List archive., click here.
>> NAML
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
> *MyTimestampExtractor.java* (4K) Download Attachment
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/attachment/8692/0/MyTimestampExtractor.java>
>
> ------------------------------
> View this message in context: Re: Dealing with Multiple sinks in Flink
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Dealing-with-Multiple-sinks-in-Flink-tp8643p8692.html>
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> at
> Nabble.com.
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-flink-user-mailing-list-archive.2336050.n4.
> nabble.com/Dealing-with-Multiple-sinks-in-Flink-tp8643p8703.html
> To start a new topic under Apache Flink User Mailing List archive., email [hidden
> email] <http:///user/SendEmail.jtp?type=node&node=8704&i=1>
> To unsubscribe from Apache Flink User Mailing List archive., click here.
> NAML
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>
> ------------------------------
> View this message in context: Re: Dealing with Multiple sinks in Flink
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Dealing-with-Multiple-sinks-in-Flink-tp8643p8704.html>
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> at
> Nabble.com.
>

Mime
View raw message