flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philip Lee <philjj...@gmail.com>
Subject Re: Hello, a question about Dashborad in Flink
Date Fri, 29 Jan 2016 07:50:44 GMT

Is there any way to measure shuffle data (read and write) on Flink or

I did not find the network usage metric in it.


On Mon, Jan 25, 2016 at 5:06 PM, Fabian Hueske <fhueske@gmail.com> wrote:

> You can start a job and then periodically request and store information
> about the running job and vertices from using corresponding REST calls [1].
> The data will be in JSON format.
> After the job finished, you can stop requesting data.
> Next you parse the JSON, extract the information you need and give it to
> some plotting library.
> As I said, it is not possible to pass this data back into Flink's
> dashboard, but you have to process and plot it yourself.
> Best, Fabian
> [1]
> https://ci.apache.org/projects/flink/flink-docs-master/internals/monitoring_rest_api.html#overview-of-jobs
> 2016-01-25 16:15 GMT+01:00 Philip Lee <philjjoon@gmail.com>:
>> Hello,
>> According to
>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Apache-Flink-Web-Dashboard-Completed-Job-history-td4067.html,
>> I cannot retrieve the job history from Dashboard after turnning off JM.
>> But as Fabian mentioned here,
>> "However, you can query all stats that are displayed by the dashboard via
>> a REST API [1] while the JM is running and save them yourself. This way you
>> can analyze the data also after the JM was stopped" could you explain about
>> this sentence in detail.
>> I want to evaluate timeline view of each function after a job is done.
>> Thanks,
>> Phil

View raw message