flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Documentation for translation of Job graph to Execution graph
Date Tue, 21 Jun 2016 15:23:05 GMT
the link has been added newly, yes.

Regarding Q1, since there is no documentation right now, I have to refer
you to our code. In the JobManager.scala class, there is a method "private
def submitJob(jobGraph, ...") where the ExecutionGraph is created. I think
that's a good starting point for looking through the code. (I also added
Till to the message if he wants to chime in)

Q2: Currently, Flink doesn't add the job name to the logs, so its indeed
not very easy to separate the log entries generated by different jobs. In
general, we recommend running one JobManager per job (multiple jobs is of
course also supported).

On Sat, Jun 18, 2016 at 1:41 AM, Bajaj, Abhinav <abhinav.bajaj@here.com>

> Hi,
> Thanks for sharing this link. I have not see it before. May be this is
> newly added in 1.0 docs. I will go through it.
> In general, there are two things I am trying to understand and get
> comfortable with -
>    1. How a Job graph is translated to Execution graph. The logs and
>    monitoring APIs are for the Execution graph. So, I need to map them to the
>    Job graph. I am trying to bridge this gap.
>    2. The job manager & task manager logs are tricky to decipher.
>    Especially when there are multiple jobs running. Is there a way to filter
>    the logs for a single job ?
> ~ Abhi
> From: Aljoscha Krettek <aljoscha@apache.org>
> Reply-To: "user@flink.apache.org" <user@flink.apache.org>
> Date: Friday, June 17, 2016 at 2:31 AM
> To: "user@flink.apache.org" <user@flink.apache.org>
> Subject: Re: Documentation for translation of Job graph to Execution graph
> Hi,
> I'm afraid there is no documentation besides the link that you posted and
> this one:
> https://ci.apache.org/projects/flink/flink-docs-release-1.0/concepts/concepts.html
> .
> With what parts are you having trouble? Maybe I can help.
> Cheers,
> Aljoscha
> On Thu, 16 Jun 2016 at 19:31 Bajaj, Abhinav <abhinav.bajaj@here.com>
> wrote:
>> Hi,
>> When troubleshooting a flink job, it is tricky to map the Job graph
>> (application code) to the logs & monitoring REST APIs.
>> So, I am trying to find documentation on how a Job graph is translated to
>> Execution graph.
>> I found this -
>> https://ci.apache.org/projects/flink/flink-docs-release-1.0/internals/job_scheduling.html
>> Any detailed documentation on the design and code components will be
>> helpful.
>> Thanks,
>> Abhi

View raw message