Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
MIME-Version: 1.0
Sender: ewenstephan@gmail.com
In-Reply-To: 
 <CAAdrtT3erknRN7W7icPtY55YsarvArV14qfhJuGPj3=tcXYf-A@mail.gmail.com>
References: <1452768969203-4290.post@n4.nabble.com>
	<CAKADb_NaYb6_nC+a=K3VSCuuxjbiJKEfgwgcZ9e4LvtzYud6jQ@mail.gmail.com>
	<1452783722677-4297.post@n4.nabble.com>
	<CAAdrtT3erknRN7W7icPtY55YsarvArV14qfhJuGPj3=tcXYf-A@mail.gmail.com>
Date: Fri, 15 Jan 2016 19:10:22 +0100
Message-ID: 
 <CANC1h_tTUv=Ddy228h_r1Gjpxk8gitLWF-TDOgiu62YREOkHXg@mail.gmail.com>
Subject: Re: Flink Execution Plan
From: Stephan Ewen <sewen@apache.org>
To: user@flink.apache.org
Content-Type: multipart/alternative; boundary=001a1136ae7c1bf15105296351c9

--001a1136ae7c1bf15105296351c9
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Actually, the thing with the JSON plans is slightly different now:

There are two types of plans:

1) The plan that describes the user program originally. That is what you
get from env.getExecutionPlan().
In the Batch API, this has the result of the optimizer, in the streaming
API the stream graph.

2) There is a JSON plan for the JobGraph / ExecutionGraph. This is what the
web dashboard uses.
The main difference to the other JSON plan is that in the JobGraph, not all
operators are visible any more. Chained operations currently look like one
operator to the JobGraph.
Hence this JSON plan usually has fewer operators, and the names indicate
that an operator is actually a chain of operations.

Greetings,
Stephan


On Thu, Jan 14, 2016 at 6:15 PM, Fabian Hueske <fhueske@gmail.com> wrote:

> @Christian: I don't think that is possible.
>
> There are quite a few things missing in the JSON including:
> - User function objects (Flink ships objects not class names)
> - Function configuration objects
> - Data types
>
> Best, Fabian
>
> 2016-01-14 16:02 GMT+01:00 lofifnc <alexander.kolb@mni.thm.de>:
>
>> Hi M=C3=A1rton,
>>
>> Thanks for your answer. But now I'm even more confused as it somehow
>> conflicts with the documentation. ;)
>> According to the wiki and the stratosphere paper the JobGraph will be
>> submitted to the JobManager. And the JobManager will then translate it
>> into
>> the ExecutionGraph.
>>
>> > In order to track the status of the parallel vertex and channel
>> > instances individually, the Job Manager spans the Job Graph
>> > to the Execution Graph, as shown in Fig. 9. The Execution
>> > Graph contains a node for each parallel instance of a vertex,
>> > which we refer to as a task.
>>
>> So the ExecutionGraph should only be available at the JobManager and
>> contain
>> a node for each parallel instance of a operator and the corresponding
>> vertices.
>>
>> The question is in the context of my master thesis as I'm trying to
>> describe
>> the deployment process of Flink. And wan't to use a visualization of the
>> execution plan as an concrete example for one of these three Graphs.
>>
>> Best Alex!
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flin=
k-Execution-Plan-tp4290p4297.html
>> Sent from the Apache Flink User Mailing List archive. mailing list
>> archive at Nabble.com.
>>
>
>

--001a1136ae7c1bf15105296351c9
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Actually, the thing with the JSON plans is slightly d=
ifferent now:</div><div><br></div><div>There are two types of plans:</div><=
div><br></div><div>1) The plan that describes the user program originally. =
That is what you get from env.getExecutionPlan().</div><div>In the Batch AP=
I, this has the result of the optimizer, in the streaming API the stream gr=
aph.</div><div><br></div><div>2) There is a JSON plan for the JobGraph / Ex=
ecutionGraph. This is what the web dashboard uses.</div><div>The main diffe=
rence to the other JSON plan is that in the JobGraph, not all operators are=
 visible any more. Chained operations currently look like one operator to t=
he JobGraph.</div><div>Hence this JSON plan usually has fewer operators, an=
d the names indicate that an operator is actually a chain of operations.</d=
iv><div><br></div><div>Greetings,<br></div><div>Stephan</div><div><br></div=
><div><br></div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_qu=
ote">On Thu, Jan 14, 2016 at 6:15 PM, Fabian Hueske <span dir=3D"ltr">&lt;<=
a href=3D"mailto:fhueske@gmail.com" target=3D"_blank">fhueske@gmail.com</a>=
&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0=
 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div>=
<div><div><div>@Christian: I don&#39;t think that is possible. <br><br>Ther=
e are quite a few things missing in the JSON including:<br></div>- User fun=
ction objects (Flink ships objects not class names)<br></div>- Function con=
figuration objects<br></div>- Data types<br><br></div>Best, Fabian<br> </di=
v><div class=3D"HOEnZb"><div class=3D"h5"><div class=3D"gmail_extra"><br><d=
iv class=3D"gmail_quote">2016-01-14 16:02 GMT+01:00 lofifnc <span dir=3D"lt=
r">&lt;<a href=3D"mailto:alexander.kolb@mni.thm.de" target=3D"_blank">alexa=
nder.kolb@mni.thm.de</a>&gt;</span>:<br><blockquote class=3D"gmail_quote" s=
tyle=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi M=
=C3=A1rton,<br>
<br>
Thanks for your answer. But now I&#39;m even more confused as it somehow<br=
>
conflicts with the documentation. ;)<br>
According to the wiki and the stratosphere paper the JobGraph will be<br>
submitted to the JobManager. And the JobManager will then translate it into=
<br>
the ExecutionGraph.<br>
<br>
&gt; In order to track the status of the parallel vertex and channel<br>
&gt; instances individually, the Job Manager spans the Job Graph<br>
&gt; to the Execution Graph, as shown in Fig. 9. The Execution<br>
&gt; Graph contains a node for each parallel instance of a vertex,<br>
&gt; which we refer to as a task.<br>
<br>
So the ExecutionGraph should only be available at the JobManager and contai=
n<br>
a node for each parallel instance of a operator and the corresponding<br>
vertices.<br>
<br>
The question is in the context of my master thesis as I&#39;m trying to des=
cribe<br>
the deployment process of Flink. And wan&#39;t to use a visualization of th=
e<br>
execution plan as an concrete example for one of these three Graphs.<br>
<br>
Best Alex!<br>
<br>
<br>
<br>
<br>
--<br>
View this message in context: <a href=3D"http://apache-flink-user-mailing-l=
ist-archive.2336050.n4.nabble.com/Flink-Execution-Plan-tp4290p4297.html" re=
l=3D"noreferrer" target=3D"_blank">http://apache-flink-user-mailing-list-ar=
chive.2336050.n4.nabble.com/Flink-Execution-Plan-tp4290p4297.html</a><br>
<div><div>Sent from the Apache Flink User Mailing List archive. mailing lis=
t archive at Nabble.com.<br>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>

--001a1136ae7c1bf15105296351c9--