flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Cannot see optimizer plan in web client
Date Mon, 03 Nov 2014 11:40:04 GMT
Hey!

Just to be sure I understand the problem correctly:

Do you not get the initial preview (on the page where you upload the jars),
or do you also not get the execution plan preview (after you clicked run),
even if you supply arguments?

Stephan




On Mon, Nov 3, 2014 at 12:29 PM, Vasiliki Kalavri <vasilikikalavri@gmail.com
> wrote:

> Hi,
>
> yes it's reproducible.
> I get the same behavior with the following dummy program:
>
> public class TestOptimizerPlan implements ProgramDescription {
>  @SuppressWarnings("serial")
> public static void main(String[] args) throws Exception {
> if (args.length < 2) {
> System.err.println("Usage: TestOptimizerPlan <input-file-path>
> <output-file-path>");
> return;
> }
>  ExecutionEnvironment env =
> ExecutionEnvironment.getExecutionEnvironment();
>  DataSet<Tuple2<Long, Long>> input = env.readCsvFile(args[0])
> .fieldDelimiter('\t').types(Long.class, Long.class);
>  DataSet<Tuple2<Long, Long>> result = input.map(
> new MapFunction<Tuple2<Long,Long>, Tuple2<Long,Long>>() {
> public Tuple2<Long, Long> map(Tuple2<Long, Long> value){
> return new Tuple2<Long, Long>(value.f0, value.f1+1);
> }
> });
> result.writeAsCsv(args[1], "\n", "\t");
> env.execute();
> }
> @Override
> public String getDescription() {
> return "TestOptimizerPlan <input-file-path> <output-file-path>";
> }
> }
>
> Cheers,
> Vasia.
>
> On 3 November 2014 09:23, Stephan Ewen <sewen@apache.org> wrote:
>
>> Hi!
>>
>> The web Frontend needs to "intercept" the execution plan when the call to
>> "execute()" comes, which is done with a special exception. The error
>> message says it failed, which is typically because that exception was
>> caught.
>>
>> However, in your code I see nothing that should prevent that plan
>> interception. Is the problem reproducible?
>>
>> Greetings,
>> Stephan
>>  Am 02.11.2014 16:36 schrieb "Vasiliki Kalavri" <
>> vasilikikalavri@gmail.com>:
>>
>> Hi all,
>>>
>>> when using the web client to upload and run a jar, the optimizer plan
>>> for my program does not appear.
>>> I have checked the "Show optimizer plan" box and provided args in the
>>> Arguments field.
>>> When I continue to submit the job, it runs correctly. There is no
>>> problem when using the plan visualizer tool.
>>>
>>> Checking the webclient logs, I see this exception in the .log file:
>>>
>>> org.apache.flink.client.program.ProgramInvocationException: The program
>>> plan could not be fetched. The program silently swallowed the control flow
>>> exceptions.
>>>         at
>>> org.apache.flink.client.program.PackagedProgram.getPreviewPlan(PackagedProgram.java:243)
>>>         at
>>> org.apache.flink.client.web.PactJobJSONServlet.doGet(PactJobJSONServlet.java:90)
>>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:734)
>>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
>>>         at
>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:532)
>>>         at
>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
>>>         at
>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
>>>         at
>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:965)
>>>         at
>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:388)
>>>         at
>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:187)
>>>         at
>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:901)
>>>         at
>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>>>         at
>>> org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:47)
>>>         at
>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:113)
>>>         at org.eclipse.jetty.server.Server.handle(Server.java:352)
>>>         at
>>> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:596)
>>>         at
>>> org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048)
>>>         at
>>> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:549)
>>>         at
>>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:211)
>>>         at
>>> org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:425)
>>>         at
>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:489)
>>>         at
>>> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:436)
>>>         at java.lang.Thread.run(Thread.java:745)
>>>
>>> and the usage message in the .out file, as if I did not provide the
>>> correct number of arguments (which I did).
>>>
>>> If it helps, here's the program I was trying to visualize:
>>> https://github.com/vasia/incubator-flink/blob/semimetric-preprocessing/flink-examples/flink-java-examples/src/main/java/org/apache/flink/examples/java/preprocessing/Jaccard.java
>>>
>>> Anything I'm missing here?
>>>
>>> Thanks,
>>> V.
>>>
>>
>

Mime
View raw message