flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vasiliki Kalavri <vasilikikala...@gmail.com>
Subject Re: Cannot see optimizer plan in web client
Date Mon, 03 Nov 2014 11:29:08 GMT
Hi,

yes it's reproducible.
I get the same behavior with the following dummy program:

public class TestOptimizerPlan implements ProgramDescription {
 @SuppressWarnings("serial")
public static void main(String[] args) throws Exception {
if (args.length < 2) {
System.err.println("Usage: TestOptimizerPlan <input-file-path>
<output-file-path>");
return;
}
 ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
 DataSet<Tuple2<Long, Long>> input = env.readCsvFile(args[0])
.fieldDelimiter('\t').types(Long.class, Long.class);
 DataSet<Tuple2<Long, Long>> result = input.map(
new MapFunction<Tuple2<Long,Long>, Tuple2<Long,Long>>() {
public Tuple2<Long, Long> map(Tuple2<Long, Long> value){
return new Tuple2<Long, Long>(value.f0, value.f1+1);
}
});
result.writeAsCsv(args[1], "\n", "\t");
env.execute();
}
@Override
public String getDescription() {
return "TestOptimizerPlan <input-file-path> <output-file-path>";
}
}

Cheers,
Vasia.

On 3 November 2014 09:23, Stephan Ewen <sewen@apache.org> wrote:

> Hi!
>
> The web Frontend needs to "intercept" the execution plan when the call to
> "execute()" comes, which is done with a special exception. The error
> message says it failed, which is typically because that exception was
> caught.
>
> However, in your code I see nothing that should prevent that plan
> interception. Is the problem reproducible?
>
> Greetings,
> Stephan
>  Am 02.11.2014 16:36 schrieb "Vasiliki Kalavri" <vasilikikalavri@gmail.com
> >:
>
> Hi all,
>>
>> when using the web client to upload and run a jar, the optimizer plan for
>> my program does not appear.
>> I have checked the "Show optimizer plan" box and provided args in the
>> Arguments field.
>> When I continue to submit the job, it runs correctly. There is no problem
>> when using the plan visualizer tool.
>>
>> Checking the webclient logs, I see this exception in the .log file:
>>
>> org.apache.flink.client.program.ProgramInvocationException: The program
>> plan could not be fetched. The program silently swallowed the control flow
>> exceptions.
>>         at
>> org.apache.flink.client.program.PackagedProgram.getPreviewPlan(PackagedProgram.java:243)
>>         at
>> org.apache.flink.client.web.PactJobJSONServlet.doGet(PactJobJSONServlet.java:90)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:734)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
>>         at
>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:532)
>>         at
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
>>         at
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
>>         at
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:965)
>>         at
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:388)
>>         at
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:187)
>>         at
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:901)
>>         at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>>         at
>> org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:47)
>>         at
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:113)
>>         at org.eclipse.jetty.server.Server.handle(Server.java:352)
>>         at
>> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:596)
>>         at
>> org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048)
>>         at
>> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:549)
>>         at
>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:211)
>>         at
>> org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:425)
>>         at
>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:489)
>>         at
>> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:436)
>>         at java.lang.Thread.run(Thread.java:745)
>>
>> and the usage message in the .out file, as if I did not provide the
>> correct number of arguments (which I did).
>>
>> If it helps, here's the program I was trying to visualize:
>> https://github.com/vasia/incubator-flink/blob/semimetric-preprocessing/flink-examples/flink-java-examples/src/main/java/org/apache/flink/examples/java/preprocessing/Jaccard.java
>>
>> Anything I'm missing here?
>>
>> Thanks,
>> V.
>>
>

Mime
View raw message