flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Debaditya Roy <roydca...@gmail.com>
Subject Writing in flink clusters
Date Wed, 13 Jul 2016 13:20:15 GMT
Hello users,

I have written and executed a flink program in a cluster. The program was
supposed to write to some text file as a sink, however I cannot find the
text files in the target directory of the cluster nodes, but when I
reexecute the program second time, it gives me the predictable error:

Caused by: java.io.IOException: File or directory already exists. Existing
files and directories are not overwritten in NO_OVERWRITE mode. Use
OVERWRITE mode to overwrite existing files and directories.
    at
org.apache.flink.core.fs.FileSystem.initOutPathLocalFS(FileSystem.java:595)
    at
org.apache.flink.api.common.io.FileOutputFormat.open(FileOutputFormat.java:227)
    at
org.apache.flink.api.java.io.TextOutputFormat.open(TextOutputFormat.java:78)
    at
org.apache.flink.streaming.api.functions.sink.OutputFormatSinkFunction.open(OutputFormatSinkFunction.java:60)
    at
org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:38)
    at
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:91)
    at
org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:317)
    at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:215)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
    at java.lang.Thread.run(Thread.java:745)

Is it due to the fact that I am executing the program in a distributed
environment, if so how can I write the text files?

Thanks in advance.

Warm Regards,
Debaditya

Mime
View raw message