flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jun aoki <ja...@apache.org>
Subject how to write dataset in a file?
Date Sat, 21 Nov 2015 03:06:32 GMT
Hi Flink community

I know I'm mistaken but could not find what I want.

final ExecutionEnvironment env =
ExecutionEnvironment.createLocalEnvironment();
DataSet<String> data = env.readTextFile("file:///text1.txt");
FilterFunction<String> filter = new MyFilterFunction();  // looks for a
line starts with "[ERROR]"
DataSet<String> filteredData = data.filter(filter);
filteredData.writeAsText("file:///output1.txt");
env.execute();

Then I expect to get a single file /output1.txt , but actually get
/output1.txt/1, /output1.txt/2, /output1.txt/3...
I assumed I was getting a single file because the method signature says
writeAsText(String filePath).  <-- filePath instead of directoryPath
Also the Javadoc comment sounds like I assumed right.
https://github.com/apache/flink/blob/master/flink-java/src/main/java/org/apache/flink/api/java/DataSet.java#L1354

Can anyone tell if the method signature and document should be fixed? or if
I am missing some configuration?

-- 
-jun

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message