flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergei Poganshev <s.pogans...@slice.com>
Subject FileInputFormat that processes files in chronological order
Date Fri, 26 Apr 2019 16:03:49 GMT
Given a directory with input files of the following format:


Is there a way to make FileInputFormat with parallelism 2 split processing
by "shard" (folder) and then process files in chronological order
(file1.json, file2.json, file3.json) in each shard? Will I have to
implement a custom FilInputFormat for that?

View raw message