flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lukas Kircher <lukas.kirc...@uni-konstanz.de>
Subject Recursive directory traversal with TextInputFormat
Date Wed, 07 Dec 2016 11:10:02 GMT
Hi all,

I am trying to read nested .csv files from a directory and want to switch from a custom SourceFunction
I implemented to the TextInputFormat. I have two questions:

1) Somehow only the file in the root directory is processed, nested files are skipped. What
am I missing? See the attachment for an SSCCE. I get the same result with flink 1.1.3 no matter
if I run it via the IDE or submit the job to the standalone binary. The file input splits
are all there, yet they don't seem to be processed.

2) What is the easiest way to read compressed .csv files (.zip)?

Thanks for your help, cheers

View raw message