flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: Flink 1.0.0 reading files from multiple directory with wildcards
Date Mon, 21 Mar 2016 10:35:47 GMT
Hi,

no, this is currently not supported. However, I agree this would be a very
valuable addition to the FileInputFormat.
Would you mind opening a JIRA issue with your suggestions?

Until this is added to Flink, it can be implemented as a custom InputFormat
based on FileInputFormat by overriding the createInputSplits() method.

Best, Fabian

2016-03-21 0:11 GMT+01:00 Sourigna Phetsarath <gna.phetsarath@teamaol.com>:

> All,
>
> Do any of the Flink Data Sources support comma separated directories with
> wildcards?
>
> For example:
>
> env.readFile("/data/2016/01/01/*/*,/data/2016/01/02/*/*,
> /data/2016/01/03/*/*")
>
>
> Thanks in advance for any help that you can provide.
> --
>
>
> *Gna Phetsarath*System Architect // AOL Platforms // Data Services //
> Applied Research Chapter
> 770 Broadway, 5th Floor, New York, NY 10003
> o: 212.402.4871 // m: 917.373.7363
> vvmr: 8890237 aim: sphetsarath20 t: @sourigna
>
> * <http://www.aolplatforms.com>*
>

Mime
View raw message