flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adarsh Jain <eradarshj...@gmail.com>
Subject Re: Recursive Traversal of the Input Path Directory, Not working
Date Thu, 22 Jun 2017 19:13:14 GMT
Hi Stefan,

Yes your understood right, when I give full path till the filename it works
fine however when I give path till
directory it does not read the data, doesn't print any exceptions too ... I
am also not sure why it is behaving like this.

Should be easily replicable, in case you can try. Will be really helpful.

Regards,
Adarsh

On Thu, Jun 22, 2017 at 9:00 PM, Stefan Richter <s.richter@data-artisans.com
> wrote:

> Hi,
>
> I am not sure I am getting the problem right: the code works if you use a
> file name, but it does not work for directories? What exactly is not
> working? Do you get any exceptions?
>
> Best,
> Stefan
>
> Am 22.06.2017 um 17:01 schrieb Adarsh Jain <eradarshjain@gmail.com>:
>
> Hi,
>
> I am trying to use "Recursive Traversal of the Input Path Directory" in
> Flink 1.3 using scala. Snippet of my code below. If I give exact file name
> it is working fine. Ref https://ci.apache.org/projects/flink/flink-docs-
> release-1.3/dev/batch/index.html
>
> import org.apache.flink.api.java.utils.ParameterTool
> import org.apache.flink.api.java.{DataSet, ExecutionEnvironment}
> import org.apache.flink.configuration.Configuration
>
> val config = new Configuration
>     config.setBoolean("recursive.file.enumeration",true)
>
> val featuresSource: String = "file:///Users/adarsh/
> Documents/testData/featurecsv/31c710ac40/2017/06/22"
>
> val testInput = env.readTextFile(featuresSource).withParameters(config)
> testInput.print()
>
> Please guide how to fix this.
>
> Regards,
> Adarsh
>
>
>

Mime
View raw message