flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Richter <s.rich...@data-artisans.com>
Subject Re: Recursive Traversal of the Input Path Directory, Not working
Date Thu, 22 Jun 2017 15:30:48 GMT
Hi,

I am not sure I am getting the problem right: the code works if you use a file name, but it
does not work for directories? What exactly is not working? Do you get any exceptions?

Best,
Stefan

> Am 22.06.2017 um 17:01 schrieb Adarsh Jain <eradarshjain@gmail.com>:
> 
> Hi,
> 
> I am trying to use "Recursive Traversal of the Input Path Directory" in Flink 1.3 using
scala. Snippet of my code below. If I give exact file name it is working fine. Ref https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/batch/index.html
<https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/batch/index.html>
> 
> import org.apache.flink.api.java.utils.ParameterTool
> import org.apache.flink.api.java.{DataSet, ExecutionEnvironment}
> import org.apache.flink.configuration.Configuration
> 
> val config = new Configuration
>     config.setBoolean("recursive.file.enumeration",true)
> 
> val featuresSource: String = "file:///Users/adarsh/Documents/testData/featurecsv/31c710ac40/2017/06/22"
> 
> val testInput = env.readTextFile(featuresSource).withParameters(config)
> testInput.print()
> 
> Please guide how to fix this.
> 
> Regards,
> Adarsh
> 


Mime
View raw message