flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adarsh Jain <eradarshj...@gmail.com>
Subject Re: Recursive Traversal of the Input Path Directory, Not working
Date Fri, 23 Jun 2017 09:25:02 GMT
Hi Stefan,

Thanks for your efforts in checking the same, still doesn't work for me.

Can you copy paste the code you used maybe I am doing some silly mistake
and am not able to figure out the same.

Thanks again.

Regards,
Adarsh


On Fri, Jun 23, 2017 at 2:32 PM, Stefan Richter <s.richter@data-artisans.com
> wrote:

> Hi,
>
> I tried this out on the current master and the 1.3 release and both work
> for me everything works exactly as expected, for file names, a directory,
> and even nested directories.
>
> Best,
> Stefan
>
> Am 22.06.2017 um 21:13 schrieb Adarsh Jain <eradarshjain@gmail.com>:
>
> Hi Stefan,
>
> Yes your understood right, when I give full path till the filename it
> works fine however when I give path till
> directory it does not read the data, doesn't print any exceptions too ...
> I am also not sure why it is behaving like this.
>
> Should be easily replicable, in case you can try. Will be really helpful.
>
> Regards,
> Adarsh
>
> On Thu, Jun 22, 2017 at 9:00 PM, Stefan Richter <
> s.richter@data-artisans.com> wrote:
>
>> Hi,
>>
>> I am not sure I am getting the problem right: the code works if you use a
>> file name, but it does not work for directories? What exactly is not
>> working? Do you get any exceptions?
>>
>> Best,
>> Stefan
>>
>> Am 22.06.2017 um 17:01 schrieb Adarsh Jain <eradarshjain@gmail.com>:
>>
>> Hi,
>>
>> I am trying to use "Recursive Traversal of the Input Path Directory" in
>> Flink 1.3 using scala. Snippet of my code below. If I give exact file name
>> it is working fine. Ref https://ci.apache.org/proj
>> ects/flink/flink-docs-release-1.3/dev/batch/index.html
>>
>> import org.apache.flink.api.java.utils.ParameterTool
>> import org.apache.flink.api.java.{DataSet, ExecutionEnvironment}
>> import org.apache.flink.configuration.Configuration
>>
>> val config = new Configuration
>>     config.setBoolean("recursive.file.enumeration",true)
>>
>> val featuresSource: String = "file:///Users/adarsh/Document
>> s/testData/featurecsv/31c710ac40/2017/06/22"
>>
>> val testInput = env.readTextFile(featuresSource).withParameters(config)
>> testInput.print()
>>
>> Please guide how to fix this.
>>
>> Regards,
>> Adarsh
>>
>>
>>
>
>

Mime
View raw message