flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <trohrm...@apache.org>
Subject Re: [Discuss] Read from S3 using AvroInputFormat
Date Fri, 08 Jul 2016 12:08:33 GMT
Hi Vinay,

have you tried only specifying the directory path you want to read [1]?

I think in the future these kind of questions should be posted to the user
mailing list user@flink.apache.org. The dev mailing list should be used for
discussions about Flink's development.

[1]
https://ci.apache.org/projects/flink/flink-docs-master/apis/batch/index.html#recursive-traversal-of-the-input-path-directory

Cheers,
Till

On Fri, Jul 8, 2016 at 1:50 PM, Vinay Patil <vinay18.patil@gmail.com> wrote:

> Hi,
>
> I am able to read avro file stored in S3 using AvroInputFormat by providing
> the full path.
>
> Is there a way I can read all the folders inside the bucket using existing
> Flink API.
>
> I know we can do this using the S3 REST Service, but I am more interested
> to do it in Flink (may be using Flink Dataset API for historical
> re-processing)
>
> Regards,
> Vinay Patil
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message