flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinay Patil <vinay18.pa...@gmail.com>
Subject Re: [Discuss] Read from S3 using AvroInputFormat
Date Fri, 08 Jul 2016 12:15:39 GMT
I was not aware that there is separate mailing list for discussions.
>From next time I will use that mailing list for my queries.

Thanks Till for that information.

Regards,
Vinay Patil

On Fri, Jul 8, 2016 at 5:38 PM, Till Rohrmann <trohrmann@apache.org> wrote:

> Hi Vinay,
>
> have you tried only specifying the directory path you want to read [1]?
>
> I think in the future these kind of questions should be posted to the user
> mailing list user@flink.apache.org. The dev mailing list should be used
> for
> discussions about Flink's development.
>
> [1]
>
> https://ci.apache.org/projects/flink/flink-docs-master/apis/batch/index.html#recursive-traversal-of-the-input-path-directory
>
> Cheers,
> Till
>
> On Fri, Jul 8, 2016 at 1:50 PM, Vinay Patil <vinay18.patil@gmail.com>
> wrote:
>
> > Hi,
> >
> > I am able to read avro file stored in S3 using AvroInputFormat by
> providing
> > the full path.
> >
> > Is there a way I can read all the folders inside the bucket using
> existing
> > Flink API.
> >
> > I know we can do this using the S3 REST Service, but I am more interested
> > to do it in Flink (may be using Flink Dataset API for historical
> > re-processing)
> >
> > Regards,
> > Vinay Patil
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message