apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pradeep Dalvi <pradeep.da...@datatorrent.com>
Subject Re: S3 Input Module
Date Fri, 18 Mar 2016 05:19:03 GMT
+1

On Thu, Mar 17, 2016 at 10:56 PM, Amol Kekre <amol@datatorrent.com> wrote:

> +1. Very common use case. Nice to have it.
>
> Thks
> Amol
>
>
> On Thu, Mar 17, 2016 at 1:49 AM, Sandeep Deshmukh <sandeep@datatorrent.com
> >
> wrote:
>
> > +1
> >
> > Many people face issues while copy data from S3 at large scale. This
> module
> > is a great contribution that can be readily used with simple
> configuration.
> >
> >
> > Regards,
> > Sandeep
> >
> > On Thu, Mar 17, 2016 at 2:04 PM, Priyanka Gugale <
> priyanka@datatorrent.com
> > >
> > wrote:
> >
> > > It's a good idea to extract out common code in parent class.
> > >
> > > +1 for this feature.
> > >
> > > -Priyanka
> > >
> > > On Thu, Mar 17, 2016 at 1:57 PM, Chaitanya Chebolu <
> > > chaitanya@datatorrent.com> wrote:
> > >
> > > > Dear Community,
> > > >
> > > >   I am proposing S3 Input Module. Primary functionality of this
> module
> > is
> > > > to parallel read files from S3 bucket.
> > > >
> > > >   Below is the JIRA created for this task:
> > > > https://issues.apache.org/jira/browse/APEXMALHAR-2019
> > > >
> > > >   Design of this module is similar to HDFS input module. So, I will
> > > extend
> > > > HDFS input module for S3 module.
> > > >
> > > >   Instead of extending HDFS input module, I will create common class
> > for
> > > > all such file system modules. JIRA for creating common class is here:
> > > > https://issues.apache.org/jira/browse/APEXMALHAR-2018
> > > >
> > > >  Please share your thoughts on this.
> > > >
> > > > Regards,
> > > > Chaitanya
> > > >
> > >
> >
>



-- 
Pradeep A. Dalvi

Software Engineer
DataTorrent (India)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message