apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chandni Singh <chan...@datatorrent.com>
Subject Re: NFS Input Module
Date Thu, 05 May 2016 05:35:03 GMT
I am sorry Chaitanya but I have more questions about this

1. why is the FS Input Module abstract when by default it can return
FileSplitter & BlockReader in com.datatorrent.lib.io.fs?
 These implementations are not specific to NFS.

2. In the NFS module that you have suggested to create, what is specific to
NFS?

Please note: I have created a ticket APEXMALHAR-2081 to remove
FSFileSplitter from library and move its feature to the base operator.

Thanks,
Chandni

On Wed, May 4, 2016 at 10:29 PM, Chaitanya Chebolu <
chaitanya@datatorrent.com> wrote:

> FSFileSplitter & BlockReader are available in com.datatorrent.lib.io.fs
> package.
>
> On Thu, May 5, 2016 at 10:47 AM, Chandni Singh <singh.chandni@gmail.com>
> wrote:
>
> > Ok. What is specific about the fileSplitter and blockReader returned by
> > this implementation?
> >
> >
> > On May 4, 2016 9:43 PM, "Chaitanya Chebolu" <chaitanya@datatorrent.com>
> > wrote:
> >
> > > Hi Chandni,
> > >
> > > Properties wise nothing specific. FS Input Module is an abstract Module
> > and
> > > NFS Module implements the abstract methods - createFileSplitter() and
> > > createBlockReader().
> > >
> > > Regards,
> > > Chaitanya
> > >
> > > On Wed, May 4, 2016 at 9:45 PM, Chandni Singh <singh.chandni@gmail.com
> >
> > > wrote:
> > >
> > > > Hi Chaitanya,
> > > >
> > > > What will be specific in NFS Input Module that is not provided by FS
> > > Input
> > > > Module?
> > > >
> > > > Thanks,
> > > > Chandni
> > > >
> > > > On Wed, May 4, 2016 at 7:12 AM, Amol Kekre <amol@datatorrent.com>
> > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Thks
> > > > > Amol
> > > > >
> > > > > On Tue, May 3, 2016 at 10:06 PM, Sandeep Deshmukh <
> > > > sandeep@datatorrent.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > +1
> > > > > >
> > > > > > Regards,
> > > > > > Sandeep
> > > > > >
> > > > > > On Fri, Apr 29, 2016 at 3:26 PM, Mohit Jotwani <
> > > mohit@datatorrent.com>
> > > > > > wrote:
> > > > > >
> > > > > > > +1
> > > > > > >
> > > > > > > Regards,
> > > > > > > Mohit
> > > > > > >
> > > > > > > On Fri, Apr 29, 2016 at 2:09 PM, Chaitanya Chebolu <
> > > > > > > chaitanya@datatorrent.com> wrote:
> > > > > > >
> > > > > > > > Hi All,
> > > > > > > >
> > > > > > > >   I am proposing NFS Input Module. Use case is to
read large
> > > files
> > > > > from
> > > > > > > NFS
> > > > > > > > in parallel.
> > > > > > > >
> > > > > > > >  Design of NFS input module:
> > > > > > > >
> > > > > > > >    There is a common interface "FSInputModule" in
Malhar for
> > the
> > > > > input
> > > > > > > > Modules. NFS input Module extends from FSInputModule
and can
> be
> > > > > > achieved
> > > > > > > by
> > > > > > > > using FSFileSplitter and BlockReader operators.
> > > > > > > >
> > > > > > > >   Please share your thoughts on this.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Chaitanya
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message