apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amol Kekre <a...@datatorrent.com>
Subject Re: Naming sugestion for HDFS output modules
Date Mon, 28 Mar 2016 18:22:08 GMT
Chinmay,
Sure, do update the devel guidelines once we get an agreement on this thead.

Thks
Amol


On Mon, Mar 28, 2016 at 11:18 AM, Chinmay Kolhatkar <chinmay@datatorrent.com
> wrote:

> @amol, that's a very explanation... Maybe it would be good point to add in
> development best practices, if we have any.
>
> ---
> Sent from mobile.
> On 28 Mar 2016 10:52 p.m., "Amol Kekre" <amol@datatorrent.com> wrote:
>
> > I agree. Whether a resuable code component is module or operator is
> > internal to the implementations. The interface is
> "name+ports+properties".
> > This is similar to a function signature in other languages. The name
> should
> > convey the functional nature of the code and nothing more.
> >
> > Thks
> > Amol
> >
> > On Mon, Mar 28, 2016 at 10:00 AM, Chinmay Kolhatkar <chinmay@apache.org>
> > wrote:
> >
> > > Sorry for delay in reply... Still catching up with emails..
> > >
> > > I'm not sure whether we should have "Module" in the names.
> > >
> > > How about HDFSTupleWriter and HDFSFileWriter?
> > >
> > > Thanks,
> > > Chinmay.
> > >
> > > ---
> > > Sent from mobile.
> > > On 23 Mar 2016 4:49 p.m., "Yogi Devendra" <yogidevendra@apache.org>
> > wrote:
> > >
> > > > Hi,
> > > >
> > > > Currently, I am in the process of developing HDFS output module:
> > > > We have two modules for HDFS output.
> > > > 1. Tuple based []
> > > > 2. File based (used for file copy)
> > > >
> > > > Currently, I am calling #1 as "HDFS output module" as this module is
> > the
> > > > one which will be mostly used to write tuples to HDFS.
> > > >
> > > > I am calling #2 as "HDFS file copy module"; because it is mainly used
> > > only
> > > > for file copy operations.
> > > >
> > > > Any suggestions for alternate names for these modules?
> > > > From the names we want to stress the following
> > > >
> > > >    - #2 to be used only for file copy operations (block by block
> copy)
> > > >    - #1 to be used for tuple by tuple write to HDFS
> > > >    - Both #1 and #2 are HDFS output modules.
> > > >
> > > > Actually, we thought of combining them into single module. But,
> problem
> > > is
> > > > port signatures for both the modules is different. Thus, combing them
> > > will
> > > > result in different ports based on configuration.
> > > > It would be confusing for the app developers to decide which ports
> > > should I
> > > > connect to if ports are changing based on the configuration.
> > > >
> > > > Question:
> > > >
> > > > 1. Name suggestion for #1?
> > > > a. HDFS output module b. HDFSTuplesWriteModule c.
> > > HDFSMsgBasedOutputModule
> > > > d. other (please specify)
> > > >
> > > > 2. Name suggestion for #2?
> > > > a. HDFS file copy module b. HDFSBlocksWriteModule c.
> > > > HDFSBlockBasedOutputModule
> > > > d.  HDFSFileCopyOutputModule e. other (please specify)
> > > >
> > > > ~ Yogi
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message