Mailing-List: contact dev-help@apex.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@apex.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAHekGF--Fs43v26iMAcAhAaSG8m2jJQ1d4z8dJ9e6GTQkn_3og@mail.gmail.com>
References: 
 <CABAipVZUdRGx5dS35bYPN46koLQU5gSLgbCeRH8aM_y9ANRU2w@mail.gmail.com>
	<CAAMMvRnw5N-rGriFqV9-Fjm73ULDAadsS0+LirQ3LbvYnsT=5Q@mail.gmail.com>
	<CAJrpbrWO8NPpH_H-L3bGz9P26HDxT521gRYRbchtRgPJNH6TQA@mail.gmail.com>
	<CAHekGF-iTMkvNJe7-gc+QurEoEMRxZYO6eVcFKDLJE0tjHF5FQ@mail.gmail.com>
	<CAHekGF--Fs43v26iMAcAhAaSG8m2jJQ1d4z8dJ9e6GTQkn_3og@mail.gmail.com>
Date: Tue, 8 Mar 2016 17:48:29 +0530
Message-ID: 
 <CALDYq2Ykjiu2jPehCy6HUYkFvnGkoaQNpWntnti9+PO6c_110w@mail.gmail.com>
Subject: Re: Adding Transform Operator to Malhar
From: Devendra Tagare <devendrat@datatorrent.com>
To: dev@apex.incubator.apache.org
Content-Type: multipart/alternative; boundary=047d7bb04c4241caa7052d8894ca

--047d7bb04c4241caa7052d8894ca
Content-Type: text/plain; charset=UTF-8

+1

Dev


On Tue, Mar 8, 2016 at 5:38 PM, Yogi Devendra <yogidevendra@apache.org>
wrote:

> Forgot to add in earlier email:
>
> +1 for this operator.
>
> ~ Yogi
>
> On 8 March 2016 at 17:03, Yogi Devendra <yogidevendra@apache.org> wrote:
>
> > Can we think of better name than Transform operator?
> > May be:
> > Expression operator
> >
> > Reason:
> > We have many operators which are doing 'transform'. So, this name looks
> > very generic.
> >
> > In fact, from the users perspective, I always consider any operator like
> a
> > black-box doing some transformation.
> >
> > ~ Yogi
> >
> > On 8 March 2016 at 16:46, Mohit Jotwani <mohit@datatorrent.com> wrote:
> >
> >> This can have as many util functions added to it to perform various
> >> transformations to the tuple.
> >>
> >> +1
> >>
> >> Regards,
> >> Mohit
> >> On 8 Mar 2016 16:42, "Sandeep Deshmukh" <sandeep@datatorrent.com>
> wrote:
> >>
> >> > Looks good to me.
> >> >
> >> > Regards,
> >> > Sandeep
> >> >
> >> > On Mon, Mar 7, 2016 at 9:51 PM, Chinmay Kolhatkar <chinmay@apache.org
> >
> >> > wrote:
> >> >
> >> > > Dear Community,
> >> > >
> >> > > We're creating a transform operator which will allow the apex users
> to
> >> > > transform data over a stream using this operator.
> >> > >
> >> > > Use Case:
> >> > > ----
> >> > > When the data has been received from input source, one can transform
> >> data
> >> > > using transform operator and then dump it to destination.
> >> > >
> >> > > Transform means:
> >> > > ----
> >> > > 1. Conversion of fields from one type to another. Eg. int to
> Integer,
> >> > epoc
> >> > > to java.util.Date object, int to String or String to int etc.
> >> > > 2. Deriving new fields from one or more input fields.
> >> > >     For eg. Input contains 2 fields viz. "firstname", "lastname" and
> >> > > derived field is "fullname" which is combination of firstname and
> >> > lastname
> >> > > in some way.
> >> > >     Another example is date of birth present in input tuple,
> generate
> >> age
> >> > > from it.
> >> > > 3. Change in schema from input to output. For eg. Input schema is
> >> subset
> >> > of
> >> > > output schema and output schema contains some extra derived fields
> >> from
> >> > > input fields.
> >> > >
> >> > >
> >> > > Functionality:
> >> > > ----
> >> > > 1. Transform operator will access POJO as input tuple and emit
> >> > another/same
> >> > > POJO output tuple.
> >> > > 2. Operator needs to be configured with input tuple schema and
> output
> >> > tuple
> >> > > schema for this. This can be done via TUPLE_CLASS attribute on
> ports.
> >> > > 3. Generation of output tuple from input tuple can be specified
> using
> >> > > simple java based expressions.
> >> > >     For eg.
> >> > >        outfield1 = <expression using fields of input POJO>
> >> > >        outfield2 = <expression using fields of output POJO>
> >> > > 4. If no expression is mentioned for a certain output field, then a
> >> > > matching field (name and type) will be picked input POJO and
> contents
> >> > will
> >> > > be copied to output POJO.
> >> > > Otherwise, the output field will be left to default empty value.
> >> > >
> >> > >
> >> > > Expression Support in PojoUtils:
> >> > > ----
> >> > > 1. For achieving above functionality, I thought about extending
> >> PojoUtils
> >> > > which already support much of what is required. Then this utility
> can
> >> be
> >> > > used in Transform operator.
> >> > > 2. Couple of minor enhancements/fixing issues required to this
> >> utility:
> >> > >     a. Multiple POJO support is not required. Hence will stick to
> >> single
> >> > > POJO.
> >> > >     b. PojoUtils does not work well with resolution of getter
> methods
> >> > when
> >> > > more than one fields are mentioned in expression. Will need to fix
> >> that.
> >> > >     c. Following interface can be added to PojoUtils. This will be
> the
> >> > > return method that can be called for executing the expression
> >> > >         interface Expression<O>
> >> > >         {
> >> > >           O execute(Object obj);
> >> > >         }
> >> > >     d. Following methods can be added to PojoUtils for addressing
> >> > > expression support:
> >> > >         // Evaluates given expression
> >> > >         Expression evaluateExpression(String expr, Class
> >> inputObjectType,
> >> > > Class returnType);
> >> > >
> >> > >         // Adds a custom method that can be used in an expression.
> >> > >         // The one who's using this library can utility this method
> to
> >> > > provide predefined functionality to user.
> >> > >        void registerCustomMethod(String qualifiedFunc)
> >> > >
> >> > >
> >> > > Please share your valuable inputs on this.
> >> > >
> >> > > Thanks,
> >> > > Chinmay.
> >> > >
> >> >
> >>
> >
> >
>

--047d7bb04c4241caa7052d8894ca--