apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashwin Chandra Putta <ashwinchand...@gmail.com>
Subject Re: JDBC Poller implementation
Date Mon, 08 Aug 2016 19:57:31 GMT
+1 for common abstract class and specific implementations that are use case
specific.

Regards,
Ashwin.

On Mon, Aug 8, 2016 at 7:55 AM, Devendra Tagare <devendrat@datatorrent.com>
wrote:

> + 1 for a common Abstract Class.
>
> 0 - for adding ports.
>
> Use-case for CSV - data migration from columnar stores.Some users may want
> to move TB's of data from a store like Greenplum and then do
> transformations using the CSV parser that is already present.
>
> Thanks,
> Dev
>
> On Mon, Aug 8, 2016 at 7:39 AM, Yogi Devendra <yogidevendra@apache.org>
> wrote:
>
> > +1 for Tushar's idea for common abstract class.
> >
> > Additionally, POJO based output should be considered for active
> > development. CSV output can be deprecated. Since it can be achieved
> easily
> > using POJO + CSV formatter.
> >
> > Operators are meant to be lego blocks for reusable functionality to
> achieve
> > higher level functionality by clubbing multiple operators together.
> >
> >
> > ~ Yogi
> >
> > On 8 August 2016 at 17:45, Tushar Gosavi <tushar@datatorrent.com> wrote:
> >
> > > I would prefer a common abstract class having emitTuple method. And
> > > two different implementation one for emitting comma separated values
> > > and other emitting pojo.
> > >
> > > Regards,
> > > -Tushar.
> > >
> > >
> > > On Mon, Aug 8, 2016 at 5:34 PM, Priyanka Gugale
> > > <priyanka@datatorrent.com> wrote:
> > > > The concrete implementation is supposed to do mainly formatting of
> > input
> > > > data and emit in required form. Also it would be tricky to implement
> > some
> > > > abstract methods like "getTuple", you have to conditionally return
> the
> > > > values.
> > > >
> > > > -Priyanka
> > > >
> > > > On Mon, Aug 8, 2016 at 4:49 PM, Bhupesh Chawda <bhupesh@apache.org>
> > > wrote:
> > > >
> > > >> I am +1 for having two ports in the same concrete implementation.
> > > >> One port for POJOs and second one for CSV strings (if this is
> actually
> > > >> needed)
> > > >>
> > > >> +0 on having multiple concrete operators.
> > > >> I don't think it is necessary to create multiple concrete
> > > implementations
> > > >> just because we want the same data in different formats. Ports
> should
> > > serve
> > > >> the purpose.
> > > >>
> > > >> ~ Bhupesh
> > > >>
> > > >>
> > > >> On Mon, Aug 8, 2016 at 4:46 PM, Priyanka Gugale <
> > > priyanka@datatorrent.com>
> > > >> wrote:
> > > >>
> > > >> > Hi,
> > > >> >
> > > >> > JDBCPollerInputOperator in malhar repository emits comma separated
> > > list
> > > >> of
> > > >> > values as result of scan. As most of our input operators emit
> POJOs
> > I
> > > am
> > > >> > planning to add an implementation which emits pojo.
> > > >> >
> > > >> > I would like to discuss, if we should have two independent jdbc
> poll
> > > >> input
> > > >> > operators, one emits csv and other which emits pojo or we should
> > have
> > > one
> > > >> > operator having two ports?
> > > >> >
> > > >> > I prefer two operators to define clear intent of each operator,
> but
> > if
> > > >> > anyone has different opinion please suggest.
> > > >> >
> > > >> > -Priyanka
> > > >> >
> > > >>
> > >
> >
>



-- 

Regards,
Ashwin.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message