apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Devendra Tagare <devend...@datatorrent.com>
Subject Re: JDBC Poller implementation
Date Mon, 08 Aug 2016 14:55:16 GMT
+ 1 for a common Abstract Class.

0 - for adding ports.

Use-case for CSV - data migration from columnar stores.Some users may want
to move TB's of data from a store like Greenplum and then do
transformations using the CSV parser that is already present.

Thanks,
Dev

On Mon, Aug 8, 2016 at 7:39 AM, Yogi Devendra <yogidevendra@apache.org>
wrote:

> +1 for Tushar's idea for common abstract class.
>
> Additionally, POJO based output should be considered for active
> development. CSV output can be deprecated. Since it can be achieved easily
> using POJO + CSV formatter.
>
> Operators are meant to be lego blocks for reusable functionality to achieve
> higher level functionality by clubbing multiple operators together.
>
>
> ~ Yogi
>
> On 8 August 2016 at 17:45, Tushar Gosavi <tushar@datatorrent.com> wrote:
>
> > I would prefer a common abstract class having emitTuple method. And
> > two different implementation one for emitting comma separated values
> > and other emitting pojo.
> >
> > Regards,
> > -Tushar.
> >
> >
> > On Mon, Aug 8, 2016 at 5:34 PM, Priyanka Gugale
> > <priyanka@datatorrent.com> wrote:
> > > The concrete implementation is supposed to do mainly formatting of
> input
> > > data and emit in required form. Also it would be tricky to implement
> some
> > > abstract methods like "getTuple", you have to conditionally return the
> > > values.
> > >
> > > -Priyanka
> > >
> > > On Mon, Aug 8, 2016 at 4:49 PM, Bhupesh Chawda <bhupesh@apache.org>
> > wrote:
> > >
> > >> I am +1 for having two ports in the same concrete implementation.
> > >> One port for POJOs and second one for CSV strings (if this is actually
> > >> needed)
> > >>
> > >> +0 on having multiple concrete operators.
> > >> I don't think it is necessary to create multiple concrete
> > implementations
> > >> just because we want the same data in different formats. Ports should
> > serve
> > >> the purpose.
> > >>
> > >> ~ Bhupesh
> > >>
> > >>
> > >> On Mon, Aug 8, 2016 at 4:46 PM, Priyanka Gugale <
> > priyanka@datatorrent.com>
> > >> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > JDBCPollerInputOperator in malhar repository emits comma separated
> > list
> > >> of
> > >> > values as result of scan. As most of our input operators emit POJOs
> I
> > am
> > >> > planning to add an implementation which emits pojo.
> > >> >
> > >> > I would like to discuss, if we should have two independent jdbc poll
> > >> input
> > >> > operators, one emits csv and other which emits pojo or we should
> have
> > one
> > >> > operator having two ports?
> > >> >
> > >> > I prefer two operators to define clear intent of each operator, but
> if
> > >> > anyone has different opinion please suggest.
> > >> >
> > >> > -Priyanka
> > >> >
> > >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message