apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chandni Singh <chan...@datatorrent.com>
Subject Re: Changes to JdbcOutputOperator
Date Sat, 12 Dec 2015 00:13:09 GMT
Hi,

Here is an example of doing Upsert with JDBC:
https://github.com/chandnisingh/Malhar/blob/examples/apps/jdbc/src/main/java/com/datatorrent/jdbc/JdbcWriter.java

Thanks,
Chandni

On Fri, Dec 11, 2015 at 11:19 AM, Chandni Singh <chandni@datatorrent.com>
wrote:

> The operators are under Malhar/lib/db/jdbc.
>
> Here is one of them:
>
> https://github.com/apache/incubator-apex-malhar/blob/devel-3/library/src/main/java/com/datatorrent/lib/db/jdbc/AbstractJdbcTransactionableOutputOperator.java
>
> They work with any kind PreparedStatement - insert or update
>
> Chandni
>
>
>
> On Fri, Dec 11, 2015 at 10:59 AM, Bhupesh Chawda <bhupesh@datatorrent.com>
> wrote:
>
>> Hi Chandni,
>>
>> I don't see an update query being handled in the operator. Could you
>> please
>> point me to the appropriate class?
>> Or did you mean that handling a update query is just a matter of extending
>> the class and providing a concrete implementation?
>>
>> Thanks.
>> -Bhupesh
>>
>> On Fri, Dec 11, 2015 at 10:42 PM, Chandni Singh <chandni@datatorrent.com>
>> wrote:
>>
>> > Hi Bhupesh,
>> >
>> > The current abstract JDBC Output Operators  in library are generic and
>> have
>> > already been used in multiple POCs and applications. In fact this
>> operator
>> > has matured through customer use cases. It is not just an insert
>> operator.
>> > We have used it to perform update and inserts.
>> >
>> > That said, I don't think it is a good idea to introduce input format in
>> >  these abstract implementations. It is written to handle any type of
>> query,
>> > be it a procedure call (that was an actual customer use case).
>> >
>> > Chandni
>> >
>> > On Fri, Dec 11, 2015 at 2:50 AM, Bhupesh Chawda <
>> bhupesh@datatorrent.com>
>> > wrote:
>> >
>> > > Hi All,
>> > >
>> > > We are planning to proceed with the following approach for JDBC
>> *update*
>> > > operator:
>> > >
>> > >    - *Update Query Configuration*
>> > >
>> > >
>> > >    - Example Update Query: *update tableName set a = ?** where b = ?
>> and
>> > c
>> > >    > ?;*
>> > >    - Example JSON input array for parameter instantiations: *[{a,
>> > >    expression, INTEGER}, {b, expression, VARCHAR}, {c, expression,
>> > DATE}]*
>> > >
>> > > We are also planning to change the JDBC Output Operator in Malhar
>> Library
>> > > which currently does just insert. We plan to make the input format
>> > > consistent for both insert and update and hence the change to the
>> current
>> > > way of configuration using JSON. Following would be the config for
>> > inserts:
>> > >
>> > >    - *Insert Query Configuration*
>> > >
>> > >
>> > >    - Example Insert Query: *insert into tableName values (?, ?, .. ,
>> ?);*
>> > >    - Example JSON input array for parameter instantiations: *[{a,
>> > >    expression, INTEGER}, {b, expression, VARCHAR}, .. , {c,
>> expression,
>> > > DATE}]*
>> > >
>> > > Please let us know your thoughts.
>> > >
>> > > Thanks.
>> > >
>> > > -Bhupesh
>> > >
>> > >
>> > >
>> > > On Wed, Dec 9, 2015 at 6:38 PM, Bhupesh Chawda <
>> bhupesh@datatorrent.com>
>> > > wrote:
>> > >
>> > > > Hi All,
>> > > >
>> > > > Would it be a good idea to introduce the update functionality to the
>> > JDBC
>> > > > output operator in Apache Apex Malhar library.
>> > > >
>> > > > The following are possible approaches:
>> > > >
>> > > >    1. Accept a update query from the user with place holders for
>> > values.
>> > > >    Example: *update tableName set a = ?, b = ? where c = ? and d >
>> ?*.
>> > > >    Here "?" will be provided by the user as java expressions which
>> will
>> > > be
>> > > >    evaluated from the incoming tuple.
>> > > >    2. Another option is to accept in some configuration format
>> (json /
>> > > >    xml) the following and formulate the query in the operator. This
>> can
>> > > become
>> > > >    arbitrarily complex.
>> > > >    1. update clause columns
>> > > >       2. update clause expressions
>> > > >       3. where clause columns
>> > > >       4. where clause expressions
>> > > >
>> > > > I am thinking about going ahead with 1. Please let me know if any
>> other
>> > > > option is possible and whether such a functionality already exists
>> in
>> > > some
>> > > > other class.
>> > > >
>> > > > Thanks.
>> > > >
>> > > > -Bhupesh
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message