nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Payne (JIRA)" <>
Subject [jira] [Commented] (NIFI-293) Add a JDBC Processor for executing arbitrary SQL queries
Date Thu, 22 Jan 2015 19:52:35 GMT


Mark Payne commented on NIFI-293:


Some of us have discussed building such a thing before, but decided we needed more input from
the community about the intended use cases - specifically, as Joe is asking, what do we do
with the result of the SQL query? Serialize the output as XML? JSON? CSV? Make this configurable
as a 'Serialization Strategy' property? Or perhaps the results should become FlowFile attributes?

In the case of UPDATE, INSERT, or DELETE, the returned value is far less interesting to serialize
as FlowFile content and would perhaps go onto the flowfile as an attribute named 'sql.updated.row.count'
or something like that.

I agree that a Controller Service would be useful to provide a connection pool. This would
likely be exposed as an optional property, so that if none is selected a 'local' connection
pool could be used.

> Add a JDBC Processor for executing arbitrary SQL queries
> --------------------------------------------------------
>                 Key: NIFI-293
>                 URL:
>             Project: Apache NiFi
>          Issue Type: New Feature
>            Reporter: Ricky Saltzer
> This could be very useful for a variety of tasks, such as updating a value in a PostgreSQL
table, or adding a new partition to Hive. 
> Ideally, SQL commands could be generated using the NiFi expression language using FlowFile
> The processor should as generic as possible so that any of the popular JDBC drivers can
be used (e.g. PostgreSQL, Hive, Impala). 
> I'm still new to how processors are architected, but it seems that using a pre-defined
service in the _services.xml_ file (like the distributed map cache) would be the most efficient
way to share a connection pool across multiple JDBC processors. 

This message was sent by Atlassian JIRA

View raw message