nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joseph Witt (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-626) ExecuteSQL processor for executing arbitrary SQL queries
Date Sat, 23 May 2015 15:26:17 GMT

    [ https://issues.apache.org/jira/browse/NIFI-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14557381#comment-14557381
] 

Joseph Witt commented on NIFI-626:
----------------------------------

Toivo

It is great that you're digging into this so much.  I would say that we should strive to support
arbitrarily large result sets.  The ideal scenario is to stream the response from the DB to
the flowfile outputstream.  This requires all parts of the design chain to support it; jdbc
client, data formats, etc..  If this cannot be done then we want to be sure to document that
well.  Such cases where large objects must be held in memory are the main sources of issues
with garbage collection and the like.

Thanks
Joe

> ExecuteSQL processor for executing arbitrary SQL queries
> --------------------------------------------------------
>
>                 Key: NIFI-626
>                 URL: https://issues.apache.org/jira/browse/NIFI-626
>             Project: Apache NiFi
>          Issue Type: Sub-task
>            Reporter: Toivo Adams
>            Assignee: Toivo Adams
>
> For example query can be:
> SELECT * FROM orders WHERE orderId = '${order.id}'
> where ${order.id} is FlowFile attribute.
> Result will serialized using Avro.
> Avro gives as possibility to have query Resultset metadata (column name and types) included
in FlowFile.
> Also Avro should allow streaming.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message