nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff <jtsw...@gmail.com>
Subject Re: Data Formats in PutSQL
Date Fri, 19 Aug 2016 18:00:58 GMT
I read up on the comments from the PR, and give it a few minutes of
thought. :) I like the idea of adding the attribute for the individual
arguments to provide the format.  In my usage of NiFi, I mostly used SQL
functions to transform the data.  Since I knew the format going in, the SQL
functions could do most of what I needed.  However, I'm thinking that the
kind of functionality you're wanting to put into PutSQL might be useful in
other places.  I'm thinking that it might be a good idea to add functions
to the NiFi EL to handle those cases, and PutSQL could be a good place to
test the introduction of that logic.

On Fri, Aug 19, 2016 at 12:05 PM Peter Wicks (pwicks) <pwicks@micron.com>
wrote:

> As part of NIFI-2591 and https://github.com/apache/nifi/pull/883 I've
> been looking at adding Binary data support to PutSQL, which until now has
> not supported binary data types at all.
>
> After submitting the PR a discussion arose around supporting more than one
> style of binary encoding.  Currently my code only supports the binary
> encoding format provided by Avro.  A suggestion was made to provide hex
> binary encoding, but there is no way to know for certain that an Avro
> encoded binary string is not a hex string or visa-versa.
>
> I have proposed adding a new optional attribute to FlowFiles routed to
> PutSQL called sql.args.N.format.  By default PutSQL would assume that
> binary data was encoded using the single byte format Avro uses, but a user
> could provide a format option, ex. `hex`, that would allow the code to use
> an alternate encoding option.
>
> Thoughts?
>
> Regards,
>   Peter
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message