nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aldrin Piri (JIRA)" <>
Subject [jira] [Commented] (NIFI-1077) Allow ConvertCharacterSet to accept expression language
Date Sun, 01 Nov 2015 04:59:27 GMT


Aldrin Piri commented on NIFI-1077:

Build and contrib were good.


I don't think we should drop the CharacterSetValidator entirely as it handles expression language
should it be specified in lieu of a String

We can do both, so at least in the case of where a user chooses to explicitly provide a value,
we should catch this and let them know. 

Let me know your thoughts.  Otherwise, looks good to go. 

> Allow ConvertCharacterSet to accept expression language
> -------------------------------------------------------
>                 Key: NIFI-1077
>                 URL:
>             Project: Apache NiFi
>          Issue Type: Improvement
>            Reporter: Joseph Percivall
>            Priority: Minor
>             Fix For: 0.4.0
>         Attachments: NIFI-1077.patch
> This issue arose from a user on the mailing list. It demonstrates the need to be able
to use expression language to set the incoming (and potentially outgoing) character sets:
> I'm looking to process many files into common formats.  The source files are coming in
various character sets, mime types, and new line terminators.
> My thinking for a data flow was along these lines:
> GetFile (from many sub directories) -> 
> ExecuteStreamCommand (file -i) ->
> ConvertCharacterSet (from previous command to utf8) ->
> ReplaceText (to change any \r\n into \n) ->
> PutFile (into a directory structure based on values found in the original file path and
> Additional steps would be added for archiving a copy of the original, converting xml
files, etc.
> Attempting to process these with Nifi leaves me confused as to how to process within
the tool.  If I want to ConvertCharacterSet, I have to know the input type.  I setup a ExecuteStreamCommand
to file -i ${absolute.path:append(${filename})} which returned the expected values.  I don't
see a way to turn these results into input for the processor, which doesn't accept expression
language for that field.
> I also considered ConvertCSVToAvro as an interim step but notice the same issue.  Any
suggestions what this dataflow should look like?

This message was sent by Atlassian JIRA

View raw message