Mailing-List: contact commits-help@nifi.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@nifi.apache.org
Date: Tue, 27 Oct 2015 21:56:27 +0000 (UTC)
From: "Joseph Percivall (JIRA)" <jira@apache.org>
To: commits@nifi.apache.org
Message-ID: <JIRA.12908360.1445980722000.76914.1445982987725@Atlassian.JIRA>
In-Reply-To: <JIRA.12908360.1445980722000@Atlassian.JIRA>
References: <JIRA.12908360.1445980722000@Atlassian.JIRA>
 <JIRA.12908360.1445980722447@arcas>
Subject: [jira] [Updated] (NIFI-1077) Allow ConvertCharacterSet to accept
 expression language
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/NIFI-1077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joseph Percivall updated NIFI-1077:
-----------------------------------
    Attachment: NIFI-1077.patch

Enabled expression language and added unit tests

> Allow ConvertCharacterSet to accept expression language
> -------------------------------------------------------
>
>                 Key: NIFI-1077
>                 URL: https://issues.apache.org/jira/browse/NIFI-1077
>             Project: Apache NiFi
>          Issue Type: Improvement
>            Reporter: Joseph Percivall
>            Assignee: Joseph Percivall
>            Priority: Minor
>         Attachments: NIFI-1077.patch
>
>
> This issue arose from a user on the mailing list. It demonstrates the need to be able to use expression language to set the incoming (and potentially outgoing) character sets:
> I'm looking to process many files into common formats.  The source files are coming in various character sets, mime types, and new line terminators.
> My thinking for a data flow was along these lines:
> GetFile (from many sub directories) -> 
> ExecuteStreamCommand (file -i) ->
> ConvertCharacterSet (from previous command to utf8) ->
> ReplaceText (to change any \r\n into \n) ->
> PutFile (into a directory structure based on values found in the original file path and filename)
> Additional steps would be added for archiving a copy of the original, converting xml files, etc.
> Attempting to process these with Nifi leaves me confused as to how to process within the tool.  If I want to ConvertCharacterSet, I have to know the input type.  I setup a ExecuteStreamCommand to file -i ${absolute.path:append(${filename})} which returned the expected values.  I don't see a way to turn these results into input for the processor, which doesn't accept expression language for that field.
> I also considered ConvertCSVToAvro as an interim step but notice the same issue.  Any suggestions what this dataflow should look like?


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)