nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julian Hyde (JIRA)" <>
Subject [jira] [Commented] (NIFI-1280) Create FilterCSVColumns Processor
Date Fri, 13 May 2016 04:49:12 GMT


Julian Hyde commented on NIFI-1280:

Another possible integration with Calcite would be for Nifi to be a source for streaming (i.e.
continuously executing) Calcite queries. Calcite wouldn't be embedded in Nifi, but rather,
Calcite (or a streaming engine such as Flink, Storm, Samza, Apex, Beam) would continuously
read from Nifi. These queries would be continuous and would therefore start with the words
"select stream ...".

> Create FilterCSVColumns Processor
> ---------------------------------
>                 Key: NIFI-1280
>                 URL:
>             Project: Apache NiFi
>          Issue Type: Task
>          Components: Extensions
>            Reporter: Mark Payne
>            Assignee: Toivo Adams
> We should have a Processor that allows users to easily filter out specific columns from
CSV data. For instance, a user would configure two different properties: "Columns of Interest"
(a comma-separated list of column indexes) and "Filtering Strategy" (Keep Only These Columns,
Remove Only These Columns).
> We can do this today with ReplaceText, but it is far more difficult than it would be
with this Processor, as the user has to use Regular Expressions, etc. with ReplaceText.
> Eventually a Custom UI could even be built that allows a user to upload a Sample CSV
and choose which columns from there, similar to the way that Excel works when importing CSV
by dragging and selecting the desired columns? That would certainly be a larger undertaking
and would not need to be done for an initial implementation.

This message was sent by Atlassian JIRA

View raw message