nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Burgess <mattyb...@apache.org>
Subject Re: Data extraction for 100 columns is possible in NiFi?
Date Mon, 30 Jan 2017 13:59:53 GMT
Prabhu,

I agree with Mark; if you want to use ExecuteScript for this, I have
an example of splitting fields (using a bar | delimiter, but you can
change to comma) [1].  If you have quoted values that can contain
commas, then like Mark said you may want to look at writing a custom
processor, or using a third-party library such as OpenCSV [2], I have
an example on including such things in an ExecuteScript configuration
[3].

Regards,
Matt

[1] http://funnifi.blogspot.com/2016/02/executescript-explained-split-fields.html
[2] http://opencsv.sourceforge.net/
[3] http://funnifi.blogspot.com/2016/02/executescript-using-modules.html

On Mon, Jan 30, 2017 at 8:43 AM, Mark Payne <markap14@hotmail.com> wrote:
> Prabhu,
>
> My guess is that you probably could find some way to do this with the
> standard out-of-the-box processors
> that come with NiFi. Perhaps by using Extract Text to extract the header
> columns, and then using ReplaceText
> and perhaps a few other processors. Going down this route though is likely
> to be incredibly inefficient, though,
> and hard to understand/maintain.
>
> I think this is a great use case for either a custom processor or a simple
> Groovy/Python script using the ExecuteScript
> Processor.
>
> Thanks
> -Mark
>
> On Jan 30, 2017, at 1:12 AM, prabhu Mahendran <prabhuu161994@gmail.com>
> wrote:
>
> I have a CSV data with 100 columns like below..,
>
> ,A,B,C,D,E,F,..........[upto 100 Header columns]
> Date,A1,B1,C1,D1,E1,F1..........[upto 100 Header columns]
> 30/01/2017 23:23:22,Majestic,32,2100.12....[upto 100 data columns]
>
> In data having first 2 header lines and 3 rd line is data in which i
> inserted with header lines in below format.
>
> In Database insert those data with following format.
>
> insert into data values('30/01/2017 23:23:22','A','A1','Majestic');
> insert into data values('30/01/2017 23:23:22','B','B1,'32');
>
> Please stop me if anything i'm doing wrong.
>
> Is this possible in apache nifi?
>
> Many thanks
>
>

Mime
View raw message