nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From prabhu Mahendran <prabhuu161...@gmail.com>
Subject Re: How to count the number of lines in csv?
Date Fri, 09 Jun 2017 09:02:27 GMT
I have found that we can easily calculate lines using SplitText processor.

It having "fragment.count" attribute which say number of splits.

So no need for create custom processor.

Thanks for your mail

On Fri, Jun 9, 2017 at 1:14 PM, Venkat Williams <venkat.williams@gmail.com>
wrote:

> Write custom processor to with csv parser either using uniocity lib or
> open csv lib and do no.of records..
>
> I don't aware if there is something along with CSVRecord processor.
>
> Regards,
> Venkat
>
> On Fri, Jun 9, 2017 at 1:02 PM, prabhu Mahendran <prabhuu161994@gmail.com>
> wrote:
>
>> Hi All,
>>
>> i try to count the number of lines in csv.
>>
>> Following Reference,
>> https://nifi.apache.org/docs/nifi-docs/html/expression-langu
>> age-guide.html#count
>>
>> with help of below method.I have stored the entire flow file in "input"
>> attribute and then delimited by using LineFeed character.
>>
>> ${allDelineatedValues(${input}, "\n"):count()}
>>
>> i have used extracttext processor for store entire flowfile into those
>> attribute.
>>
>> I can able to get number of lines in smaller csv files only it cannot
>> work for huge files(10 GB files) due to extract text only holds
>> 2147463746 <(214)%20746-3746> bytes in Buffer size only.it leads wrong
>> number of count stored in attribute.
>>
>> *Is there is any other way to count number of lines in csv without use
>> Script?*
>>
>
>

Mime
View raw message