Mailing-List: contact users-help@nifi.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@nifi.apache.org
MIME-Version: 1.0
In-Reply-To: <CAE5XTi8HWciqgtMdWUQpM1OEwGTFJ4svd1=cFKccSnETKT6GWA@mail.gmail.com>
References: <CAFuL2BZb9giBdAH_LEjk-xuHChpYSDvhgiF_egp6_un4nJHj5A@mail.gmail.com>
 <CAE5XTi8HWciqgtMdWUQpM1OEwGTFJ4svd1=cFKccSnETKT6GWA@mail.gmail.com>
From: prabhu Mahendran <prabhuu161994@gmail.com>
Date: Fri, 9 Jun 2017 14:32:27 +0530
Message-ID: <CAFuL2BbstO6n=29G6=1u_HcUJK1+rq8VVX3GaHCEPig5q8jD_w@mail.gmail.com>
Subject: Re: How to count the number of lines in csv?
To: users@nifi.apache.org
Content-Type: multipart/alternative; boundary="f40304361f0280a5f70551833ae3"
archived-at: Fri, 09 Jun 2017 09:02:42 -0000

--f40304361f0280a5f70551833ae3
Content-Type: text/plain; charset="UTF-8"

I have found that we can easily calculate lines using SplitText processor.

It having "fragment.count" attribute which say number of splits.

So no need for create custom processor.

Thanks for your mail

On Fri, Jun 9, 2017 at 1:14 PM, Venkat Williams <venkat.williams@gmail.com>
wrote:

> Write custom processor to with csv parser either using uniocity lib or
> open csv lib and do no.of records..
>
> I don't aware if there is something along with CSVRecord processor.
>
> Regards,
> Venkat
>
> On Fri, Jun 9, 2017 at 1:02 PM, prabhu Mahendran <prabhuu161994@gmail.com>
> wrote:
>
>> Hi All,
>>
>> i try to count the number of lines in csv.
>>
>> Following Reference,
>> https://nifi.apache.org/docs/nifi-docs/html/expression-langu
>> age-guide.html#count
>>
>> with help of below method.I have stored the entire flow file in "input"
>> attribute and then delimited by using LineFeed character.
>>
>> ${allDelineatedValues(${input}, "\n"):count()}
>>
>> i have used extracttext processor for store entire flowfile into those
>> attribute.
>>
>> I can able to get number of lines in smaller csv files only it cannot
>> work for huge files(10 GB files) due to extract text only holds
>> 2147463746 <(214)%20746-3746> bytes in Buffer size only.it leads wrong
>> number of count stored in attribute.
>>
>> *Is there is any other way to count number of lines in csv without use
>> Script?*
>>
>
>

--f40304361f0280a5f70551833ae3
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I have found that we can easily calculate lines using Spli=
tText processor.<br><br>It having &quot;fragment.count&quot; attribute whic=
h say number of splits.<br><br>So no need for create custom processor.<br><=
br>Thanks for your mail</div><div class=3D"gmail_extra"><br><div class=3D"g=
mail_quote">On Fri, Jun 9, 2017 at 1:14 PM, Venkat Williams <span dir=3D"lt=
r">&lt;<a href=3D"mailto:venkat.williams@gmail.com" target=3D"_blank">venka=
t.williams@gmail.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_qu=
ote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex=
"><div dir=3D"ltr">Write custom processor to with csv parser either using u=
niocity lib or open csv lib and do no.of records..=C2=A0<div><br></div><div=
>I don&#39;t aware if there is something along with CSVRecord processor.<br=
><div><br></div><div>Regards,</div><div>Venkat</div></div></div><div class=
=3D"HOEnZb"><div class=3D"h5"><div class=3D"gmail_extra"><br><div class=3D"=
gmail_quote">On Fri, Jun 9, 2017 at 1:02 PM, prabhu Mahendran <span dir=3D"=
ltr">&lt;<a href=3D"mailto:prabhuu161994@gmail.com" target=3D"_blank">prabh=
uu161994@gmail.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quot=
e" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">=
<div dir=3D"ltr"><div>Hi All,<br><br>i try to count the number of lines in =
csv.<br><br>Following Reference,<br><a href=3D"https://nifi.apache.org/docs=
/nifi-docs/html/expression-language-guide.html#count" target=3D"_blank">htt=
ps://nifi.apache.org/docs/n<wbr>ifi-docs/html/expression-langu<wbr>age-guid=
e.html#count</a><br></div><div><br></div><div>with help of below method.I h=
ave stored the entire flow file in &quot;input&quot; attribute and then del=
imited by using LineFeed character.<br><br><span style=3D"color:rgba(0,0,0,=
0.9);font-family:&quot;Droid Sans Mono&quot;,&quot;DejaVu Sans Mono&quot;,m=
onospace;font-size:15px;word-spacing:-2.25px;background-color:rgb(248,248,2=
47)">${allDelineatedValues(${input}<wbr>, &quot;\n&quot;):count()}<br></spa=
n><br>i have used extracttext processor for store entire flowfile into thos=
e attribute.<br><br>I can able to get number of lines in smaller csv files =
only it cannot work for huge files(10 GB files) due to extract text only ho=
lds <a href=3D"tel:(214)%20746-3746" value=3D"+12147463746" target=3D"_blan=
k">2147463746</a> bytes in Buffer size <a href=3D"http://only.it" target=3D=
"_blank">only.it</a> leads wrong number of count stored in attribute.<br><b=
r><b>Is there is any other way to count number of lines in csv without use =
Script?</b></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>

--f40304361f0280a5f70551833ae3--