nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From prabhu Mahendran <prabhuu161...@gmail.com>
Subject Re: How to ensure the rows moved into SQL?
Date Tue, 13 Jun 2017 04:34:42 GMT
Thanks for your response.

i am having 10 files vary between number of rows.

For example one file has contains 30 lakhs rows and one having 70 lakhs of
rows.Since i have using two split text for split files with Line Split
Count.hence "fragment.count" not worked in it.

i have used nifi in windows so "wc -l" count not worked.

Is there any other way to ensure number of rows?
On 12-Jun-2017 11:20 PM, "Andy LoPresto" <alopresto@apache.org> wrote:

> Prabhu,
>
> You can get a row count on the incoming CSV files by routing them through
> SplitText and using the “fragment.count” value as the total number of
> (non-header) lines, or by using an ExecuteStreamCommand with the command
> “wc -l” which counts the number of lines in text. With this knowledge, you
> can then use a SQL query in ExecuteSQL to check the number of rows inserted
> in the last time slice or with a special identifier range by checking
> “max(id)” before the insert begins.
>
>
> Andy LoPresto
> alopresto@apache.org
> *alopresto.apache@gmail.com <alopresto.apache@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Jun 12, 2017, at 5:47 AM, prabhu Mahendran <prabhuu161994@gmail.com>
> wrote:
>
> Hi All,
>
> Since i need to know how to check all rows in csv moved in SQL Server.
>
> I have download csv files from HTTP.
>
> Just imagine i have 10 files i could move files one after another into SQL
> Server.
>
> i need to ensure if number of rows in csv moves correctly insert into SQL
> Server.
>
> For example if file having 100 rows to be insert into SQL Server how to
> check 100 rows correctly moved into SQL in NiFi.After move that 100 files
> then row next file moved into SQL.
>
> Is this possible to check number of rows in files equal to sql moved rows
> without using Script?
>
>
>

Mime
View raw message