nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Igor Kravzov <igork.ine...@gmail.com>
Subject Re: Need to measure start and end time for a processor
Date Sat, 28 May 2016 15:36:18 GMT
Thanks guys.

On Fri, May 27, 2016 at 6:44 PM Andy LoPresto <alopresto@apache.org> wrote:

> To add on to Lee’s excellent points, if you are concerned about capturing
> the total lifetime, you may be able to intercept the messages coming from
> the console application and insert an attribute for timing before Kafka
> processes those messages. You can then evaluate against that in NiFi. This
> will include the Kafka batching time, as the provenance metrics are
> NiFi-only.
>
>
> Andy LoPresto
> alopresto@apache.org
> *alopresto.apache@gmail.com <alopresto.apache@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On May 27, 2016, at 2:43 PM, Lee Laim <lee.laim@gmail.com> wrote:
>
> Igor,
>
> Here are 3 methods which may help:
>
>
> 1,  Processor Statistics  - 'Average Task Duration'  is one of the
> processor statistics you can use to determine, on average, how fast an
> individual process takes.  Additionally, 'Average Lineage duration' might
> give you some insight across multiple processors and queue times.  (right
> click on the processor)
>
> 2. The Data Provenance feature has a graphical representation of the
> progression of the data flow, with an interactive slider bar.
> (4th icon on the upper right side of the browser)
>
> 3. You could also use two Update Attribute processors to make a 'elapsed'
> attribute to profile a specific portion of your data flow:
>
> start_time =   ${now():toNumber()}
>
> data flow of some duration
>
> elapsed_time =  ${now():toNumber():minus(${start_time})}
>
>
> -Lee
>
>
>
>
> On Fri, May 27, 2016 at 2:21 PM, Igor Kravzov <igork.inexso@gmail.com>
> wrote:
>
>> It's probably a strange requirement.
>> I want to measure how long will take for Kafka/NiFi to process a certain
>> amount of messages.
>> From one pint there is a console application which will send messages to
>> Kafka in a loop.
>> GetKafka will read messages in batches and Splittext will split these.
>> Basically I  want to see how long it took NiFi to process messages from
>> Kafka.
>> Any idea how to achieve this?
>>
>> Thanks in advance.
>>
>
>
>

Mime
View raw message