nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: How to access the counter and provenance Info
Date Thu, 02 Jun 2016 17:12:32 GMT
Kumiko,

It would be up to you to choose if you want to reset it...

You could implement methods annotated with @OnSchedule and @OnStopped and
reset as needed, but by default it would only be reset if NiFi restarted,
which is the same for counters which are only held in memory.

Are you trying to get a rolling window over 24 hours? or do you just want
to start counting and every 24 hours reset to 0? Just trying to understand
the use case and what you would like to achieve.

There are no plans that I know of to allow querying counters and provenance
from with in processors.
My gut tells me that querying counters might be reasonable since it is like
just a simple lookup into a map by counter name, but would need input from
others here.
Querying provenance could be an extremely heavy operation and not something
that we want to expose to processor developers.

Outside of processors, there are two ways to get provenance events:

1) You can query the REST API:
https://nifi.apache.org/docs/nifi-docs/rest-api/index.html

2) You can implement a custom ReportingTask which gets access to provenance
events, example:
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/SiteToSiteProvenanceReportingTask.java#L153

Let us know if that helps.

-Bryan


On Thu, Jun 2, 2016 at 1:09 PM, Kumiko Yada <Kumiko.Yada@ds-iq.com> wrote:

> Is there a way to query all the rows in provenance?  This does't have to
> be in processors.
>
> Thanks
> Kumiko
> ________________________________
> From: Kumiko Yada <Kumiko.Yada@ds-iq.com>
> Sent: Thursday, June 2, 2016 9:27:46 AM
> To: users@nifi.apache.org
> Subject: Re: How to access the counter and provenance Info
>
> Hi Brian,
>
> Would the AtomicInteger counter reset when the custom processor is stopped
> and restarted?
>
> Is there a plan to add the ability to query the counter or provenance from
> processors?
>
> Thanks
> Kumiko
> ________________________________
> From: Bryan Bende <bbende@gmail.com>
> Sent: Thursday, June 2, 2016 7:55:15 AM
> To: users@nifi.apache.org
> Subject: Re: How to access the counter and provenance Info
>
> Hi Kumiko,
>
> I think you can only increment counters and report provenance events from
> processors, but not query counters or query provenance.
>
> If you are in a custom processor, could have an AtomicInteger counter that
> increments on every onTrigger and reset when 24 hours has passed?
>
> Thanks,
>
> Bryan
>
>
> On Thu, Jun 2, 2016 at 10:50 AM, Kumiko Yada <Kumiko.Yada@ds-iq.com
> <mailto:Kumiko.Yada@ds-iq.com>> wrote:
> Could anyone help me on this?
>
> Thanks
> Kumiko
> ________________________________
> From: Kumiko Yada <Kumiko.Yada@ds-iq.com<mailto:Kumiko.Yada@ds-iq.com>>
> Sent: Wednesday, June 1, 2016 4:22:25 PM
> To: users@nifi.apache.org<mailto:users@nifi.apache.org>
> Subject: RE: How to access the counter and provenance Info
>
> When I clicked the Data Provenance from context menu of processor, the
> item history are showed the NiFi Flow Data Provenance UI.  How can I get
> this item list from the custom processor?
>
> Thanks
> Kumiko
>
> From: Kumiko Yada [mailto:Kumiko.Yada@ds-iq.com<mailto:
> Kumiko.Yada@ds-iq.com>]
> Sent: Wednesday, June 1, 2016 12:56 PM
> To: users@nifi.apache.org<mailto:users@nifi.apache.org>
> Subject: How to access the counter and provenance Info
>
> Hello,
>
> I’d like to get the how many times custom processor run past 24 hours in
> onTrigger method.  How can I get this using counter or provenance Info?
>
> Thanks
> Kumiko
>
>

Mime
View raw message