nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kothareddy (karthikk) [CONT - Type 2]" <karth...@micron.com>
Subject Correlate Processor ID in Logs
Date Tue, 22 Aug 2017 19:56:42 GMT
Hello All,

I am trying to build a monitoring mechanism for our flows and I'm considering using the "nifi-app.log"
as a primary source and filter them based on the messages. However, I see that a particular
message only has Processor name and ID for example,

ERROR [Timer-Driven Process Thread-36] o.a.nifi.processors.standard.ExecuteSQL ExecuteSQL[id=015a1007-548f-1bf5-1836-e4e53164d184]
Unable to execute SQL select query SELECT * FROM table WHERE comp_datetime <= '2017-01-31
23:59:59.813' ORDER BY datetime OFFSET 324000000 ROWS FETCH NEXT 1000000 ROWS ONLY for StandardFlowFileRecord[uuid=fc425c66-b83d-46d2-94bc-332e43345960,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1499803802779-112000, container=default, section=384],
offset=265042, length=114613],offset=53992,name=16290968101533439,size=167]

Given the above Error message it is really hard to correlate the ProcessorName/ID to the actual
name of the Processor or it's parent ProcessorGroup. Is there a way that I can correlate them
easily?

Also , I have considered using Bulletins as the source which is more fine grained to the actual
processor and ProcessorGroup it belongs to but problem with this approach is the rest call
only returns 5 bulletins back each time. And according to this post https://community.hortonworks.com/questions/72411/nifi-bulletinrepository-api-returns-maximum-5-bull.html
 it is a fixed value and practically not feasible to capture all of them if the flow has multiple
failures every second.


Any thoughts around this are much appreciated.

Thanks
Karthik

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message