ambari-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Quiroga <quirogadf4w...@gmail.com>
Subject Re: Ranger Logging Health Check Ambari Alert
Date Fri, 29 Dec 2017 06:46:43 GMT
Just to close the loop on this one. Went ahead and created a JIRA against
Ambari.

https://issues.apache.org/jira/browse/AMBARI-22708
Ranger HDFS logging health Ambari Alert


On Fri, Dec 22, 2017 at 12:16 PM, David Quiroga <quirogadf4work@gmail.com>
wrote:

> Hello
>
> First some background:
>
> We were directed to retain audit/access records "forever" (technically 7
> years but that is basically forever in electronic log time).
>
> Each Hadoop component generates local audit logs as per their log4j
> settings. In our production system these logs would frequently fill up the
> disk. At first we would just compress them in place but that only works for
> so long and there was no redundancy with local disk storage. In others
> words, no long term plan.
>
> We started to discuss moving them to HDFS or a different storage solution.
> One of our team members pointed out the Ranger plugins are already logging
> the "same data" into HDFS.
> Probably after several meeting with the higher-ups, using Ranger logs as
> the record truth was approved. Components log4j settings were updated to
> purge data automatically.
>
> Purging local logs felt like operating with out a safety net.
> Thought it we be good to check that Ranger was successful logging to HDFS
> each day. Should mention this is a kerberized cluster, not that anything
> ever goes wrong with kerberos.
>
> Checking this would have certainly been possible with a shell script, but
> we have been pushing to centralize warning/alerts in Ambari. And so an
> Ambari alert python script to check on Ranger Logging Health was crafted.
>
> For the most part the alert was modeled after some of the hive alerts.
> At the moment it just checks that the daily /ranger/audit/<component> HDFS
> directory has been created.
>
> I am sure there is room for improvement but I was curious:
>
> 1. Has anyone run into this type of concern?
>     a. Would an alert like this be helpful?
>     b. Did you come up with another solution?
>
> 2. What is best way to get this out into the community (e.g. JIRA, if so
> Ranger or Ambari - I am checking with both mailing list)?
>   a. Any other advice on how to best share?
>
> Thank you for your time.
> -David
>

Mime
View raw message