ranger-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Don Bosco Durai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (RANGER-1938) Solr for Audit setup doesn't use DocValues effectively
Date Thu, 21 Dec 2017 04:33:00 GMT

    [ https://issues.apache.org/jira/browse/RANGER-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16299544#comment-16299544

Don Bosco Durai commented on RANGER-1938:

bq. Does it make sense for me to open a JIRA against the Ambari project as well to fix this
schema? I can link to this JIRA since they are related.
Yes it would be good to create the JIRA. The Ranger members in Ambari team would be able to
replicate your changes there.

bq. I don't think there is a great reason to not enable at the "global" fieldType level. We
can disable at the individual field level if necessary. 
We can do either way. We can explicitly set the docValues for each field or do the global
setting. If we are going to do global changes, then let's document it in the file itself,
so anyone looking into it in the future will be aware of the reason.

Thanks for the details of your environment. It is very useful.

> Solr for Audit setup doesn't use DocValues effectively
> ------------------------------------------------------
>                 Key: RANGER-1938
>                 URL: https://issues.apache.org/jira/browse/RANGER-1938
>             Project: Ranger
>          Issue Type: Improvement
>          Components: audit
>    Affects Versions: 0.6.0, 0.7.0, 0.6.1, 0.6.2, 0.6.3, 0.7.1
>            Reporter: Kevin Risden
>            Assignee: Kevin Risden
>              Labels: performance
>             Fix For: 1.0.0, 0.7.2
>         Attachments: 0001-RANGER-1938-Enable-DocValues-for-more-fields-in-Solr.patch
> Ranger uses Ambari Infra Solr (or another Apache Solr install) for storing Ranger Audit
events for displaying in Ranger Admin. In our case, we have noticed quite a few Ambari Infra
Solr OOM due to Ranger. I've talked with a few other people who are having very similar problems
with OOM errors.
> I've typed up some details about how the way Ranger is using Solr requires a lot of heap.
I've also outlined the fix for this which significantly reduced the amount of heap memory
required. I'm an Apache Lucene/Solr committer so this optimization/usage might not be immediately
obvious to those using Solr especially version 5.x.
> https://risdenk.github.io/2017/12/18/ambari-infra-solr-ranger.html

This message was sent by Atlassian JIRA

View raw message