Return-Path: X-Original-To: apmail-ranger-commits-archive@www.apache.org Delivered-To: apmail-ranger-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D404A18A04 for ; Thu, 10 Dec 2015 20:18:06 +0000 (UTC) Received: (qmail 41441 invoked by uid 500); 10 Dec 2015 20:18:06 -0000 Delivered-To: apmail-ranger-commits-archive@ranger.apache.org Received: (qmail 41411 invoked by uid 500); 10 Dec 2015 20:18:06 -0000 Mailing-List: contact commits-help@ranger.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ranger.incubator.apache.org Delivered-To: mailing list commits@ranger.incubator.apache.org Received: (qmail 41402 invoked by uid 99); 10 Dec 2015 20:18:06 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Dec 2015 20:18:06 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 0F2BFC449F for ; Thu, 10 Dec 2015 20:18:06 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.991 X-Spam-Level: ** X-Spam-Status: No, score=2.991 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, KAM_MANYCOMMENTS=1.2, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id 1w7J83QFiWl9 for ; Thu, 10 Dec 2015 20:17:54 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with SMTP id C8EAD429D3 for ; Thu, 10 Dec 2015 20:17:52 +0000 (UTC) Received: (qmail 41252 invoked by uid 99); 10 Dec 2015 20:17:52 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Dec 2015 20:17:52 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 0ED0AE1790; Thu, 10 Dec 2015 20:17:51 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: bosco@apache.org To: commits@ranger.incubator.apache.org Date: Thu, 10 Dec 2015 20:17:52 -0000 Message-Id: <4f50633d32894eb091f5b6d9ef956f1f@git.apache.org> In-Reply-To: <42a1baff0a6a4e1bab58c97ea8213375@git.apache.org> References: <42a1baff0a6a4e1bab58c97ea8213375@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [2/3] incubator-ranger git commit: RANGER-728: Update Solr script to resolve issues with ZK and creating collection http://git-wip-us.apache.org/repos/asf/incubator-ranger/blob/269617d5/security-admin/contrib/solr_for_audit_setup/conf/solrconfig.xml.j2 ---------------------------------------------------------------------- diff --git a/security-admin/contrib/solr_for_audit_setup/conf/solrconfig.xml.j2 b/security-admin/contrib/solr_for_audit_setup/conf/solrconfig.xml.j2 new file mode 100644 index 0000000..b6017f3 --- /dev/null +++ b/security-admin/contrib/solr_for_audit_setup/conf/solrconfig.xml.j2 @@ -0,0 +1,1878 @@ + + + + + + + + + 5.0.0 + + + + + + + + + + + + + + + + + + + + + + + ${solr.data.dir:} + + + + + + + + + ${solr.hdfs.home:} + + ${solr.hdfs.confdir:} + + ${solr.hdfs.blockcache.enabled:true} + + ${solr.hdfs.blockcache.global:true} + + + + + + + + + + true + managed-schema + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ${solr.lock.type:native} + + + + + + + + + + + + + + + + true + + + + + + + + + + + + + + + + ${solr.ulog.dir:} + + + + + ${solr.autoCommit.maxTime:15000} + false + + + + + + ${solr.autoSoftCommit.maxTime:5000} + + + + + + + + + + + + + + + + 1024 + + + + + + + + + + + + + + + + + + + + + + + + + true + + + + + + 20 + + + 200 + + + + + + + + + + + + static firstSearcher warming in solrconfig.xml + + + + + + false + + + 2 + + + + + + + + + + + + + + + + + + + + + + solr-data-config.xml + + + + + + + + explicit + 10 + text + + + + + + + + + + + + + + explicit + json + true + text + + + + + + + + true + json + true + + + + + + + + explicit + + + velocity + browse + layout + + + edismax + *:* + 10 + *,score + + + on + 1 + + + + + + + text + add-unknown-fields-to-the-schema + + + + + + + + + + + + + true + ignored_ + + + true + links + ignored_ + + + + + + + + + + + + + + + + + + + + + + + + solrpingquery + + + all + + + + + + + + + explicit + true + + + + + + + + + + + + + + + + text_general + + + + + + default + text + solr.DirectSolrSpellChecker + + internal + + 0.5 + + 2 + + 1 + + 5 + + 4 + + 0.01 + + + + + + wordbreak + solr.WordBreakSolrSpellChecker + name + true + true + 10 + + + + + + + + + + + + + + + + text + + default + wordbreak + on + true + 10 + 5 + 5 + true + true + 10 + 5 + + + spellcheck + + + + + + mySuggester + FuzzyLookupFactory + DocumentDictionaryFactory + cat + price + string + + + + + + true + 10 + + + suggest + + + + + + + + + text + true + + + tvComponent + + + + + + + lingo + + + org.carrot2.clustering.lingo.LingoClusteringAlgorithm + + + clustering/carrot2 + + + + + stc + org.carrot2.clustering.stc.STCClusteringAlgorithm + + + + + kmeans + org.carrot2.clustering.kmeans.BisectingKMeansClusteringAlgorithm + + + + + + + true + true + + name + + id + + features + + true + + + + false + + + edismax + + text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4 + + *:* + 10 + *,score + + + clustering + + + + + + + + + + true + false + + + terms + + + + + + + + string + elevate.xml + + + + + + explicit + text + + + elevator + + + + + + + + + + + 100 + + + + + + + + 70 + + 0.5 + + [-\w ,/\n\"']{20,200} + + + + + + + ]]> + ]]> + + + + + + + + + + + + + + + + + + + + + + + + ,, + ,, + ,, + ,, + ,]]> + ]]> + + + + + + 10 + .,!? + + + + + + + WORD + + + en + US + + + + + + + + + + + _ttl_ + +{{MAX_AUDIT_RETENTION_DAYS}}DAYS + + + 300 + _ttl_ + _expire_at_ + + + _expire_at_ + + + + + + + + + yyyy-MM-dd'T'HH:mm:ss.SSSZ + yyyy-MM-dd'T'HH:mm:ss,SSSZ + yyyy-MM-dd'T'HH:mm:ss.SSS + yyyy-MM-dd'T'HH:mm:ss,SSS + yyyy-MM-dd'T'HH:mm:ssZ + yyyy-MM-dd'T'HH:mm:ss + yyyy-MM-dd'T'HH:mmZ + yyyy-MM-dd'T'HH:mm + yyyy-MM-dd HH:mm:ss.SSSZ + yyyy-MM-dd HH:mm:ss,SSSZ + yyyy-MM-dd HH:mm:ss.SSS + yyyy-MM-dd HH:mm:ss,SSS + yyyy-MM-dd HH:mm:ssZ + yyyy-MM-dd HH:mm:ss + yyyy-MM-dd HH:mmZ + yyyy-MM-dd HH:mm + yyyy-MM-dd + + + + text_general + + java.lang.Boolean + booleans + + + java.util.Date + tdates + + + java.lang.Long + java.lang.Integer + tlongs + + + java.lang.Number + tdoubles + + + + + + + + + + + + + + + + + + + + + + + text/plain; charset=UTF-8 + + + + + ${velocity.template.base.dir:} + + + + + 5 + + + + + + + + + + + + + + + + + + *:* + + + http://git-wip-us.apache.org/repos/asf/incubator-ranger/blob/269617d5/security-admin/contrib/solr_for_audit_setup/install.properties ---------------------------------------------------------------------- diff --git a/security-admin/contrib/solr_for_audit_setup/install.properties b/security-admin/contrib/solr_for_audit_setup/install.properties index 92ec1e6..bed980b 100644 --- a/security-admin/contrib/solr_for_audit_setup/install.properties +++ b/security-admin/contrib/solr_for_audit_setup/install.properties @@ -26,6 +26,9 @@ #The operating system (linux) user used by Solr process. You need to run Solr as the below user SOLR_USER=solr +#How long to keep the audit logs. Default is 3 years (1095 days) +MAX_AUDIT_RETENTION_DAYS=1095 + #If you want this script to isntall Solr, set the value to true. If it is already installed, then set this to false #If it is true, then it will download and install it. #NOTE: If you want the script to install Solr, then this script needs to be executed as root. http://git-wip-us.apache.org/repos/asf/incubator-ranger/blob/269617d5/security-admin/contrib/solr_for_audit_setup/resources/log4j.properties.j2 ---------------------------------------------------------------------- diff --git a/security-admin/contrib/solr_for_audit_setup/resources/log4j.properties.j2 b/security-admin/contrib/solr_for_audit_setup/resources/log4j.properties.j2 new file mode 100644 index 0000000..67ad656 --- /dev/null +++ b/security-admin/contrib/solr_for_audit_setup/resources/log4j.properties.j2 @@ -0,0 +1,40 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Logging level +solr.log={{SOLR_LOG_FOLDER}} +#log4j.rootLogger=INFO, file, CONSOLE +log4j.rootLogger=INFO, file + +log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender + +log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout +log4j.appender.CONSOLE.layout.ConversionPattern=%-4r [%t] %-5p %c %x [%X{collection} %X{shard} %X{replica} %X{core}] \u2013 %m%n + +#- size rotation with log cleanup. +log4j.appender.file=org.apache.log4j.RollingFileAppender +log4j.appender.file.MaxFileSize=4MB +log4j.appender.file.MaxBackupIndex=9 + +#- File to log to and log format +log4j.appender.file.File=${solr.log}/solr.log +log4j.appender.file.layout=org.apache.log4j.PatternLayout +log4j.appender.file.layout.ConversionPattern=%d{ISO8601} [%t] %-5p [%X{collection} %X{shard} %X{replica} %X{core}] %C (%F:%L) - %m%n + +log4j.logger.org.apache.zookeeper=WARN +log4j.logger.org.apache.hadoop=WARN + +# set to INFO to enable infostream log messages +log4j.logger.org.apache.solr.update.LoggingInfoStream=OFF http://git-wip-us.apache.org/repos/asf/incubator-ranger/blob/269617d5/security-admin/contrib/solr_for_audit_setup/resources/log4j.properties.template ---------------------------------------------------------------------- diff --git a/security-admin/contrib/solr_for_audit_setup/resources/log4j.properties.template b/security-admin/contrib/solr_for_audit_setup/resources/log4j.properties.template deleted file mode 100644 index 61b25ff..0000000 --- a/security-admin/contrib/solr_for_audit_setup/resources/log4j.properties.template +++ /dev/null @@ -1,39 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# Logging level -solr.log=__SOLR_LOG_FOLDER__ -log4j.rootLogger=INFO, file, CONSOLE - -log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender - -log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout -log4j.appender.CONSOLE.layout.ConversionPattern=%-4r [%t] %-5p %c %x [%X{collection} %X{shard} %X{replica} %X{core}] \u2013 %m%n - -#- size rotation with log cleanup. -log4j.appender.file=org.apache.log4j.RollingFileAppender -log4j.appender.file.MaxFileSize=4MB -log4j.appender.file.MaxBackupIndex=9 - -#- File to log to and log format -log4j.appender.file.File=${solr.log}/solr.log -log4j.appender.file.layout=org.apache.log4j.PatternLayout -log4j.appender.file.layout.ConversionPattern=%d{ISO8601} [%t] %-5p [%X{collection} %X{shard} %X{replica} %X{core}] %C (%F:%L) - %m%n - -log4j.logger.org.apache.zookeeper=WARN -log4j.logger.org.apache.hadoop=WARN - -# set to INFO to enable infostream log messages -log4j.logger.org.apache.solr.update.LoggingInfoStream=OFF