Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id B8291200CC5 for ; Tue, 27 Jun 2017 04:46:42 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id B6C4E160BDA; Tue, 27 Jun 2017 02:46:42 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 61536160BF8 for ; Tue, 27 Jun 2017 04:46:40 +0200 (CEST) Received: (qmail 11300 invoked by uid 500); 27 Jun 2017 02:46:39 -0000 Mailing-List: contact commits-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list commits@lucene.apache.org Received: (qmail 11284 invoked by uid 99); 27 Jun 2017 02:46:39 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Jun 2017 02:46:39 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 8AF6FE1154; Tue, 27 Jun 2017 02:46:38 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: ishan@apache.org To: commits@lucene.apache.org Date: Tue, 27 Jun 2017 02:46:39 -0000 Message-Id: In-Reply-To: <3ddb1ee9499d4e0daf70e514cf4d025c@git.apache.org> References: <3ddb1ee9499d4e0daf70e514cf4d025c@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [2/3] lucene-solr:jira/solr-10272: Merging master into this branch archived-at: Tue, 27 Jun 2017 02:46:42 -0000 Merging master into this branch Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/cf6d8b9d Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/cf6d8b9d Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/cf6d8b9d Branch: refs/heads/jira/solr-10272 Commit: cf6d8b9d3aa208350441e6defa057cb4745310f9 Parents: 407f828 Author: Ishan Chattopadhyaya Authored: Tue Jun 27 08:09:46 2017 +0530 Committer: Ishan Chattopadhyaya Committed: Tue Jun 27 08:09:46 2017 +0530 ---------------------------------------------------------------------- solr/CHANGES.txt | 12 ++--- .../accumulator/FacetingAccumulator.java | 24 ++------- .../solr/analytics/expression/Expression.java | 15 ++---- .../statistics/MinMaxStatsCollector.java | 23 ++------ .../statistics/NumericStatsCollector.java | 8 ++- .../StatsCollectorSupplierFactory.java | 14 ++--- .../analytics/requestFiles/facetSorting.txt | 4 -- .../analytics/requestXMLFiles/facetSorting.xml | 14 ----- .../solr/collection1/conf/schema-analytics.xml | 10 ++-- .../collection1/conf/solrconfig-analytics.xml | 40 -------------- .../solr/collection1/conf/solrconfig-basic.xml | 40 ++++++++++++++ .../analytics/AbstractAnalyticsStatsTest.java | 1 + .../org/apache/solr/analytics/NoFacetTest.java | 3 +- .../analytics/expression/ExpressionTest.java | 2 +- .../facet/AbstractAnalyticsFacetTest.java | 16 +----- .../solr/analytics/facet/FacetSortingTest.java | 56 -------------------- .../analytics/facet/FieldFacetExtrasTest.java | 2 +- .../solr/analytics/facet/FieldFacetTest.java | 21 ++++---- .../solr/analytics/facet/QueryFacetTest.java | 2 +- .../solr/analytics/facet/RangeFacetTest.java | 2 +- .../util/valuesource/FunctionTest.java | 2 +- .../clustering/solr/collection1/conf/schema.xml | 18 +++---- .../conf/dataimport-schema-no-unique-key.xml | 18 +++---- .../solr/collection1/conf/dataimport-schema.xml | 10 ++-- .../conf/dataimport-solr_id-schema.xml | 18 +++---- .../handler/extraction/SolrContentHandler.java | 4 +- .../extraction/solr/collection1/conf/schema.xml | 18 +++---- .../test-files/solr/collection1/conf/schema.xml | 10 ++-- .../uima/solr/collection1/conf/schema.xml | 20 +++---- .../test-files/uima/uima-tokenizers-schema.xml | 20 +++---- .../collection1/conf/schema-replication1.xml | 2 +- .../solrj/solr/collection1/conf/schema-sql.xml | 20 +++---- .../solrj/solr/collection1/conf/schema.xml | 20 +++---- .../solrj/solr/configsets/ml/conf/schema.xml | 20 +++---- .../solr/configsets/shared/conf/schema.xml | 2 +- .../solr/configsets/streaming/conf/schema.xml | 20 +++---- .../solrj/solr/multicore/core0/conf/schema.xml | 2 +- .../solrj/solr/multicore/core1/conf/schema.xml | 2 +- .../client/solrj/io/stream/JDBCStreamTest.java | 2 + .../solr/client/solrj/request/SchemaTest.java | 23 ++++---- 40 files changed, 213 insertions(+), 347 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/CHANGES.txt ---------------------------------------------------------------------- diff --git a/solr/CHANGES.txt b/solr/CHANGES.txt index d83b4ab..76722fe 100644 --- a/solr/CHANGES.txt +++ b/solr/CHANGES.txt @@ -207,6 +207,8 @@ Bug Fixes thus disabling the global check, and replaces it with specific checks where desired via QueryUtils.build(). (yonik) +* SOLR-10948: Fix extraction component to treat DatePointField the same as TrieDateField (hossman) + Optimizations ---------------------- @@ -227,12 +229,6 @@ Optimizations * SOLR-10727: Avoid polluting the filter cache for certain types of faceting (typically ranges) when the base docset is empty. (David Smiley) -* SOLR-9981: Performance improvements and bug fixes for the Analytics component. Performance fix that - stops the reading of ALL lucene segments over and again for each stats collector. The AtomicReaderContext - that refers to the "current " segment is reused. This fix shows an improvement of about 25% in query - time for a dataset of ~10M (=9.8M) records. Given the nature of the fix, the improvement should get - better as the dataset increases. Fix for the NPE during comparison (Houston Putman) - Other Changes ---------------------- * SOLR-10236: Removed FieldType.getNumericType(). Use getNumberType() instead. (Tomás Fernández Löbbe) @@ -338,6 +334,10 @@ Other Changes * SOLR-10823: Add reporting period to SolrMetricReporter base class. (Christine Poerschke) +* SOLR-10807: Randomize Points based numeric field types in (more) test schemas + - SOLR-10946: Randomize the usage of Points based numerics in solrj test schemas (hossman) + - SOLR-10947: Randomize the usage of Points based numerics in contrib test schemas (hossman) + ================== 6.7.0 ================== Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/java/org/apache/solr/analytics/accumulator/FacetingAccumulator.java ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/java/org/apache/solr/analytics/accumulator/FacetingAccumulator.java b/solr/contrib/analytics/src/java/org/apache/solr/analytics/accumulator/FacetingAccumulator.java index 92add72..d8828a6 100644 --- a/solr/contrib/analytics/src/java/org/apache/solr/analytics/accumulator/FacetingAccumulator.java +++ b/solr/contrib/analytics/src/java/org/apache/solr/analytics/accumulator/FacetingAccumulator.java @@ -28,7 +28,7 @@ import java.util.List; import java.util.Map; import java.util.Map.Entry; import java.util.Set; -import java.util.HashMap; +import java.util.TreeMap; import com.google.common.collect.Iterables; import org.apache.lucene.index.LeafReaderContext; @@ -98,7 +98,7 @@ public class FacetingAccumulator extends BasicAccumulator implements FacetValueA List rangeFreqs = request.getRangeFacets(); List queryFreqs = request.getQueryFacets(); - this.fieldFacetExpressions = new HashMap<>(); + this.fieldFacetExpressions = new TreeMap<>(); this.rangeFacetExpressions = new LinkedHashMap<>(rangeFreqs.size()); this.queryFacetExpressions = new LinkedHashMap<>(queryFreqs.size()); this.fieldFacetCollectors = new LinkedHashMap<>(fieldFreqs.size()); @@ -120,8 +120,8 @@ public class FacetingAccumulator extends BasicAccumulator implements FacetValueA final SchemaField ff = fr.getField(); final FieldFacetAccumulator facc = FieldFacetAccumulator.create(searcher, this, ff); facetAccumulators.add(facc); - fieldFacetExpressions.put(freq.getName(), new HashMap() ); - fieldFacetCollectors.put(freq.getName(), new HashMap()); + fieldFacetExpressions.put(freq.getName(), new TreeMap() ); + fieldFacetCollectors.put(freq.getName(), new TreeMap()); } /** * For each range and query facet request add a bucket to the corresponding @@ -299,22 +299,6 @@ public class FacetingAccumulator extends BasicAccumulator implements FacetValueA @Override public int compare(Entry o1, Entry o2) { - - // Handle nulls. Null is treated as an infinitely big number so that in case of ASCENDING sorts, - // Nulls will appear last. In case of DESC sorts, Nulls will appear last. - boolean firstIsNull = false; - if (o1 == null || o1.getValue() == null || o1.getValue()[comparatorExpressionPlace] == null) - firstIsNull = true; - boolean secondIsNull = false; - if (o2 == null || o2.getValue() == null || o2.getValue()[comparatorExpressionPlace] == null) - secondIsNull = true; - if (firstIsNull && secondIsNull) - return 0; - else if (firstIsNull) - return 1; - else if (secondIsNull) - return -1; - return comp.compare(o1.getValue()[comparatorExpressionPlace], o2.getValue()[comparatorExpressionPlace]); } } http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/java/org/apache/solr/analytics/expression/Expression.java ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/java/org/apache/solr/analytics/expression/Expression.java b/solr/contrib/analytics/src/java/org/apache/solr/analytics/expression/Expression.java index c0c3da3..ba26d9a 100644 --- a/solr/contrib/analytics/src/java/org/apache/solr/analytics/expression/Expression.java +++ b/solr/contrib/analytics/src/java/org/apache/solr/analytics/expression/Expression.java @@ -29,19 +29,10 @@ public abstract class Expression { public Comparator comparator(final FacetSortDirection direction) { return (a, b) -> { - boolean aIsNull = a.getValue() == null; - boolean bIsNull = b.getValue() == null; - - if (aIsNull && bIsNull) return 0; - - if( direction == FacetSortDirection.ASCENDING ){ // nulls are last for ASC sort - return aIsNull ? 1 - : bIsNull ? -1 - : a.getValue().compareTo(b.getValue()); + if( direction == FacetSortDirection.ASCENDING ){ + return a.getValue().compareTo(b.getValue()); } else { - return aIsNull ? -1 - : bIsNull ? 1 - : b.getValue().compareTo(a.getValue()); + return b.getValue().compareTo(a.getValue()); } }; } http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/java/org/apache/solr/analytics/statistics/MinMaxStatsCollector.java ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/java/org/apache/solr/analytics/statistics/MinMaxStatsCollector.java b/solr/contrib/analytics/src/java/org/apache/solr/analytics/statistics/MinMaxStatsCollector.java index f41a032..c21b045 100644 --- a/solr/contrib/analytics/src/java/org/apache/solr/analytics/statistics/MinMaxStatsCollector.java +++ b/solr/contrib/analytics/src/java/org/apache/solr/analytics/statistics/MinMaxStatsCollector.java @@ -37,32 +37,19 @@ public class MinMaxStatsCollector implements StatsCollector{ protected MutableValue value; protected final Set statsList; protected final ValueSource source; + protected FunctionValues function; protected ValueFiller valueFiller; - private CollectorState state; - public MinMaxStatsCollector(ValueSource source, Set statsList, CollectorState state) { + public MinMaxStatsCollector(ValueSource source, Set statsList) { this.source = source; this.statsList = statsList; - this.state = state; } public void setNextReader(LeafReaderContext context) throws IOException { - state.setNextReader(source, context); - valueFiller = state.function.getValueFiller(); + function = source.getValues(null, context); + valueFiller = function.getValueFiller(); value = valueFiller.getValue(); } - - public static class CollectorState { - FunctionValues function; - LeafReaderContext context = null; - - public void setNextReader(ValueSource source, LeafReaderContext context) throws IOException { - if (this.context != context) { - this.context = context; - this.function = source.getValues(null, context); - } - } - } public void collect(int doc) throws IOException { valueFiller.fillValue(doc); @@ -114,7 +101,7 @@ public class MinMaxStatsCollector implements StatsCollector{ @Override public FunctionValues getFunction() { - return state.function; + return function; } public String valueSourceString() { http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/java/org/apache/solr/analytics/statistics/NumericStatsCollector.java ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/java/org/apache/solr/analytics/statistics/NumericStatsCollector.java b/solr/contrib/analytics/src/java/org/apache/solr/analytics/statistics/NumericStatsCollector.java index 4b1ca22..1f22baa 100644 --- a/solr/contrib/analytics/src/java/org/apache/solr/analytics/statistics/NumericStatsCollector.java +++ b/solr/contrib/analytics/src/java/org/apache/solr/analytics/statistics/NumericStatsCollector.java @@ -29,16 +29,14 @@ public class NumericStatsCollector extends MinMaxStatsCollector { protected double sumOfSquares = 0; protected double mean = 0; protected double stddev = 0; - protected CollectorState state; - public NumericStatsCollector(ValueSource source, Set statsList, CollectorState state) { - super(source, statsList, state); - this.state = state; + public NumericStatsCollector(ValueSource source, Set statsList) { + super(source, statsList); } public void collect(int doc) throws IOException { super.collect(doc); - double value = state.function.doubleVal(doc); + double value = function.doubleVal(doc); sum += value; sumOfSquares += (value * value); } http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/java/org/apache/solr/analytics/statistics/StatsCollectorSupplierFactory.java ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/java/org/apache/solr/analytics/statistics/StatsCollectorSupplierFactory.java b/solr/contrib/analytics/src/java/org/apache/solr/analytics/statistics/StatsCollectorSupplierFactory.java index c7c91d4..e22362d 100644 --- a/solr/contrib/analytics/src/java/org/apache/solr/analytics/statistics/StatsCollectorSupplierFactory.java +++ b/solr/contrib/analytics/src/java/org/apache/solr/analytics/statistics/StatsCollectorSupplierFactory.java @@ -33,7 +33,6 @@ import org.apache.lucene.queries.function.valuesource.IntFieldSource; import org.apache.lucene.queries.function.valuesource.LongFieldSource; import org.apache.solr.analytics.expression.ExpressionFactory; import org.apache.solr.analytics.request.ExpressionRequest; -import org.apache.solr.analytics.statistics.MinMaxStatsCollector.CollectorState; import org.apache.solr.analytics.util.AnalyticsParams; import org.apache.solr.analytics.util.valuesource.AbsoluteValueDoubleFunction; import org.apache.solr.analytics.util.valuesource.AddDoubleFunction; @@ -214,32 +213,25 @@ public class StatsCollectorSupplierFactory { } } } - - final CollectorState states[] = new CollectorState[statsArr.length]; - for (int count = 0; count < statsArr.length; count++) { - states[count] = new CollectorState(); - } // Making the Supplier return new Supplier() { - private final CollectorState collectorState[] = states; - public StatsCollector[] get() { StatsCollector[] collectors = new StatsCollector[statsArr.length]; for (int count = 0; count < statsArr.length; count++) { if(numericBools[count]){ - StatsCollector sc = new NumericStatsCollector(sourceArr[count], statsArr[count], collectorState[count]); + StatsCollector sc = new NumericStatsCollector(sourceArr[count], statsArr[count]); if(uniqueBools[count]) sc = new UniqueStatsCollector(sc); if(medianBools[count]) sc = new MedianStatsCollector(sc); if(percsArr[count]!=null) sc = new PercentileStatsCollector(sc,percsArr[count],percsNames[count]); collectors[count]=sc; } else if (dateBools[count]) { - StatsCollector sc = new MinMaxStatsCollector(sourceArr[count], statsArr[count], collectorState[count]); + StatsCollector sc = new MinMaxStatsCollector(sourceArr[count], statsArr[count]); if(uniqueBools[count]) sc = new UniqueStatsCollector(sc); if(medianBools[count]) sc = new DateMedianStatsCollector(sc); if(percsArr[count]!=null) sc = new PercentileStatsCollector(sc,percsArr[count],percsNames[count]); collectors[count]=sc; } else { - StatsCollector sc = new MinMaxStatsCollector(sourceArr[count], statsArr[count], collectorState[count]); + StatsCollector sc = new MinMaxStatsCollector(sourceArr[count], statsArr[count]); if(uniqueBools[count]) sc = new UniqueStatsCollector(sc); if(medianBools[count]) sc = new MedianStatsCollector(sc); if(percsArr[count]!=null) sc = new PercentileStatsCollector(sc,percsArr[count],percsNames[count]); http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/test-files/analytics/requestFiles/facetSorting.txt ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/test-files/analytics/requestFiles/facetSorting.txt b/solr/contrib/analytics/src/test-files/analytics/requestFiles/facetSorting.txt deleted file mode 100644 index 4663217..0000000 --- a/solr/contrib/analytics/src/test-files/analytics/requestFiles/facetSorting.txt +++ /dev/null @@ -1,4 +0,0 @@ -o.ar.s.min=min(double_dd) -o.ar.s.max=max(long_ld) -o.ar.ff=string_sd -o.ar.ff.string_sd.sortstatistic=min http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/test-files/analytics/requestXMLFiles/facetSorting.xml ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/test-files/analytics/requestXMLFiles/facetSorting.xml b/solr/contrib/analytics/src/test-files/analytics/requestXMLFiles/facetSorting.xml deleted file mode 100644 index 54e698a..0000000 --- a/solr/contrib/analytics/src/test-files/analytics/requestXMLFiles/facetSorting.xml +++ /dev/null @@ -1,14 +0,0 @@ - - - - MinMax Request - - min(double(double_dd)) - min - - - max(long(long_ld)) - max - - - http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/test-files/solr/collection1/conf/schema-analytics.xml ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/test-files/solr/collection1/conf/schema-analytics.xml b/solr/contrib/analytics/src/test-files/solr/collection1/conf/schema-analytics.xml index 2d90359..04b22b1 100644 --- a/solr/contrib/analytics/src/test-files/solr/collection1/conf/schema-analytics.xml +++ b/solr/contrib/analytics/src/test-files/solr/collection1/conf/schema-analytics.xml @@ -39,14 +39,14 @@ These are provided more for backward compatability, allowing one to create a schema that matches an existing lucene index. --> - - - - + + + + - + http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/test-files/solr/collection1/conf/solrconfig-analytics.xml ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/test-files/solr/collection1/conf/solrconfig-analytics.xml b/solr/contrib/analytics/src/test-files/solr/collection1/conf/solrconfig-analytics.xml deleted file mode 100644 index 604bb17..0000000 --- a/solr/contrib/analytics/src/test-files/solr/collection1/conf/solrconfig-analytics.xml +++ /dev/null @@ -1,40 +0,0 @@ - - - - - - ${tests.luceneMatchVersion:LATEST} - ${solr.data.dir:} - - - - - - - query - facet - analytics - highlight - debug - expand - - - - - - http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/test-files/solr/collection1/conf/solrconfig-basic.xml ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/test-files/solr/collection1/conf/solrconfig-basic.xml b/solr/contrib/analytics/src/test-files/solr/collection1/conf/solrconfig-basic.xml new file mode 100644 index 0000000..604bb17 --- /dev/null +++ b/solr/contrib/analytics/src/test-files/solr/collection1/conf/solrconfig-basic.xml @@ -0,0 +1,40 @@ + + + + + + ${tests.luceneMatchVersion:LATEST} + ${solr.data.dir:} + + + + + + + query + facet + analytics + highlight + debug + expand + + + + + + http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/test/org/apache/solr/analytics/AbstractAnalyticsStatsTest.java ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/test/org/apache/solr/analytics/AbstractAnalyticsStatsTest.java b/solr/contrib/analytics/src/test/org/apache/solr/analytics/AbstractAnalyticsStatsTest.java index 881822c..c3d2f28 100644 --- a/solr/contrib/analytics/src/test/org/apache/solr/analytics/AbstractAnalyticsStatsTest.java +++ b/solr/contrib/analytics/src/test/org/apache/solr/analytics/AbstractAnalyticsStatsTest.java @@ -48,6 +48,7 @@ import org.xml.sax.SAXException; import com.google.common.collect.ObjectArrays; +@SolrTestCaseJ4.SuppressPointFields(bugUrl="https://issues.apache.org/jira/browse/SOLR-10949") public class AbstractAnalyticsStatsTest extends SolrTestCaseJ4 { protected static final String[] BASEPARMS = new String[]{ "q", "*:*", "indent", "true", "olap", "true", "rows", "0" }; http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/test/org/apache/solr/analytics/NoFacetTest.java ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/test/org/apache/solr/analytics/NoFacetTest.java b/solr/contrib/analytics/src/test/org/apache/solr/analytics/NoFacetTest.java index 7b67d00..f4d6275 100644 --- a/solr/contrib/analytics/src/test/org/apache/solr/analytics/NoFacetTest.java +++ b/solr/contrib/analytics/src/test/org/apache/solr/analytics/NoFacetTest.java @@ -16,7 +16,6 @@ */ package org.apache.solr.analytics; - import java.util.ArrayList; import java.util.List; @@ -60,7 +59,7 @@ public class NoFacetTest extends AbstractAnalyticsStatsTest { @BeforeClass public static void beforeClass() throws Exception { - initCore("solrconfig-analytics.xml","schema-analytics.xml"); + initCore("solrconfig-basic.xml","schema-analytics.xml"); h.update("*:*"); defaults.put("int_id", new Integer(0)); defaults.put("long_ld", new Long(0)); http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/test/org/apache/solr/analytics/expression/ExpressionTest.java ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/test/org/apache/solr/analytics/expression/ExpressionTest.java b/solr/contrib/analytics/src/test/org/apache/solr/analytics/expression/ExpressionTest.java index 245ea1c..4a3276b 100644 --- a/solr/contrib/analytics/src/test/org/apache/solr/analytics/expression/ExpressionTest.java +++ b/solr/contrib/analytics/src/test/org/apache/solr/analytics/expression/ExpressionTest.java @@ -48,7 +48,7 @@ public class ExpressionTest extends AbstractAnalyticsStatsTest { @BeforeClass public static void beforeClass() throws Exception { - initCore("solrconfig-analytics.xml", "schema-analytics.xml"); + initCore("solrconfig-basic.xml", "schema-analytics.xml"); h.update("*:*"); for (int j = 0; j < NUM_LOOPS; ++j) { http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/AbstractAnalyticsFacetTest.java ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/AbstractAnalyticsFacetTest.java b/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/AbstractAnalyticsFacetTest.java index dd331fd..3660208 100644 --- a/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/AbstractAnalyticsFacetTest.java +++ b/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/AbstractAnalyticsFacetTest.java @@ -52,6 +52,7 @@ import javax.xml.xpath.XPathConstants; import javax.xml.xpath.XPathExpressionException; import javax.xml.xpath.XPathFactory; +@SolrTestCaseJ4.SuppressPointFields(bugUrl="https://issues.apache.org/jira/browse/SOLR-10949") public class AbstractAnalyticsFacetTest extends SolrTestCaseJ4 { protected static final HashMap defaults = new HashMap<>(); @@ -312,19 +313,4 @@ public class AbstractAnalyticsFacetTest extends SolrTestCaseJ4 { IOUtils.closeWhileHandlingException(file, in); } } - - protected void removeNodes(String xPath, List string) throws XPathExpressionException { - NodeList missingNodes = getNodes(xPath); - List result = new ArrayList(); - for (int idx = 0; idx < missingNodes.getLength(); ++idx) { - result.add(Double.parseDouble(missingNodes.item(idx).getTextContent())); - } - string.removeAll(result); - } - - protected NodeList getNodes(String xPath) throws XPathExpressionException { - StringBuilder sb = new StringBuilder(xPath); - return (NodeList) xPathFact.newXPath().compile(sb.toString()).evaluate(doc, XPathConstants.NODESET); - } - } http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/FacetSortingTest.java ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/FacetSortingTest.java b/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/FacetSortingTest.java deleted file mode 100644 index d91aa18..0000000 --- a/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/FacetSortingTest.java +++ /dev/null @@ -1,56 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.solr.analytics.facet; - -import org.apache.lucene.util.LuceneTestCase.SuppressCodecs; -import org.apache.solr.analytics.AbstractAnalyticsStatsTest; -import org.apache.solr.analytics.expression.ExpressionTest; -import org.junit.BeforeClass; -import org.junit.Test; - -@SuppressCodecs({"Lucene3x","Lucene40","Lucene41","Lucene42","Appending","Asserting"}) -public class FacetSortingTest extends AbstractAnalyticsStatsTest { - private static String fileName = "/analytics/requestFiles/facetSorting.txt"; - - @BeforeClass - public static void beforeClass() throws Exception { - initCore("solrconfig-analytics.xml", "schema-analytics.xml"); - h.update("*:*"); - - // The data set below is so generated that in bucket corresponding fieldFacet B, double_dd column has null values - // and in bucket C corresponding to fieldFacet C has null values for column long_ld. - // FieldFaceting occurs on string_sd field - assertU(adoc("id", "1001", "string_sd", "A", "double_dd", "" + 3, "long_ld", "" + 1)); - assertU(adoc("id", "1002", "string_sd", "A", "double_dd", "" + 25, "long_ld", "" + 2)); - assertU(adoc("id", "1003", "string_sd", "B", "long_ld", "" + 3)); - assertU(adoc("id", "1004", "string_sd", "B", "long_ld", "" + 4)); - assertU(adoc("id", "1005", "string_sd", "C", "double_dd", "" + 17)); - - assertU(commit()); - String response = h.query(request(fileToStringArr(ExpressionTest.class, fileName))); - System.out.println("Response=" + response); - setResponse(response); - } - - @Test - public void addTest() throws Exception { - Double minResult = (Double) getStatResult("ar", "min", VAL_TYPE.DOUBLE); - Long maxResult = (Long) getStatResult("ar", "max", VAL_TYPE.LONG); - assertEquals(Double.valueOf(minResult), Double.valueOf(3.0)); - assertEquals(Long.valueOf(maxResult),Long.valueOf(4)); - } -} http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/FieldFacetExtrasTest.java ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/FieldFacetExtrasTest.java b/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/FieldFacetExtrasTest.java index afa0c3c..fc7df4b 100644 --- a/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/FieldFacetExtrasTest.java +++ b/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/FieldFacetExtrasTest.java @@ -44,7 +44,7 @@ public class FieldFacetExtrasTest extends AbstractAnalyticsFacetTest { @BeforeClass public static void beforeClass() throws Exception { - initCore("solrconfig-analytics.xml","schema-analytics.xml"); + initCore("solrconfig-basic.xml","schema-analytics.xml"); h.update("*:*"); //INT http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/FieldFacetTest.java ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/FieldFacetTest.java b/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/FieldFacetTest.java index 2230b02..bf67213 100644 --- a/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/FieldFacetTest.java +++ b/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/FieldFacetTest.java @@ -24,7 +24,6 @@ import java.util.List; import org.junit.Assert; import org.junit.BeforeClass; import org.junit.Test; -import org.w3c.dom.Node; public class FieldFacetTest extends AbstractAnalyticsFacetTest{ @@ -88,7 +87,7 @@ public class FieldFacetTest extends AbstractAnalyticsFacetTest{ @BeforeClass public static void beforeClass() throws Exception { - initCore("solrconfig-analytics.xml","schema-analytics.xml"); + initCore("solrconfig-basic.xml","schema-analytics.xml"); h.update("*:*"); defaults.put("int", new Integer(0)); @@ -1038,33 +1037,31 @@ public class FieldFacetTest extends AbstractAnalyticsFacetTest{ public void missingFacetTest() throws Exception { //int MultiDate String xPath = "/response/lst[@name='stats']/lst[@name='missingf']/lst[@name='fieldFacets']/lst[@name='date_dtdm']/lst[@name='(MISSING)']"; - Node missingNodeXPath = getNode(xPath); - assertNotNull(getRawResponse(), missingNodeXPath); + assertNotNull(getRawResponse(), getNode(xPath)); ArrayList string = getDoubleList("missingf", "fieldFacets", "date_dtdm", "double", "mean"); - super.removeNodes(xPath, string); + string.remove(0); ArrayList stringTest = calculateNumberStat(multiDateTestStart, "mean"); assertEquals(getRawResponse(), string,stringTest); - + //Int String xPath = "/response/lst[@name='stats']/lst[@name='missingf']/lst[@name='fieldFacets']/lst[@name='string_sd']/lst[@name='(MISSING)']"; - missingNodeXPath = getNode(xPath); - String missingNodeXPathStr = xPath; - assertNotNull(getRawResponse(), missingNodeXPath); + assertNotNull(getRawResponse(), getNode(xPath)); xPath = "/response/lst[@name='stats']/lst[@name='missingf']/lst[@name='fieldFacets']/lst[@name='string_sd']/lst[@name='str0']"; assertNull(getRawResponse(), getNode(xPath)); - List intString = getDoubleList("missingf", "fieldFacets", "string_sd", "double", "mean"); - removeNodes(missingNodeXPathStr, intString); + intString.remove(0); ArrayList intStringTest = calculateNumberStat(intStringTestStart, "mean"); assertEquals(getRawResponse(), intString,intStringTest); - + //Int Date Collection intDate = getDoubleList("missingf", "fieldFacets", "date_dtd", "double", "mean"); ArrayList> intDateMissingTestStart = (ArrayList>) intDateTestStart.clone(); ArrayList intDateTest = calculateNumberStat(intDateMissingTestStart, "mean"); assertEquals(getRawResponse(),intDate,intDateTest); + + } private void checkStddevs(ArrayList list1, ArrayList list2) { http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/QueryFacetTest.java ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/QueryFacetTest.java b/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/QueryFacetTest.java index 99ce02d..f62a82b 100644 --- a/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/QueryFacetTest.java +++ b/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/QueryFacetTest.java @@ -35,7 +35,7 @@ public class QueryFacetTest extends AbstractAnalyticsFacetTest { @BeforeClass public static void beforeClass() throws Exception { - initCore("solrconfig-analytics.xml","schema-analytics.xml"); + initCore("solrconfig-basic.xml","schema-analytics.xml"); } @SuppressWarnings("unchecked") http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/RangeFacetTest.java ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/RangeFacetTest.java b/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/RangeFacetTest.java index cf5e36b..cda8202 100644 --- a/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/RangeFacetTest.java +++ b/solr/contrib/analytics/src/test/org/apache/solr/analytics/facet/RangeFacetTest.java @@ -46,7 +46,7 @@ public class RangeFacetTest extends AbstractAnalyticsFacetTest { @BeforeClass public static void beforeClass() throws Exception { - initCore("solrconfig-analytics.xml","schema-analytics.xml"); + initCore("solrconfig-basic.xml","schema-analytics.xml"); h.update("*:*"); //INT http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/analytics/src/test/org/apache/solr/analytics/util/valuesource/FunctionTest.java ---------------------------------------------------------------------- diff --git a/solr/contrib/analytics/src/test/org/apache/solr/analytics/util/valuesource/FunctionTest.java b/solr/contrib/analytics/src/test/org/apache/solr/analytics/util/valuesource/FunctionTest.java index f9a9620..5981cc0 100644 --- a/solr/contrib/analytics/src/test/org/apache/solr/analytics/util/valuesource/FunctionTest.java +++ b/solr/contrib/analytics/src/test/org/apache/solr/analytics/util/valuesource/FunctionTest.java @@ -35,7 +35,7 @@ public class FunctionTest extends AbstractAnalyticsStatsTest { @BeforeClass public static void beforeClass() throws Exception { - initCore("solrconfig-analytics.xml","schema-analytics.xml"); + initCore("solrconfig-basic.xml","schema-analytics.xml"); h.update("*:*"); for (int j = 0; j < NUM_LOOPS; ++j) { http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/contrib/clustering/src/test-files/clustering/solr/collection1/conf/schema.xml ---------------------------------------------------------------------- diff --git a/solr/contrib/clustering/src/test-files/clustering/solr/collection1/conf/schema.xml b/solr/contrib/clustering/src/test-files/clustering/solr/collection1/conf/schema.xml index 66a6223..e14c607 100644 --- a/solr/contrib/clustering/src/test-files/clustering/solr/collection1/conf/schema.xml +++ b/solr/contrib/clustering/src/test-files/clustering/solr/collection1/conf/schema.xml @@ -72,10 +72,10 @@ - - - - + + + + - - - - + + + + - + - - - - + + + + @@ -90,10 +90,10 @@ indexed per value, slightly larger index size, and faster range queries. A precisionStep of 0 disables indexing at different precision levels. --> - - - - + + + + - + - - - - + + + + - - - - + + + + - + - - - - + + + + - - - - + + + + @@ -105,7 +105,7 @@ - + - - - - - - - - - - - - - - - - - - - - - - - - + + - - - - + + + + - - + + - - - - + + + + - - - - + + + + @@ -94,8 +94,8 @@ - - + + - - + + http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/solrj/src/test-files/solrj/solr/configsets/shared/conf/schema.xml ---------------------------------------------------------------------- diff --git a/solr/solrj/src/test-files/solrj/solr/configsets/shared/conf/schema.xml b/solr/solrj/src/test-files/solrj/solr/configsets/shared/conf/schema.xml index 15fc982..a3559b9 100644 --- a/solr/solrj/src/test-files/solrj/solr/configsets/shared/conf/schema.xml +++ b/solr/solrj/src/test-files/solrj/solr/configsets/shared/conf/schema.xml @@ -46,7 +46,7 @@ - + http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/solrj/src/test-files/solrj/solr/configsets/streaming/conf/schema.xml ---------------------------------------------------------------------- diff --git a/solr/solrj/src/test-files/solrj/solr/configsets/streaming/conf/schema.xml b/solr/solrj/src/test-files/solrj/solr/configsets/streaming/conf/schema.xml index f17da41..1a3b96f 100644 --- a/solr/solrj/src/test-files/solrj/solr/configsets/streaming/conf/schema.xml +++ b/solr/solrj/src/test-files/solrj/solr/configsets/streaming/conf/schema.xml @@ -40,15 +40,15 @@ --> - - - - + + + + - - - - + + + + @@ -115,8 +115,8 @@ - - + + http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/solrj/src/test-files/solrj/solr/multicore/core1/conf/schema.xml ---------------------------------------------------------------------- diff --git a/solr/solrj/src/test-files/solrj/solr/multicore/core1/conf/schema.xml b/solr/solrj/src/test-files/solrj/solr/multicore/core1/conf/schema.xml index 260b106..9f4cde8 100644 --- a/solr/solrj/src/test-files/solrj/solr/multicore/core1/conf/schema.xml +++ b/solr/solrj/src/test-files/solrj/solr/multicore/core1/conf/schema.xml @@ -19,7 +19,7 @@ - + http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/cf6d8b9d/solr/solrj/src/test/org/apache/solr/client/solrj/io/stream/JDBCStreamTest.java ---------------------------------------------------------------------- diff --git a/solr/solrj/src/test/org/apache/solr/client/solrj/io/stream/JDBCStreamTest.java b/solr/solrj/src/test/org/apache/solr/client/solrj/io/stream/JDBCStreamTest.java index 8813f24..501c6dd 100644 --- a/solr/solrj/src/test/org/apache/solr/client/solrj/io/stream/JDBCStreamTest.java +++ b/solr/solrj/src/test/org/apache/solr/client/solrj/io/stream/JDBCStreamTest.java @@ -41,6 +41,7 @@ import org.apache.solr.client.solrj.request.CollectionAdminRequest; import org.apache.solr.client.solrj.request.UpdateRequest; import org.apache.solr.cloud.AbstractDistribZkTestBase; import org.apache.solr.cloud.SolrCloudTestCase; +import org.apache.solr.SolrTestCaseJ4.SuppressPointFields; import org.junit.AfterClass; import org.junit.Before; import org.junit.BeforeClass; @@ -49,6 +50,7 @@ import org.junit.Test; /** */ +@SuppressPointFields(bugUrl="https://issues.apache.org/jira/browse/SOLR-10960") @LuceneTestCase.SuppressCodecs({"Lucene3x", "Lucene40","Lucene41","Lucene42","Lucene45"}) public class JDBCStreamTest extends SolrCloudTestCase {