asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ian Maxon (Code Review)" <do-not-re...@asterixdb.incubator.apache.org>
Subject Change in asterixdb[master]: ASTERIXDB-1905: Incorrect filter for post-load sidx
Date Tue, 16 May 2017 18:58:30 GMT
Ian Maxon has submitted this change and it was merged.

Change subject: ASTERIXDB-1905: Incorrect filter for post-load sidx
......................................................................


ASTERIXDB-1905: Incorrect filter for post-load sidx

The issue seems to be limited to RTrees. The MBR evaluators
were not accounting for the point MBR optimization as they
were in the compiled load pidx+sidx case

Change-Id: Iea158ad4c29ad4421020a28a72e68637bc538560
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1743
Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Jianfeng Jia <jianfeng.jia@gmail.com>
---
A asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/load-with-secondary-rtree/load-with-secondary-rtree.6.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/filters/load-with-secondary-rtree/load-with-secondary-rtree.6.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/results/filters/load-with-secondary-rtree/load-with-secondary-rtree.2.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/filters/load-with-secondary-rtree/load-with-secondary-rtree.6.ast
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryRTreeOperationsHelper.java
M asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/base/IDataFormat.java
M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/formats/NonTaggedDataFormat.java
7 files changed, 107 insertions(+), 4 deletions(-)

Approvals:
  Jianfeng Jia: Looks good to me, approved
  Jenkins: Verified; No violations found; No violations found; Verified



diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/load-with-secondary-rtree/load-with-secondary-rtree.6.query.aql
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/load-with-secondary-rtree/load-with-secondary-rtree.6.query.aql
new file mode 100644
index 0000000..0a02b14
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/load-with-secondary-rtree/load-with-secondary-rtree.6.query.aql
@@ -0,0 +1,27 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+use dataverse test;
+
+for $m in dataset('FacebookMessages')
+where spatial-intersect($m.sender-location, create-polygon([40.0,79.87,30.0,75.0,50.0,80.0,10.0,10.0]))
+and $m.send-time > datetime("2012-11-20T10:10:00.000Z")
+order by $m.send-time
+return $m
+
+
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/filters/load-with-secondary-rtree/load-with-secondary-rtree.6.query.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/filters/load-with-secondary-rtree/load-with-secondary-rtree.6.query.sqlpp
new file mode 100644
index 0000000..c082af4
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/filters/load-with-secondary-rtree/load-with-secondary-rtree.6.query.sqlpp
@@ -0,0 +1,26 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+use test;
+
+select element m
+from  FacebookMessages as m
+where (test.`spatial-intersect`(m.`sender-location`,test.`create-polygon`([40.0,79.87,30.0,75.0,50.0,80.0,10.0,10.0]))
and (m.`send-time` > test.datetime('2012-11-20T10:10:00.000Z')))
+order by m.`send-time`
+;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/filters/load-with-secondary-rtree/load-with-secondary-rtree.2.adm
b/asterixdb/asterix-app/src/test/resources/runtimets/results/filters/load-with-secondary-rtree/load-with-secondary-rtree.2.adm
new file mode 100644
index 0000000..f87754d
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/filters/load-with-secondary-rtree/load-with-secondary-rtree.2.adm
@@ -0,0 +1 @@
+{ "message-id": 12, "author-id": 10, "in-response-to": 6, "sender-location": point("42.26,77.76"),
"message": " can't stand t-mobile its voicemail-service is OMG:(", "send-time": datetime("2012-12-20T10:10:00.000Z")
}
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/filters/load-with-secondary-rtree/load-with-secondary-rtree.6.ast
b/asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/filters/load-with-secondary-rtree/load-with-secondary-rtree.6.ast
new file mode 100644
index 0000000..1d51f60
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/filters/load-with-secondary-rtree/load-with-secondary-rtree.6.ast
@@ -0,0 +1,49 @@
+DataverseUse test
+Query:
+SELECT ELEMENT [
+Variable [ Name=$m ]
+]
+FROM [  FunctionCall Metadata.dataset@1[
+    LiteralExpr [STRING] [FacebookMessages]
+  ]
+  AS Variable [ Name=$m ]
+]
+Where
+  OperatorExpr [
+    FunctionCall test.spatial-intersect@2[
+      FieldAccessor [
+        Variable [ Name=$m ]
+        Field=sender-location
+      ]
+      FunctionCall test.create-polygon@1[
+        OrderedListConstructor [
+          LiteralExpr [DOUBLE] [40.0]
+          LiteralExpr [DOUBLE] [79.87]
+          LiteralExpr [DOUBLE] [30.0]
+          LiteralExpr [DOUBLE] [75.0]
+          LiteralExpr [DOUBLE] [50.0]
+          LiteralExpr [DOUBLE] [80.0]
+          LiteralExpr [DOUBLE] [10.0]
+          LiteralExpr [DOUBLE] [10.0]
+        ]
+      ]
+    ]
+    and
+    OperatorExpr [
+      FieldAccessor [
+        Variable [ Name=$m ]
+        Field=send-time
+      ]
+      >
+      FunctionCall test.datetime@1[
+        LiteralExpr [STRING] [2012-11-20T10:10:00.000Z]
+      ]
+    ]
+  ]
+Orderby
+  FieldAccessor [
+    Variable [ Name=$m ]
+    Field=send-time
+  ]
+  ASC
+
diff --git a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryRTreeOperationsHelper.java
b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryRTreeOperationsHelper.java
index 33affe9..21fa754 100644
--- a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryRTreeOperationsHelper.java
+++ b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryRTreeOperationsHelper.java
@@ -103,7 +103,7 @@
         int recordColumn = dataset.getDatasetType() == DatasetType.INTERNAL ? numPrimaryKeys
: 0;
         secondaryFieldAccessEvalFactories =
                 metadataProvider.getFormat().createMBRFactory(isEnforcingKeyTypes ? enforcedItemType
: itemType,
-                        secondaryKeyFields.get(0), recordColumn, numDimensions, filterFieldName);
+                        secondaryKeyFields.get(0), recordColumn, numDimensions, filterFieldName,
isPointMBR);
         secondaryComparatorFactories = new IBinaryComparatorFactory[numNestedSecondaryKeyFields];
         valueProviderFactories = new IPrimitiveValueProviderFactory[numNestedSecondaryKeyFields];
         ISerializerDeserializer[] secondaryRecFields =
diff --git a/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/base/IDataFormat.java
b/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/base/IDataFormat.java
index 3c31f00..ded0eca 100644
--- a/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/base/IDataFormat.java
+++ b/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/base/IDataFormat.java
@@ -82,7 +82,7 @@
     public IScalarEvaluatorFactory getConstantEvalFactory(IAlgebricksConstantValue value)
throws AlgebricksException;
 
     public IScalarEvaluatorFactory[] createMBRFactory(ARecordType recType, List<String>
fldName, int recordColumn,
-            int dimension, List<String> filterFieldName) throws AlgebricksException;
+            int dimension, List<String> filterFieldName, boolean isPointMBR) throws
AlgebricksException;
 
     public IExpressionEvalSizeComputer getExpressionEvalSizeComputer();
 
diff --git a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/formats/NonTaggedDataFormat.java
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/formats/NonTaggedDataFormat.java
index e3ab7b3..b0e1788 100644
--- a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/formats/NonTaggedDataFormat.java
+++ b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/formats/NonTaggedDataFormat.java
@@ -242,9 +242,9 @@
     @SuppressWarnings("unchecked")
     @Override
     public IScalarEvaluatorFactory[] createMBRFactory(ARecordType recType, List<String>
fldName, int recordColumn,
-            int dimension, List<String> filterFieldName) throws AlgebricksException
{
+            int dimension, List<String> filterFieldName, boolean isPointMBR) throws
AlgebricksException {
         IScalarEvaluatorFactory evalFactory = getFieldAccessEvaluatorFactory(recType, fldName,
recordColumn);
-        int numOfFields = dimension * 2;
+        int numOfFields = isPointMBR ? dimension : dimension * 2;
         IScalarEvaluatorFactory[] evalFactories =
                 new IScalarEvaluatorFactory[numOfFields + (filterFieldName == null ? 0 :
1)];
 

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1743
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Iea158ad4c29ad4421020a28a72e68637bc538560
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon <imaxon@apache.org>
Gerrit-Reviewer: Ian Maxon <imaxon@apache.org>
Gerrit-Reviewer: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Jianfeng Jia <jianfeng.jia@gmail.com>

Mime
View raw message