asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luo Chen (Code Review)" <do-not-re...@asterixdb.incubator.apache.org>
Subject Change in asterixdb[master]: Add Create Secondary BTree for Correlated Datasets
Date Thu, 08 Jun 2017 23:34:02 GMT
Hello Jenkins,

I'd like you to reexamine a change.  Please visit

    https://asterix-gerrit.ics.uci.edu/1813

to look at the new patch set (#4).

Change subject: Add Create Secondary BTree for Correlated Datasets
......................................................................

Add Create Secondary BTree for Correlated Datasets

Implemented create seconary BTree index for datasets using
correlated merge policy. Instead of creating one component
for the new index, this change creates one component for each
component of the primary index to maintain the correlation.
The current implementation assumes when a secondary index is being
created, the dataset is locked with no modifications.

Change-Id: I2a3435e6720f07bd6a5092d4d9ce42e8d4b7894c
---
M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/scan-delete-btree-correlated-secondary-index-nullable/scan-delete-btree-correlated-secondary-index-nullable.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/scan-delete-btree-correlated-secondary-index-nullable/scan-delete-btree-correlated-secondary-index-nullable.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/scan-delete-btree-correlated-secondary-index-nullable/scan-delete-btree-correlated-secondary-index-nullable.3.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/scan-delete-btree-correlated-secondary-index-nullable/scan-delete-btree-correlated-secondary-index-nullable.4.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.3.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.4.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/scan-insert-btree-correlated-secondary-index-nullable/scan-insert-btree-correlated-secondary-index-nullable.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/scan-insert-btree-correlated-secondary-index-nullable/scan-insert-btree-correlated-secondary-index-nullable.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/scan-insert-btree-correlated-secondary-index-nullable/scan-insert-btree-correlated-secondary-index-nullable.3.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/scan-insert-btree-correlated-secondary-index-nullable/scan-insert-btree-correlated-secondary-index-nullable.4.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/scan-insert-btree-correlated-secondary-index-open/scan-insert-btree-correlated-secondary-index-open.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/scan-insert-btree-correlated-secondary-index-open/scan-insert-btree-correlated-secondary-index-open.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/scan-insert-btree-correlated-secondary-index-open/scan-insert-btree-correlated-secondary-index-open.3.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/scan-insert-btree-correlated-secondary-index-open/scan-insert-btree-correlated-secondary-index-open.4.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/using-correlated-prefix-merge-policy-with-feed/using-correlated-prefix-merge-policy-with-feed.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/using-correlated-prefix-merge-policy-with-feed/using-correlated-prefix-merge-policy-with-feed.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/using-correlated-prefix-merge-policy-with-feed/using-correlated-prefix-merge-policy-with-feed.3.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/dml/using-correlated-prefix-merge-policy-with-feed/using-correlated-prefix-merge-policy-with-feed.4.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/insert-with-correlated-secondary-btree/insert-with-correlated-secondary-btree.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/insert-with-correlated-secondary-btree/insert-with-correlated-secondary-btree.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/insert-with-correlated-secondary-btree/insert-with-correlated-secondary-btree.3.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/insert-with-correlated-secondary-btree/insert-with-correlated-secondary-btree.4.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/temp-dataset/insert-and-scan-dataset-with-correlated-index/insert-and-scan-dataset-with-correlated-index.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/temp-dataset/insert-and-scan-dataset-with-correlated-index/insert-and-scan-dataset-with-correlated-index.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/temp-dataset/insert-and-scan-dataset-with-correlated-index/insert-and-scan-dataset-with-correlated-index.3.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/temp-dataset/insert-and-scan-dataset-with-correlated-index/insert-and-scan-dataset-with-correlated-index.4.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/upsert/primary-correlated-secondary-btree/primary-correlated-secondary-btree.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/upsert/primary-correlated-secondary-btree/primary-correlated-secondary-btree.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/upsert/primary-correlated-secondary-btree/primary-correlated-secondary-btree.3.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/upsert/primary-correlated-secondary-btree/primary-correlated-secondary-btree.4.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-nullable/scan-delete-btree-correlated-secondary-index-nullable.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-nullable/scan-delete-btree-correlated-secondary-index-nullable.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-nullable/scan-delete-btree-correlated-secondary-index-nullable.3.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-nullable/scan-delete-btree-correlated-secondary-index-nullable.4.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.3.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.4.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-insert-btree-correlated-secondary-index-nullable/scan-insert-btree-correlated-secondary-index-nullable.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-insert-btree-correlated-secondary-index-nullable/scan-insert-btree-correlated-secondary-index-nullable.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-insert-btree-correlated-secondary-index-nullable/scan-insert-btree-correlated-secondary-index-nullable.3.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-insert-btree-correlated-secondary-index-nullable/scan-insert-btree-correlated-secondary-index-nullable.4.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-insert-btree-correlated-secondary-index-open/scan-insert-btree-correlated-secondary-index-open.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-insert-btree-correlated-secondary-index-open/scan-insert-btree-correlated-secondary-index-open.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-insert-btree-correlated-secondary-index-open/scan-insert-btree-correlated-secondary-index-open.3.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-insert-btree-correlated-secondary-index-open/scan-insert-btree-correlated-secondary-index-open.4.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/using-correlated-prefix-merge-policy-with-feed/using-correlated-prefix-merge-policy-with-feed.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/using-correlated-prefix-merge-policy-with-feed/using-correlated-prefix-merge-policy-with-feed.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/using-correlated-prefix-merge-policy-with-feed/using-correlated-prefix-merge-policy-with-feed.3.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/using-correlated-prefix-merge-policy-with-feed/using-correlated-prefix-merge-policy-with-feed.5.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/filters/insert-with-correlated-secondary-btree/insert-with-correlated-secondary-btree.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/filters/insert-with-correlated-secondary-btree/insert-with-correlated-secondary-btree.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/filters/insert-with-correlated-secondary-btree/insert-with-correlated-secondary-btree.3.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/filters/insert-with-correlated-secondary-btree/insert-with-correlated-secondary-btree.4.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/temp-dataset/insert-and-scan-dataset-with-correlated-index/insert-and-scan-dataset-with-correlated-index.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/temp-dataset/insert-and-scan-dataset-with-correlated-index/insert-and-scan-dataset-with-correlated-index.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/temp-dataset/insert-and-scan-dataset-with-correlated-index/insert-and-scan-dataset-with-correlated-index.3.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/temp-dataset/insert-and-scan-dataset-with-correlated-index/insert-and-scan-dataset-with-correlated-index.4.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/upsert/primary-correlated-secondary-btree/primary-secondary-btree.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/upsert/primary-correlated-secondary-btree/primary-secondary-btree.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/upsert/primary-correlated-secondary-btree/primary-secondary-btree.3.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/upsert/primary-correlated-secondary-btree/primary-secondary-btree.4.query.sqlpp
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
A asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.1.script.aql
A asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.10.script.aql
A asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.2.ddl.aql
A asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.3.update.aql
A asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.4.txneu.aql
A asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.5.ddl.aql
A asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.6.txnqbc.aql
A asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.7.script.aql
A asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.8.script.aql
A asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.9.txnqar.aql
A asterixdb/asterix-installer/src/test/resources/transactionts/scripts/recover_after_abort/primary_plus_default_correlated_secondary_index/create_and_start.sh
A asterixdb/asterix-installer/src/test/resources/transactionts/scripts/recover_after_abort/primary_plus_default_correlated_secondary_index/kill_cc_and_nc.sh
A asterixdb/asterix-installer/src/test/resources/transactionts/scripts/recover_after_abort/primary_plus_default_correlated_secondary_index/stop_and_delete.sh
A asterixdb/asterix-installer/src/test/resources/transactionts/scripts/recover_after_abort/primary_plus_default_correlated_secondary_index/stop_and_start.sh
M asterixdb/asterix-installer/src/test/resources/transactionts/testsuite.xml
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Dataset.java
A asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryCorrelatedBTreeOperationsHelper.java
A asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryCorrelatedTreeIndexOperationsHelper.java
A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/AbstractLSMSecondaryCreationNodePushable.java
A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/LSMSecondaryCreationBulkLoadNodePushable.java
A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/LSMSecondaryCreationBulkLoadOperatorDescriptor.java
A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/LSMSecondaryCreationTupleProcessorNodePushable.java
A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/LSMSecondaryCreationTupleProcessorOperatorDescriptor.java
A asterixdb/asterix-runtime/src/test/java/org/apache/asterix/runtime/operators/LSMSecondaryCreationTupleProcessorTest.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/pom.xml
A hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/LSMBTreeDiskComponentScanOperatorDescriptor.java
A hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/LSMBTreeDiskComponentScanOperatorNodePushable.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/ExternalBTreeWithBuddy.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTree.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeDiskComponentScanCursor.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/api/ILSMIndex.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/impls/LSMInvertedIndex.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/impls/LSMRTree.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/impls/LSMRTreeWithAntiMatterTuples.java
101 files changed, 4,629 insertions(+), 23 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/13/1813/4
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1813
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2a3435e6720f07bd6a5092d4d9ce42e8d4b7894c
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen <cluo8@uci.edu>
Gerrit-Reviewer: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Mime
View raw message