asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yingyi Bu (Code Review)" <do-not-re...@asterixdb.incubator.apache.org>
Subject Change in asterixdb[master]: Remove default node group.
Date Sun, 04 Jun 2017 02:03:47 GMT
Yingyi Bu has submitted this change and it was merged.

Change subject: Remove default node group.
......................................................................


Remove default node group.

In this way, CREATE DATASET statement can adjust to dynamic
cluster topology.

When we create a dataset:
- if the node group name is not given, we create a new node group
  using all currently available nodes;
- if the node group name is give, we use the given node group for
  the dataset.

When we drop a dataset:
- if no other dataset depends on the node group of the dataset to
  be dropped, we also drop the node group.

Change-Id: If68dc6a7c1270ab1f5049c9334e3318425fd8287
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1799
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Michael Blow <mblow@apache.org>
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/AbstractLangTranslator.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/RebalanceUtil.java
M asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-1/dataset_with_meta-1.1.adm
M asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-2/dataset_with_meta-2.1.adm
M asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-3/dataset_with_meta-2.3.adm
M asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-4/dataset_with_meta-4.3.adm
M asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-5/dataset_with_meta-5.3.adm
M asterixdb/asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_2/issue_251_dataset_hint_2.1.adm
M asterixdb/asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_3/issue_251_dataset_hint_3.1.adm
M asterixdb/asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_4/issue_251_dataset_hint_4.1.adm
M asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta02/meta02.1.adm
M asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta08/meta08.1.adm
M asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta09/meta09.1.adm
M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_nodegroup/metadata_nodegroup.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/queries/index-selection/intersection_with_nodegroup/intersection_with_nodegroup.4.ddl.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/index-selection/multi-index/multi-index.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset_nodegroup/dataset_nodegroup.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset_nodegroup/dataset_nodegroup.2.query.sqlpp
M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/rebalance/duplicate_location/duplicate_location.5.query.sqlpp
M asterixdb/asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv02/cross-dv02.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv04/cross-dv04.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv19/cross-dv19.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/dataset_nodegroup/dataset_nodegroup.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.14.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.15.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.8.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.9.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/duplicate_location/duplicate_location.4.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/duplicate_location/duplicate_location.5.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/empty_location/empty_location.3.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/identical_location/identical_location.4.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset/single_dataset.11.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset/single_dataset.7.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset_with_index/single_dataset_with_index.12.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset_with_index/single_dataset_with_index.7.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.14.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.15.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.8.adm
M asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.9.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/statement/DatasetDecl.java
M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/visitor/FormatPrintVisitor.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataManager.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataNode.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IMetadataManager.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IMetadataNode.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Dataset.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/lock/MetadataLockManager.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/DatasetUtil.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/MetadataConstants.java
52 files changed, 244 insertions(+), 197 deletions(-)

Approvals:
  Michael Blow: Looks good to me, approved
  Jenkins: Verified; No violations found; Verified

Objections:
  Jenkins: Violations found



diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/AbstractLangTranslator.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/AbstractLangTranslator.java
index e2589da..eee6bdc 100644
--- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/AbstractLangTranslator.java
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/AbstractLangTranslator.java
@@ -33,7 +33,6 @@
 import org.apache.asterix.lang.common.statement.DeleteStatement;
 import org.apache.asterix.lang.common.statement.DropDatasetStatement;
 import org.apache.asterix.lang.common.statement.InsertStatement;
-import org.apache.asterix.lang.common.statement.NodeGroupDropStatement;
 import org.apache.asterix.metadata.dataset.hints.DatasetHints;
 import org.apache.asterix.metadata.entities.Dataverse;
 import org.apache.asterix.metadata.utils.MetadataConstants;
@@ -125,14 +124,6 @@
                 if (invalidOperation) {
                     message = "Delete operation is not permitted in dataverse "
                             + MetadataConstants.METADATA_DATAVERSE_NAME;
-                }
-                break;
-
-            case Statement.Kind.NODEGROUP_DROP:
-                String nodegroupName = ((NodeGroupDropStatement) stmt).getNodeGroupName().getValue();
-                invalidOperation = MetadataConstants.METADATA_DEFAULT_NODEGROUP_NAME.equals(nodegroupName);
-                if (invalidOperation) {
-                    message = "Cannot drop nodegroup:" + nodegroupName;
                 }
                 break;
 
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
index 90a07af..64edead 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
@@ -26,14 +26,16 @@
 import java.io.InputStreamReader;
 import java.rmi.RemoteException;
 import java.util.ArrayList;
+import java.util.Collections;
 import java.util.Date;
 import java.util.HashMap;
 import java.util.Iterator;
+import java.util.LinkedHashSet;
 import java.util.List;
 import java.util.Map;
 import java.util.Map.Entry;
 import java.util.Properties;
-import java.util.Random;
+import java.util.Set;
 import java.util.concurrent.ExecutorService;
 import java.util.logging.Level;
 import java.util.logging.Logger;
@@ -151,6 +153,7 @@
 import org.apache.asterix.om.types.ATypeTag;
 import org.apache.asterix.om.types.IAType;
 import org.apache.asterix.om.types.TypeSignature;
+import org.apache.asterix.runtime.utils.ClusterStateManager;
 import org.apache.asterix.transaction.management.service.transaction.DatasetIdFactory;
 import org.apache.asterix.translator.AbstractLangTranslator;
 import org.apache.asterix.translator.CompiledStatements.CompiledDeleteStatement;
@@ -508,7 +511,7 @@
         String metaItemTypeDataverseName = getActiveDataverse(dd.getMetaItemTypeDataverse());
         String metaItemTypeName = dd.getMetaItemTypeName().getValue();
         Identifier ngNameId = dd.getNodegroupName();
-        String nodegroupName = getNodeGroupName(ngNameId, dd, dataverseName);
+        String nodegroupName = ngNameId == null ? null : ngNameId.getValue();
         String compactionPolicy = dd.getCompactionPolicy();
         Map<String, String> compactionPolicyProperties = dd.getCompactionPolicyProperties();
         boolean defaultCompactionPolicy = compactionPolicy == null;
@@ -539,7 +542,7 @@
                 throw new AlgebricksException(": type " + itemTypeName + " could not be found.");
             }
             String ngName = ngNameId != null ? ngNameId.getValue()
-                    : configureNodegroupForDataset(appCtx, dd.getHints(), dataverseName, datasetName, mdTxnCtx);
+                    : configureNodegroupForDataset(appCtx, dd.getHints(), dataverseName, datasetName, metadataProvider);
 
             if (compactionPolicy == null) {
                 compactionPolicy = GlobalConfig.DEFAULT_COMPACTION_POLICY_NAME;
@@ -697,28 +700,15 @@
         }
     }
 
-    protected String getNodeGroupName(Identifier ngNameId, DatasetDecl dd, String dataverse) {
-        if (ngNameId != null) {
-            return ngNameId.getValue();
-        }
-        String hintValue = dd.getHints().get(DatasetNodegroupCardinalityHint.NAME);
-        if (hintValue == null) {
-            return MetadataConstants.METADATA_DEFAULT_NODEGROUP_NAME;
-        } else {
-            return dataverse + ":" + dd.getName().getValue();
-        }
-    }
-
     protected static String configureNodegroupForDataset(ICcApplicationContext appCtx, Map<String, String> hints,
-            String dataverseName, String datasetName, MetadataTransactionContext mdTxnCtx) throws CompilationException {
-        int nodegroupCardinality;
-        String nodegroupName;
+            String dataverseName, String datasetName, MetadataProvider metadataProvider) throws Exception {
+        Set<String> allNodes = ClusterStateManager.INSTANCE.getParticipantNodes();
+        Set<String> selectedNodes = new LinkedHashSet<>();
         String hintValue = hints.get(DatasetNodegroupCardinalityHint.NAME);
         if (hintValue == null) {
-            nodegroupName = MetadataConstants.METADATA_DEFAULT_NODEGROUP_NAME;
-            return nodegroupName;
+            selectedNodes.addAll(allNodes);
         } else {
-            int numChosen = 0;
+            int nodegroupCardinality;
             boolean valid = DatasetHints.validate(appCtx, DatasetNodegroupCardinalityHint.NAME,
                     hints.get(DatasetNodegroupCardinalityHint.NAME)).first;
             if (!valid) {
@@ -726,36 +716,12 @@
             } else {
                 nodegroupCardinality = Integer.parseInt(hints.get(DatasetNodegroupCardinalityHint.NAME));
             }
-            List<String> nodeNames = appCtx.getMetadataProperties().getNodeNames();
-            List<String> nodeNamesClone = new ArrayList<>(nodeNames);
-            String metadataNodeName = appCtx.getMetadataProperties().getMetadataNodeName();
-            List<String> selectedNodes = new ArrayList<>();
-            selectedNodes.add(metadataNodeName);
-            numChosen++;
-            nodeNamesClone.remove(metadataNodeName);
-
-            if (numChosen < nodegroupCardinality) {
-                Random random = new Random();
-                String[] nodes = nodeNamesClone.toArray(new String[] {});
-                int[] b = new int[nodeNamesClone.size()];
-                for (int i = 0; i < b.length; i++) {
-                    b[i] = i;
-                }
-
-                for (int i = 0; i < nodegroupCardinality - numChosen; i++) {
-                    int selected = i + random.nextInt(nodeNamesClone.size() - i);
-                    int selNodeIndex = b[selected];
-                    selectedNodes.add(nodes[selNodeIndex]);
-                    int temp = b[0];
-                    b[0] = b[selected];
-                    b[selected] = temp;
-                }
-            }
-            nodegroupName = dataverseName + ":" + datasetName;
-            MetadataManager.INSTANCE.addNodegroup(mdTxnCtx, new NodeGroup(nodegroupName, selectedNodes));
-            return nodegroupName;
+            List<String> allNodeList = new ArrayList<>(allNodes);
+            Collections.shuffle(allNodeList);
+            selectedNodes.addAll(allNodeList.subList(0, nodegroupCardinality));
         }
-
+        // Creates the associated node group for the dataset.
+        return DatasetUtil.createNodeGroupForNewDataset(dataverseName, datasetName, selectedNodes, metadataProvider);
     }
 
     protected void handleCreateIndexStatement(MetadataProvider metadataProvider, Statement stmt,
@@ -1184,19 +1150,15 @@
 
             // #. prepare jobs which will drop corresponding datasets with indexes.
             List<Dataset> datasets = MetadataManager.INSTANCE.getDataverseDatasets(mdTxnCtx, dataverseName);
-            for (int j = 0; j < datasets.size(); j++) {
-                String datasetName = datasets.get(j).getDatasetName();
-                DatasetType dsType = datasets.get(j).getDatasetType();
+            for (Dataset dataset : datasets) {
+                String datasetName = dataset.getDatasetName();
+                DatasetType dsType = dataset.getDatasetType();
                 if (dsType == DatasetType.INTERNAL) {
                     List<Index> indexes =
                             MetadataManager.INSTANCE.getDatasetIndexes(mdTxnCtx, dataverseName, datasetName);
-                    for (int k = 0; k < indexes.size(); k++) {
-                        if (indexes.get(k).isSecondaryIndex()) {
-                            jobsToExecute.add(
-                                    IndexUtil.buildDropIndexJobSpec(indexes.get(k), metadataProvider, datasets.get(j)));
-                        }
+                    for (Index index : indexes) {
+                        jobsToExecute.add(IndexUtil.buildDropIndexJobSpec(index, metadataProvider, dataset));
                     }
-                    jobsToExecute.add(DatasetUtil.dropDatasetJobSpec(datasets.get(j), metadataProvider));
                 } else {
                     // External dataset
                     List<Index> indexes =
@@ -1204,13 +1166,13 @@
                     for (int k = 0; k < indexes.size(); k++) {
                         if (ExternalIndexingOperations.isFileIndex(indexes.get(k))) {
                             jobsToExecute.add(ExternalIndexingOperations.buildDropFilesIndexJobSpec(metadataProvider,
-                                    datasets.get(j)));
+                                    dataset));
                         } else {
                             jobsToExecute.add(
-                                    IndexUtil.buildDropIndexJobSpec(indexes.get(k), metadataProvider, datasets.get(j)));
+                                    IndexUtil.buildDropIndexJobSpec(indexes.get(k), metadataProvider, dataset));
                         }
                     }
-                    ExternalDatasetsRegistry.INSTANCE.removeDatasetInfo(datasets.get(j));
+                    ExternalDatasetsRegistry.INSTANCE.removeDatasetInfo(dataset);
                 }
             }
             jobsToExecute.add(DataverseUtil.dropDataverseJobSpec(dv, metadataProvider));
@@ -1236,6 +1198,16 @@
 
             // #. finally, delete the dataverse.
             MetadataManager.INSTANCE.dropDataverse(mdTxnCtx, dataverseName);
+
+            // Drops all node groups that no longer needed
+            for (Dataset dataset : datasets) {
+                String nodeGroup = dataset.getNodeGroupName();
+                MetadataLockManager.INSTANCE.acquireNodeGroupWriteLock(metadataProvider.getLocks(), nodeGroup);
+                if (MetadataManager.INSTANCE.getNodegroup(mdTxnCtx, nodeGroup) != null) {
+                    MetadataManager.INSTANCE.dropNodegroup(mdTxnCtx, nodeGroup, true);
+                }
+            }
+
             if (activeDataverse != null && activeDataverse.getDataverseName() == dataverseName) {
                 activeDataverse = null;
             }
@@ -1303,14 +1275,14 @@
         MetadataLockManager.INSTANCE.dropDatasetBegin(metadataProvider.getLocks(), dataverseName,
                 dataverseName + "." + datasetName);
         try {
-            doDropDataset(dataverseName, datasetName, metadataProvider, stmtDelete.getIfExists(), hcc);
+            doDropDataset(dataverseName, datasetName, metadataProvider, stmtDelete.getIfExists(), hcc, true);
         } finally {
             metadataProvider.getLocks().unlock();
         }
     }
 
     public static void doDropDataset(String dataverseName, String datasetName, MetadataProvider metadataProvider,
-            boolean ifExists, IHyracksClientConnection hcc) throws Exception {
+            boolean ifExists, IHyracksClientConnection hcc, boolean dropCorrespondingNodeGroup) throws Exception {
         MutableObject<ProgressState> progress = new MutableObject<>(ProgressState.NO_PROGRESS);
         MutableObject<MetadataTransactionContext> mdTxnCtx =
                 new MutableObject<>(MetadataManager.INSTANCE.beginTransaction());
@@ -1328,7 +1300,7 @@
                             + dataverseName + ".");
                 }
             }
-            ds.drop(metadataProvider, mdTxnCtx, jobsToExecute, bActiveTxn, progress, hcc);
+            ds.drop(metadataProvider, mdTxnCtx, jobsToExecute, bActiveTxn, progress, hcc, dropCorrespondingNodeGroup);
             MetadataManager.INSTANCE.commitTransaction(mdTxnCtx.getValue());
         } catch (Exception e) {
             if (bActiveTxn.booleanValue()) {
@@ -1599,7 +1571,7 @@
                     throw new AlgebricksException("There is no nodegroup with this name " + nodegroupName + ".");
                 }
             } else {
-                MetadataManager.INSTANCE.dropNodegroup(mdTxnCtx, nodegroupName);
+                MetadataManager.INSTANCE.dropNodegroup(mdTxnCtx, nodegroupName, false);
             }
 
             MetadataManager.INSTANCE.commitTransaction(mdTxnCtx);
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/RebalanceUtil.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/RebalanceUtil.java
index d715410..dae88c2 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/RebalanceUtil.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/RebalanceUtil.java
@@ -24,7 +24,6 @@
 import java.util.HashSet;
 import java.util.List;
 import java.util.Set;
-import java.util.UUID;
 import java.util.stream.IntStream;
 
 import org.apache.asterix.common.exceptions.AsterixException;
@@ -37,12 +36,10 @@
 import org.apache.asterix.metadata.declared.MetadataProvider;
 import org.apache.asterix.metadata.entities.Dataset;
 import org.apache.asterix.metadata.entities.Index;
-import org.apache.asterix.metadata.entities.NodeGroup;
 import org.apache.asterix.metadata.lock.LockList;
 import org.apache.asterix.metadata.lock.MetadataLockManager;
 import org.apache.asterix.metadata.utils.DatasetUtil;
 import org.apache.asterix.metadata.utils.IndexUtil;
-import org.apache.asterix.metadata.utils.MetadataConstants;
 import org.apache.asterix.runtime.job.listener.JobEventListenerFactory;
 import org.apache.asterix.transaction.management.service.transaction.JobIdFactory;
 import org.apache.hyracks.algebricks.common.constraints.AlgebricksPartitionConstraint;
@@ -108,11 +105,10 @@
                 return;
             }
 
-            // The new node group name.
-            String nodeGroupName = datasetName + "_" + (sourceDataset.getRebalanceCount() + 1);
-
             // Creates a node group for rebalance.
-            createNodeGroup(nodeGroupName, targetNcNames, metadataProvider);
+            String nodeGroupName = DatasetUtil.createNodeGroupForNewDataset(sourceDataset.getDataverseName(),
+                    sourceDataset.getDatasetName(), sourceDataset.getRebalanceCount() + 1, targetNcNames,
+                    metadataProvider);
 
             // The target dataset for rebalance.
             targetDataset = new Dataset(sourceDataset, true, nodeGroupName);
@@ -144,20 +140,6 @@
         } finally {
             metadataProvider.getLocks().reset();
         }
-    }
-
-    // Creates a node group for the rebalance target dataset.
-    private static void createNodeGroup(String ngName, Set<String> ncNames, MetadataProvider metadataProvider)
-            throws Exception {
-        String nodeGroup = ngName;
-        MetadataTransactionContext mdTxnCtx = metadataProvider.getMetadataTxnContext();
-        MetadataLockManager.INSTANCE.acquireNodeGroupWriteLock(metadataProvider.getLocks(), nodeGroup);
-        NodeGroup ng = MetadataManager.INSTANCE.getNodegroup(mdTxnCtx, nodeGroup);
-        if (ng != null) {
-            nodeGroup = ngName + UUID.randomUUID().toString();
-            MetadataLockManager.INSTANCE.acquireNodeGroupWriteLock(metadataProvider.getLocks(), nodeGroup);
-        }
-        MetadataManager.INSTANCE.addNodegroup(mdTxnCtx, new NodeGroup(nodeGroup, new ArrayList<>(ncNames)));
     }
 
     // Rebalances from the source to the target.
@@ -197,9 +179,8 @@
 
         // Drops the metadata entry of source dataset's node group.
         String sourceNodeGroup = source.getNodeGroupName();
-        if (!sourceNodeGroup.equals(MetadataConstants.METADATA_DEFAULT_NODEGROUP_NAME)) {
-            MetadataManager.INSTANCE.dropNodegroup(mdTxnCtx, sourceNodeGroup);
-        }
+        MetadataLockManager.INSTANCE.acquireNodeGroupWriteLock(metadataProvider.getLocks(), sourceNodeGroup);
+        MetadataManager.INSTANCE.dropNodegroup(mdTxnCtx, sourceNodeGroup, true);
     }
 
     // Creates the files for the rebalance target dataset.
diff --git a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-1/dataset_with_meta-1.1.adm b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-1/dataset_with_meta-1.1.adm
index a644ebe..6615204 100644
--- a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-1/dataset_with_meta-1.1.adm
+++ b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-1/dataset_with_meta-1.1.adm
@@ -1 +1 @@
-{ "DataverseName": "test", "DatasetName": "Book", "DatatypeDataverseName": "test", "DatatypeName": "LineType", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 15:54:25 PDT 2016", "DatasetId": 101, "PendingOp": 0, "MetatypeDataverseName": "test", "MetatypeName": "AuxiliaryType" }
+{ "DataverseName": "test", "DatasetName": "Book", "DatatypeDataverseName": "test", "DatatypeName": "LineType", "DatasetType": "INTERNAL", "GroupName": "test.Book", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 15:54:25 PDT 2016", "DatasetId": 101, "PendingOp": 0, "MetatypeDataverseName": "test", "MetatypeName": "AuxiliaryType" }
diff --git a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-2/dataset_with_meta-2.1.adm b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-2/dataset_with_meta-2.1.adm
index 7160a28..94cc963 100644
--- a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-2/dataset_with_meta-2.1.adm
+++ b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-2/dataset_with_meta-2.1.adm
@@ -1 +1 @@
-{ "DataverseName": "test", "DatasetName": "Book", "DatatypeDataverseName": "test", "DatatypeName": "LineType", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 15:54:26 PDT 2016", "DatasetId": 102, "PendingOp": 0, "MetatypeDataverseName": "meta", "MetatypeName": "AuxiliaryType" }
+{ "DataverseName": "test", "DatasetName": "Book", "DatatypeDataverseName": "test", "DatatypeName": "LineType", "DatasetType": "INTERNAL", "GroupName": "test.Book", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 15:54:26 PDT 2016", "DatasetId": 102, "PendingOp": 0, "MetatypeDataverseName": "meta", "MetatypeName": "AuxiliaryType" }
diff --git a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-3/dataset_with_meta-2.3.adm b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-3/dataset_with_meta-2.3.adm
index 45fa2be..c8a4ce8 100644
--- a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-3/dataset_with_meta-2.3.adm
+++ b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-3/dataset_with_meta-2.3.adm
@@ -1 +1 @@
-{ "DataverseName": "test", "DatasetName": "Book", "DatatypeDataverseName": "test", "DatatypeName": "LineType", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 15:54:26 PDT 2016", "DatasetId": 103, "PendingOp": 0, "MetatypeDataverseName": "test", "MetatypeName": "LineType" }
+{ "DataverseName": "test", "DatasetName": "Book", "DatatypeDataverseName": "test", "DatatypeName": "LineType", "DatasetType": "INTERNAL", "GroupName": "test.Book", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 15:54:26 PDT 2016", "DatasetId": 103, "PendingOp": 0, "MetatypeDataverseName": "test", "MetatypeName": "LineType" }
diff --git a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-4/dataset_with_meta-4.3.adm b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-4/dataset_with_meta-4.3.adm
index b42a0c9..d58eaab 100644
--- a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-4/dataset_with_meta-4.3.adm
+++ b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-4/dataset_with_meta-4.3.adm
@@ -1 +1 @@
-{ "DataverseName": "test", "DatasetName": "Book", "DatatypeDataverseName": "test", "DatatypeName": "LineType", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false, "KeySourceIndicator": [ 1 ] }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 15:54:26 PDT 2016", "DatasetId": 104, "PendingOp": 0, "MetatypeDataverseName": "test", "MetatypeName": "LineType" }
+{ "DataverseName": "test", "DatasetName": "Book", "DatatypeDataverseName": "test", "DatatypeName": "LineType", "DatasetType": "INTERNAL", "GroupName": "test.Book", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false, "KeySourceIndicator": [ 1 ] }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 15:54:26 PDT 2016", "DatasetId": 104, "PendingOp": 0, "MetatypeDataverseName": "test", "MetatypeName": "LineType" }
diff --git a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-5/dataset_with_meta-5.3.adm b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-5/dataset_with_meta-5.3.adm
index ca2e7c0..5cffa59 100644
--- a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-5/dataset_with_meta-5.3.adm
+++ b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-5/dataset_with_meta-5.3.adm
@@ -1 +1 @@
-{ "DataverseName": "test", "DatasetName": "Book", "DatatypeDataverseName": "test", "DatatypeName": "LineType", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "key" ] ], "PrimaryKey": [ [ "key" ] ], "Autogenerated": false, "KeySourceIndicator": [ 1 ] }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 15:54:26 PDT 2016", "DatasetId": 105, "PendingOp": 0, "MetatypeDataverseName": "test", "MetatypeName": "AuxiliaryType" }
+{ "DataverseName": "test", "DatasetName": "Book", "DatatypeDataverseName": "test", "DatatypeName": "LineType", "DatasetType": "INTERNAL", "GroupName": "test.Book", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "key" ] ], "PrimaryKey": [ [ "key" ] ], "Autogenerated": false, "KeySourceIndicator": [ 1 ] }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 15:54:26 PDT 2016", "DatasetId": 105, "PendingOp": 0, "MetatypeDataverseName": "test", "MetatypeName": "AuxiliaryType" }
diff --git a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_2/issue_251_dataset_hint_2.1.adm b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_2/issue_251_dataset_hint_2.1.adm
index 1ad75f5..ea579ac 100644
--- a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_2/issue_251_dataset_hint_2.1.adm
+++ b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_2/issue_251_dataset_hint_2.1.adm
@@ -1 +1 @@
-{ "DataverseName": "test", "DatasetName": "Book", "DatatypeDataverseName": "test", "DatatypeName": "LineType", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{ { "Name": "CARDINALITY", "Value": "2000" } }}, "Timestamp": "Tue Jun 21 15:54:28 PDT 2016", "DatasetId": 114, "PendingOp": 0 }
+{ "DataverseName": "test", "DatasetName": "Book", "DatatypeDataverseName": "test", "DatatypeName": "LineType", "DatasetType": "INTERNAL", "GroupName": "test.Book", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{ { "Name": "CARDINALITY", "Value": "2000" } }}, "Timestamp": "Tue Jun 21 15:54:28 PDT 2016", "DatasetId": 114, "PendingOp": 0 }
diff --git a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_3/issue_251_dataset_hint_3.1.adm b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_3/issue_251_dataset_hint_3.1.adm
index 9f2e9ea..2c9fc43 100644
--- a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_3/issue_251_dataset_hint_3.1.adm
+++ b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_3/issue_251_dataset_hint_3.1.adm
@@ -1 +1 @@
-{ "DataverseName": "test", "DatasetName": "Book", "DatatypeDataverseName": "test", "DatatypeName": "LineType", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{ { "Name": "CARDINALITY", "Value": "2000" } }}, "Timestamp": "Tue Jun 21 15:54:28 PDT 2016", "DatasetId": 115, "PendingOp": 0 }
+{ "DataverseName": "test", "DatasetName": "Book", "DatatypeDataverseName": "test", "DatatypeName": "LineType", "DatasetType": "INTERNAL", "GroupName": "test.Book", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{ { "Name": "CARDINALITY", "Value": "2000" } }}, "Timestamp": "Tue Jun 21 15:54:28 PDT 2016", "DatasetId": 115, "PendingOp": 0 }
diff --git a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_4/issue_251_dataset_hint_4.1.adm b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_4/issue_251_dataset_hint_4.1.adm
index f59f892..d72c8bd 100644
--- a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_4/issue_251_dataset_hint_4.1.adm
+++ b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_4/issue_251_dataset_hint_4.1.adm
@@ -1 +1 @@
-{ "DataverseName": "test", "DatasetName": "Book", "DatatypeDataverseName": "test", "DatatypeName": "LineType", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 15:54:28 PDT 2016", "DatasetId": 116, "PendingOp": 0 }
+{ "DataverseName": "test", "DatasetName": "Book", "DatatypeDataverseName": "test", "DatatypeName": "LineType", "DatasetType": "INTERNAL", "GroupName": "test.Book", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 15:54:28 PDT 2016", "DatasetId": 116, "PendingOp": 0 }
diff --git a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta02/meta02.1.adm b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta02/meta02.1.adm
index 65ec178..7a24dab 100644
--- a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta02/meta02.1.adm
+++ b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta02/meta02.1.adm
@@ -1 +1 @@
-{ "DataverseName": "testdv", "DatasetName": "dst01", "DatatypeDataverseName": "testdv", "DatatypeName": "testtype", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 15:54:27 PDT 2016", "DatasetId": 108, "PendingOp": 0 }
+{ "DataverseName": "testdv", "DatasetName": "dst01", "DatatypeDataverseName": "testdv", "DatatypeName": "testtype", "DatasetType": "INTERNAL", "GroupName": "testdv.dst01", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 15:54:27 PDT 2016", "DatasetId": 108, "PendingOp": 0 }
diff --git a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta08/meta08.1.adm b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta08/meta08.1.adm
index fb590b3..f9158a4 100644
--- a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta08/meta08.1.adm
+++ b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta08/meta08.1.adm
@@ -1,2 +1 @@
-{ "GroupName": "DEFAULT_NG_ALL_NODES", "NodeNames": {{ "asterix_nc1", "asterix_nc2" }}, "Timestamp": "Mon Sep 17 12:31:45 PDT 2012" }
 { "GroupName": "MetadataGroup", "NodeNames": {{ "asterix_nc1" }}, "Timestamp": "Mon Sep 17 12:31:45 PDT 2012" }
diff --git a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta09/meta09.1.adm b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta09/meta09.1.adm
index 43f0487..a6c6bb1 100644
--- a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta09/meta09.1.adm
+++ b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/meta09/meta09.1.adm
@@ -1 +1 @@
-{ "DataverseName": "test", "DatasetName": "t1", "DatatypeDataverseName": "test", "DatatypeName": "testtype", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 15:54:27 PDT 2016", "DatasetId": 110, "PendingOp": 0 }
+{ "DataverseName": "test", "DatasetName": "t1", "DatatypeDataverseName": "test", "DatatypeName": "testtype", "DatasetType": "INTERNAL", "GroupName": "test.t1", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 15:54:27 PDT 2016", "DatasetId": 110, "PendingOp": 0 }
diff --git a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_nodegroup/metadata_nodegroup.1.adm b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_nodegroup/metadata_nodegroup.1.adm
index bcf68bb..981fc7e 100644
--- a/asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_nodegroup/metadata_nodegroup.1.adm
+++ b/asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_nodegroup/metadata_nodegroup.1.adm
@@ -1,2 +1,2 @@
-{ "GroupName": "DEFAULT_NG_ALL_NODES", "NodeNames": {{ "asterix_nc1", "asterix_nc2" }}, "Timestamp": "Thu Sep 13 11:42:20 PDT 2012" }
 { "GroupName": "MetadataGroup", "NodeNames": {{ "asterix_nc1" }}, "Timestamp": "Thu Sep 13 11:42:20 PDT 2012" }
+{ "GroupName": "testdv.t1", "NodeNames": {{ "asterix_nc1", "asterix_nc2" }}, "Timestamp": "Thu Jun 01 16:07:46 PDT 2017" }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/index-selection/intersection_with_nodegroup/intersection_with_nodegroup.4.ddl.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/index-selection/intersection_with_nodegroup/intersection_with_nodegroup.4.ddl.aql
index 5e33117..058c36c 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries/index-selection/intersection_with_nodegroup/intersection_with_nodegroup.4.ddl.aql
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/index-selection/intersection_with_nodegroup/intersection_with_nodegroup.4.ddl.aql
@@ -18,4 +18,4 @@
  */
 drop dataverse TinySocial if exists;
 
-drop nodegroup group_test;
\ No newline at end of file
+drop nodegroup group_test if exists;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/index-selection/multi-index/multi-index.1.ddl.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/index-selection/multi-index/multi-index.1.ddl.aql
index 0efffb2..f32bae0 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/index-selection/multi-index/multi-index.1.ddl.aql
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/open-index-enforced/index-selection/multi-index/multi-index.1.ddl.aql
@@ -35,7 +35,7 @@
   misc: string
 }
 
-create nodegroup group1 if not exists on nc1, nc2;
+create nodegroup group1 if not exists on asterix_nc1, asterix_nc2;
 
 create dataset DBLP(DBLPType)
   primary key id on group1;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset_nodegroup/dataset_nodegroup.1.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset_nodegroup/dataset_nodegroup.1.ddl.sqlpp
new file mode 100644
index 0000000..1a9b76f
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset_nodegroup/dataset_nodegroup.1.ddl.sqlpp
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+drop  dataverse twitter if exists;
+create  dataverse twitter;
+
+use twitter;
+
+create type Tweet as
+{
+  id : bigint,
+  tweetid : bigint,
+  loc : point,
+  time : datetime,
+  text : string
+}
+
+create dataset TwitterData(Tweet) primary key id;
+
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset_nodegroup/dataset_nodegroup.2.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset_nodegroup/dataset_nodegroup.2.query.sqlpp
new file mode 100644
index 0000000..77d9e14
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset_nodegroup/dataset_nodegroup.2.query.sqlpp
@@ -0,0 +1,24 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+use twitter;
+
+SELECT ds.GroupName, ng.NodeNames
+FROM Metadata.`Dataset` ds, Metadata.`Nodegroup` ng
+WHERE ds.GroupName = ng.GroupName AND ds.DatasetName="TwitterData";
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/rebalance/duplicate_location/duplicate_location.5.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/rebalance/duplicate_location/duplicate_location.5.query.sqlpp
index 6045a79..0bf7758 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/rebalance/duplicate_location/duplicate_location.5.query.sqlpp
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/rebalance/duplicate_location/duplicate_location.5.query.sqlpp
@@ -19,4 +19,4 @@
 
 select GroupName, NodeNames
 from Metadata.`Nodegroup`
-where GroupName like "LineItem%";
+where GroupName like "tpch.LineItem%";
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv02/cross-dv02.1.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv02/cross-dv02.1.adm
index d522a0a..b541030 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv02/cross-dv02.1.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv02/cross-dv02.1.adm
@@ -1,4 +1,4 @@
-{ "DataverseName": "student", "DatasetName": "gdstd", "DatatypeDataverseName": "student", "DatatypeName": "stdType", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:36 PDT 2016", "DatasetId": 1860, "PendingOp": 0 }
-{ "DataverseName": "teacher", "DatasetName": "prof", "DatatypeDataverseName": "teacher", "DatatypeName": "tchrType", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:36 PDT 2016", "DatasetId": 1861, "PendingOp": 0 }
-{ "DataverseName": "teacher", "DatasetName": "pstdoc", "DatatypeDataverseName": "teacher", "DatatypeName": "tchrType", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:36 PDT 2016", "DatasetId": 1862, "PendingOp": 0 }
-{ "DataverseName": "student", "DatasetName": "ugdstd", "DatatypeDataverseName": "student", "DatatypeName": "stdType", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:36 PDT 2016", "DatasetId": 1859, "PendingOp": 0 }
+{ "DataverseName": "student", "DatasetName": "gdstd", "DatatypeDataverseName": "student", "DatatypeName": "stdType", "DatasetType": "INTERNAL", "GroupName": "student.gdstd", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:36 PDT 2016", "DatasetId": 1860, "PendingOp": 0 }
+{ "DataverseName": "teacher", "DatasetName": "prof", "DatatypeDataverseName": "teacher", "DatatypeName": "tchrType", "DatasetType": "INTERNAL", "GroupName": "teacher.prof", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:36 PDT 2016", "DatasetId": 1861, "PendingOp": 0 }
+{ "DataverseName": "teacher", "DatasetName": "pstdoc", "DatatypeDataverseName": "teacher", "DatatypeName": "tchrType", "DatasetType": "INTERNAL", "GroupName": "teacher.pstdoc", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:36 PDT 2016", "DatasetId": 1862, "PendingOp": 0 }
+{ "DataverseName": "student", "DatasetName": "ugdstd", "DatatypeDataverseName": "student", "DatatypeName": "stdType", "DatasetType": "INTERNAL", "GroupName": "student.ugdstd", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:36 PDT 2016", "DatasetId": 1859, "PendingOp": 0 }
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv04/cross-dv04.1.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv04/cross-dv04.1.adm
index c8f9ec7..4401cf4 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv04/cross-dv04.1.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv04/cross-dv04.1.adm
@@ -1,4 +1,4 @@
-{ "DataverseName": "student", "DatasetName": "gdstd", "DatatypeDataverseName": "student", "DatatypeName": "stdType", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:37 PDT 2016", "DatasetId": 1872, "PendingOp": 0 }
-{ "DataverseName": "teacher", "DatasetName": "prof", "DatatypeDataverseName": "teacher", "DatatypeName": "tchrType", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:37 PDT 2016", "DatasetId": 1873, "PendingOp": 0 }
-{ "DataverseName": "teacher", "DatasetName": "pstdoc", "DatatypeDataverseName": "teacher", "DatatypeName": "tchrType", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:37 PDT 2016", "DatasetId": 1874, "PendingOp": 0 }
-{ "DataverseName": "student", "DatasetName": "ugdstd", "DatatypeDataverseName": "student", "DatatypeName": "stdType", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:37 PDT 2016", "DatasetId": 1871, "PendingOp": 0 }
+{ "DataverseName": "student", "DatasetName": "gdstd", "DatatypeDataverseName": "student", "DatatypeName": "stdType", "DatasetType": "INTERNAL", "GroupName": "student.gdstd", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:37 PDT 2016", "DatasetId": 1872, "PendingOp": 0 }
+{ "DataverseName": "teacher", "DatasetName": "prof", "DatatypeDataverseName": "teacher", "DatatypeName": "tchrType", "DatasetType": "INTERNAL", "GroupName": "teacher.prof", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:37 PDT 2016", "DatasetId": 1873, "PendingOp": 0 }
+{ "DataverseName": "teacher", "DatasetName": "pstdoc", "DatatypeDataverseName": "teacher", "DatatypeName": "tchrType", "DatasetType": "INTERNAL", "GroupName": "teacher.pstdoc", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:37 PDT 2016", "DatasetId": 1874, "PendingOp": 0 }
+{ "DataverseName": "student", "DatasetName": "ugdstd", "DatatypeDataverseName": "student", "DatatypeName": "stdType", "DatasetType": "INTERNAL", "GroupName": "student.ugdstd", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:37 PDT 2016", "DatasetId": 1871, "PendingOp": 0 }
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv19/cross-dv19.1.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv19/cross-dv19.1.adm
index d007dd7..5bf8498 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv19/cross-dv19.1.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv19/cross-dv19.1.adm
@@ -1,7 +1,7 @@
-{ "DataverseName": "test1", "DatasetName": "TwitterData", "DatatypeDataverseName": "test1", "DatatypeName": "Tweet", "DatasetType": "EXTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "ExternalDetails": { "DatasourceAdapter": "localfs", "Properties": [ { "Name": "path", "Value": "asterix_nc1://data/twitter/extrasmalltweets.txt" }, { "Name": "format", "Value": "adm" } ], "LastRefreshTime": datetime("2016-06-21T23:06:38.287Z"), "TransactionState": 0 }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:38 PDT 2016", "DatasetId": 1882, "PendingOp": 0 }
-{ "DataverseName": "test1", "DatasetName": "t1", "DatatypeDataverseName": "test1", "DatatypeName": "testtype", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:38 PDT 2016", "DatasetId": 1876, "PendingOp": 0 }
-{ "DataverseName": "test1", "DatasetName": "t2", "DatatypeDataverseName": "test1", "DatatypeName": "testtype", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:38 PDT 2016", "DatasetId": 1879, "PendingOp": 0 }
-{ "DataverseName": "test1", "DatasetName": "t3", "DatatypeDataverseName": "test1", "DatatypeName": "testtype", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:38 PDT 2016", "DatasetId": 1880, "PendingOp": 0 }
-{ "DataverseName": "test2", "DatasetName": "t2", "DatatypeDataverseName": "test2", "DatatypeName": "testtype", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:38 PDT 2016", "DatasetId": 1877, "PendingOp": 0 }
-{ "DataverseName": "test2", "DatasetName": "t3", "DatatypeDataverseName": "test2", "DatatypeName": "testtype", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:38 PDT 2016", "DatasetId": 1878, "PendingOp": 0 }
-{ "DataverseName": "test2", "DatasetName": "t4", "DatatypeDataverseName": "test2", "DatatypeName": "testtype", "DatasetType": "INTERNAL", "GroupName": "DEFAULT_NG_ALL_NODES", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:38 PDT 2016", "DatasetId": 1881, "PendingOp": 0 }
+{ "DataverseName": "test1", "DatasetName": "TwitterData", "DatatypeDataverseName": "test1", "DatatypeName": "Tweet", "DatasetType": "EXTERNAL", "GroupName": "test1.TwitterData", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "ExternalDetails": { "DatasourceAdapter": "localfs", "Properties": [ { "Name": "path", "Value": "asterix_nc1://data/twitter/extrasmalltweets.txt" }, { "Name": "format", "Value": "adm" } ], "LastRefreshTime": datetime("2016-06-21T23:06:38.287Z"), "TransactionState": 0 }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:38 PDT 2016", "DatasetId": 1882, "PendingOp": 0 }
+{ "DataverseName": "test1", "DatasetName": "t1", "DatatypeDataverseName": "test1", "DatatypeName": "testtype", "DatasetType": "INTERNAL", "GroupName": "test1.t1", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:38 PDT 2016", "DatasetId": 1876, "PendingOp": 0 }
+{ "DataverseName": "test1", "DatasetName": "t2", "DatatypeDataverseName": "test1", "DatatypeName": "testtype", "DatasetType": "INTERNAL", "GroupName": "test1.t2", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:38 PDT 2016", "DatasetId": 1879, "PendingOp": 0 }
+{ "DataverseName": "test1", "DatasetName": "t3", "DatatypeDataverseName": "test1", "DatatypeName": "testtype", "DatasetType": "INTERNAL", "GroupName": "test1.t3", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:38 PDT 2016", "DatasetId": 1880, "PendingOp": 0 }
+{ "DataverseName": "test2", "DatasetName": "t2", "DatatypeDataverseName": "test2", "DatatypeName": "testtype", "DatasetType": "INTERNAL", "GroupName": "test2.t2", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:38 PDT 2016", "DatasetId": 1877, "PendingOp": 0 }
+{ "DataverseName": "test2", "DatasetName": "t3", "DatatypeDataverseName": "test2", "DatatypeName": "testtype", "DatasetType": "INTERNAL", "GroupName": "test2.t3", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:38 PDT 2016", "DatasetId": 1878, "PendingOp": 0 }
+{ "DataverseName": "test2", "DatasetName": "t4", "DatatypeDataverseName": "test2", "DatatypeName": "testtype", "DatasetType": "INTERNAL", "GroupName": "test2.t4", "CompactionPolicy": "prefix", "CompactionPolicyProperties": [ { "Name": "max-mergable-component-size", "Value": "1073741824" }, { "Name": "max-tolerance-component-count", "Value": "5" } ], "InternalDetails": { "FileStructure": "BTREE", "PartitioningStrategy": "HASH", "PartitioningKey": [ [ "id" ] ], "PrimaryKey": [ [ "id" ] ], "Autogenerated": false }, "Hints": {{  }}, "Timestamp": "Tue Jun 21 16:06:38 PDT 2016", "DatasetId": 1881, "PendingOp": 0 }
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/misc/dataset_nodegroup/dataset_nodegroup.1.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/misc/dataset_nodegroup/dataset_nodegroup.1.adm
new file mode 100644
index 0000000..26eff43
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/misc/dataset_nodegroup/dataset_nodegroup.1.adm
@@ -0,0 +1 @@
+{ "GroupName": "twitter.TwitterData", "NodeNames": {{ "asterix_nc1", "asterix_nc2" }} }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.14.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.14.adm
index 3c6303a..3c8a3de 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.14.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.14.adm
@@ -1 +1 @@
-{ "DatasetName": "LineItem", "GroupName": "LineItem_2", "rebalanceCount": 2 }
\ No newline at end of file
+{ "DatasetName": "LineItem", "GroupName": "tpch1.LineItem_2", "rebalanceCount": 2 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.15.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.15.adm
index 77972f2..06a2a13 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.15.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.15.adm
@@ -1 +1 @@
-{ "DatasetName": "Orders", "GroupName": "Orders_2", "rebalanceCount": 2 }
\ No newline at end of file
+{ "DatasetName": "Orders", "GroupName": "tpch2.Orders_2", "rebalanceCount": 2 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.8.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.8.adm
index 2760db0..356deeb 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.8.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.8.adm
@@ -1 +1 @@
-{ "DatasetName": "LineItem", "GroupName": "LineItem_1", "rebalanceCount": 1 }
\ No newline at end of file
+{ "DatasetName": "LineItem", "GroupName": "tpch1.LineItem_1", "rebalanceCount": 1 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.9.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.9.adm
index 9eb181a..0f829fa 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.9.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.9.adm
@@ -1 +1 @@
-{ "DatasetName": "Orders", "GroupName": "Orders_1", "rebalanceCount": 1 }
\ No newline at end of file
+{ "DatasetName": "Orders", "GroupName": "tpch2.Orders_1", "rebalanceCount": 1 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/duplicate_location/duplicate_location.4.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/duplicate_location/duplicate_location.4.adm
index 2760db0..6b2fe84 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/duplicate_location/duplicate_location.4.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/duplicate_location/duplicate_location.4.adm
@@ -1 +1 @@
-{ "DatasetName": "LineItem", "GroupName": "LineItem_1", "rebalanceCount": 1 }
\ No newline at end of file
+{ "DatasetName": "LineItem", "GroupName": "tpch.LineItem_1", "rebalanceCount": 1 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/duplicate_location/duplicate_location.5.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/duplicate_location/duplicate_location.5.adm
index 5ec3f11..4a47b46 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/duplicate_location/duplicate_location.5.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/duplicate_location/duplicate_location.5.adm
@@ -1 +1 @@
-{ "GroupName": "LineItem_1", "NodeNames": {{ "asterix_nc1" }} }
\ No newline at end of file
+{ "GroupName": "tpch.LineItem_1", "NodeNames": {{ "asterix_nc1" }} }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/empty_location/empty_location.3.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/empty_location/empty_location.3.adm
index 398b082..7498ed7 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/empty_location/empty_location.3.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/empty_location/empty_location.3.adm
@@ -1 +1 @@
-{ "DatasetName": "LineItem", "GroupName": "DEFAULT_NG_ALL_NODES" }
\ No newline at end of file
+{ "DatasetName": "LineItem", "GroupName": "tpch.LineItem" }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/identical_location/identical_location.4.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/identical_location/identical_location.4.adm
index 398b082..7498ed7 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/identical_location/identical_location.4.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/identical_location/identical_location.4.adm
@@ -1 +1 @@
-{ "DatasetName": "LineItem", "GroupName": "DEFAULT_NG_ALL_NODES" }
\ No newline at end of file
+{ "DatasetName": "LineItem", "GroupName": "tpch.LineItem" }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset/single_dataset.11.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset/single_dataset.11.adm
index 3c6303a..c9dfdaa 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset/single_dataset.11.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset/single_dataset.11.adm
@@ -1 +1 @@
-{ "DatasetName": "LineItem", "GroupName": "LineItem_2", "rebalanceCount": 2 }
\ No newline at end of file
+{ "DatasetName": "LineItem", "GroupName": "tpch.LineItem_2", "rebalanceCount": 2 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset/single_dataset.7.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset/single_dataset.7.adm
index 2760db0..6b2fe84 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset/single_dataset.7.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset/single_dataset.7.adm
@@ -1 +1 @@
-{ "DatasetName": "LineItem", "GroupName": "LineItem_1", "rebalanceCount": 1 }
\ No newline at end of file
+{ "DatasetName": "LineItem", "GroupName": "tpch.LineItem_1", "rebalanceCount": 1 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset_with_index/single_dataset_with_index.12.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset_with_index/single_dataset_with_index.12.adm
index 3c6303a..c9dfdaa 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset_with_index/single_dataset_with_index.12.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset_with_index/single_dataset_with_index.12.adm
@@ -1 +1 @@
-{ "DatasetName": "LineItem", "GroupName": "LineItem_2", "rebalanceCount": 2 }
\ No newline at end of file
+{ "DatasetName": "LineItem", "GroupName": "tpch.LineItem_2", "rebalanceCount": 2 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset_with_index/single_dataset_with_index.7.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset_with_index/single_dataset_with_index.7.adm
index 2760db0..6b2fe84 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset_with_index/single_dataset_with_index.7.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset_with_index/single_dataset_with_index.7.adm
@@ -1 +1 @@
-{ "DatasetName": "LineItem", "GroupName": "LineItem_1", "rebalanceCount": 1 }
\ No newline at end of file
+{ "DatasetName": "LineItem", "GroupName": "tpch.LineItem_1", "rebalanceCount": 1 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.14.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.14.adm
index 3c6303a..c9dfdaa 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.14.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.14.adm
@@ -1 +1 @@
-{ "DatasetName": "LineItem", "GroupName": "LineItem_2", "rebalanceCount": 2 }
\ No newline at end of file
+{ "DatasetName": "LineItem", "GroupName": "tpch.LineItem_2", "rebalanceCount": 2 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.15.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.15.adm
index 77972f2..38ed329 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.15.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.15.adm
@@ -1 +1 @@
-{ "DatasetName": "Orders", "GroupName": "Orders_2", "rebalanceCount": 2 }
\ No newline at end of file
+{ "DatasetName": "Orders", "GroupName": "tpch.Orders_2", "rebalanceCount": 2 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.8.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.8.adm
index 2760db0..6b2fe84 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.8.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.8.adm
@@ -1 +1 @@
-{ "DatasetName": "LineItem", "GroupName": "LineItem_1", "rebalanceCount": 1 }
\ No newline at end of file
+{ "DatasetName": "LineItem", "GroupName": "tpch.LineItem_1", "rebalanceCount": 1 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.9.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.9.adm
index 9eb181a..6461f3c 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.9.adm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.9.adm
@@ -1 +1 @@
-{ "DatasetName": "Orders", "GroupName": "Orders_1", "rebalanceCount": 1 }
\ No newline at end of file
+{ "DatasetName": "Orders", "GroupName": "tpch.Orders_1", "rebalanceCount": 1 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
index f9ae2fa..4574c66 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
@@ -3192,6 +3192,11 @@
       </compilation-unit>
     </test-case>
     <test-case FilePath="misc">
+      <compilation-unit name="dataset_nodegroup">
+        <output-dir compare="Text">dataset_nodegroup</output-dir>
+      </compilation-unit>
+    </test-case>
+    <test-case FilePath="misc">
       <compilation-unit name="partition-by-nonexistent-field">
         <output-dir compare="Text">partition-by-nonexistent-field</output-dir>
         <expected-error>Field "id" is not found</expected-error>
diff --git a/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/statement/DatasetDecl.java b/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/statement/DatasetDecl.java
index 2eba48a..79d1774 100644
--- a/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/statement/DatasetDecl.java
+++ b/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/statement/DatasetDecl.java
@@ -25,7 +25,6 @@
 import org.apache.asterix.lang.common.base.Statement;
 import org.apache.asterix.lang.common.struct.Identifier;
 import org.apache.asterix.lang.common.visitor.base.ILangVisitor;
-import org.apache.asterix.metadata.utils.MetadataConstants;
 
 public class DatasetDecl implements Statement {
     protected final Identifier name;
@@ -61,8 +60,7 @@
         } else {
             this.metaItemTypeDataverse = metaItemTypeDataverse;
         }
-        this.nodegroupName = nodeGroupName == null ? new Identifier(MetadataConstants.METADATA_DEFAULT_NODEGROUP_NAME)
-                : nodeGroupName;
+        this.nodegroupName = nodeGroupName;
         this.compactionPolicy = compactionPolicy;
         this.compactionPolicyProperties = compactionPolicyProperties;
         this.hints = hints;
diff --git a/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/visitor/FormatPrintVisitor.java b/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/visitor/FormatPrintVisitor.java
index 35d0a29..8b4cca1 100644
--- a/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/visitor/FormatPrintVisitor.java
+++ b/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/visitor/FormatPrintVisitor.java
@@ -60,7 +60,6 @@
 import org.apache.asterix.lang.common.expression.UnaryExpr;
 import org.apache.asterix.lang.common.expression.UnorderedListTypeDefinition;
 import org.apache.asterix.lang.common.expression.VariableExpr;
-import org.apache.asterix.lang.common.literal.IntegerLiteral;
 import org.apache.asterix.lang.common.statement.CompactStatement;
 import org.apache.asterix.lang.common.statement.ConnectFeedStatement;
 import org.apache.asterix.lang.common.statement.CreateDataverseStatement;
@@ -98,7 +97,6 @@
 import org.apache.asterix.lang.common.struct.QuantifiedPair;
 import org.apache.asterix.lang.common.struct.UnaryExprType;
 import org.apache.asterix.lang.common.visitor.base.ILangVisitor;
-import org.apache.asterix.metadata.utils.MetadataConstants;
 import org.apache.hyracks.algebricks.common.utils.Pair;
 import org.apache.hyracks.algebricks.core.algebra.expressions.IExpressionAnnotation;
 
@@ -490,8 +488,7 @@
             printConfiguration(externalDetails.getProperties());
         }
         Identifier nodeGroupName = dd.getNodegroupName();
-        if (nodeGroupName != null
-                && !nodeGroupName.getValue().equals(MetadataConstants.METADATA_DEFAULT_NODEGROUP_NAME)) {
+        if (nodeGroupName != null) {
             out.print(" on " + nodeGroupName.getValue());
         }
         Map<String, String> hints = dd.getHints();
diff --git a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataManager.java b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataManager.java
index 0cc1958..11645e8 100644
--- a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataManager.java
+++ b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataManager.java
@@ -515,13 +515,17 @@
     }
 
     @Override
-    public void dropNodegroup(MetadataTransactionContext ctx, String nodeGroupName) throws MetadataException {
+    public void dropNodegroup(MetadataTransactionContext ctx, String nodeGroupName, boolean failSilently)
+            throws MetadataException {
+        boolean dropped;
         try {
-            metadataNode.dropNodegroup(ctx.getJobId(), nodeGroupName);
+            dropped = metadataNode.dropNodegroup(ctx.getJobId(), nodeGroupName, failSilently);
         } catch (RemoteException e) {
             throw new MetadataException(e);
         }
-        ctx.dropNodeGroup(nodeGroupName);
+        if (dropped) {
+            ctx.dropNodeGroup(nodeGroupName);
+        }
     }
 
     @Override
diff --git a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataNode.java b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataNode.java
index ea7a47c..4880a73 100644
--- a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataNode.java
+++ b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataNode.java
@@ -645,10 +645,13 @@
     }
 
     @Override
-    public void dropNodegroup(JobId jobId, String nodeGroupName) throws MetadataException, RemoteException {
-        List<String> datasetNames;
-        datasetNames = getDatasetNamesPartitionedOnThisNodeGroup(jobId, nodeGroupName);
+    public boolean dropNodegroup(JobId jobId, String nodeGroupName, boolean failSilently)
+            throws MetadataException, RemoteException {
+        List<String> datasetNames = getDatasetNamesPartitionedOnThisNodeGroup(jobId, nodeGroupName);
         if (!datasetNames.isEmpty()) {
+            if (failSilently) {
+                return false;
+            }
             StringBuilder sb = new StringBuilder();
             sb.append("Nodegroup '" + nodeGroupName
                     + "' cannot be dropped; it was used for partitioning these datasets:");
@@ -665,6 +668,7 @@
             deleteTupleFromIndex(jobId, MetadataPrimaryIndexes.NODEGROUP_DATASET, tuple);
             // TODO: Change this to be a BTree specific exception, e.g.,
             // BTreeKeyDoesNotExistException.
+            return true;
         } catch (HyracksDataException e) {
             if (e.getComponent().equals(ErrorCode.HYRACKS)
                     && e.getErrorCode() == ErrorCode.UPDATE_OR_DELETE_NON_EXISTENT_KEY) {
diff --git a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IMetadataManager.java b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IMetadataManager.java
index f36d510..57fcc5e 100644
--- a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IMetadataManager.java
+++ b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IMetadataManager.java
@@ -349,11 +349,14 @@
      *            MetadataTransactionContext of an active metadata transaction.
      * @param nodeGroupName
      *            Name of node group to be deleted.
+     * @param failSilently
+     *            true means it's a no-op if the node group cannot be dropped; false means it will throw an exception.
      * @throws MetadataException
      *             For example, there are still datasets partitioned on the node
      *             group to be deleted.
      */
-    void dropNodegroup(MetadataTransactionContext ctx, String nodeGroupName) throws MetadataException;
+    void dropNodegroup(MetadataTransactionContext ctx, String nodeGroupName, boolean failSilently)
+            throws MetadataException;
 
     /**
      * Inserts a node (machine).
diff --git a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IMetadataNode.java b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IMetadataNode.java
index c4ae70c..cfbbda3 100644
--- a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IMetadataNode.java
+++ b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IMetadataNode.java
@@ -367,12 +367,16 @@
      *            A globally unique id for an active metadata transaction.
      * @param nodeGroupName
      *            Name of node group to be deleted.
+     * @param failSilently
+     *            true means it's a no-op if the node group cannot be dropped; false means it will throw an exception.
+     * @return Whether the node group has been successfully dropped.
      * @throws MetadataException
      *             For example, there are still datasets partitioned on the node
      *             group to be deleted.
      * @throws RemoteException
      */
-    void dropNodegroup(JobId jobId, String nodeGroupName) throws MetadataException, RemoteException;
+    boolean dropNodegroup(JobId jobId, String nodeGroupName, boolean failSilently)
+            throws MetadataException, RemoteException;
 
     /**
      * Inserts a node (compute node), acquiring local locks on behalf of the
diff --git a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
index 673a5ae..93b19f1 100644
--- a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
+++ b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
@@ -26,7 +26,6 @@
 import java.util.logging.Level;
 import java.util.logging.Logger;
 
-import org.apache.asterix.common.api.IDatasetLifecycleManager;
 import org.apache.asterix.common.api.INcApplicationContext;
 import org.apache.asterix.common.cluster.ClusterPartition;
 import org.apache.asterix.common.config.ClusterProperties;
@@ -84,14 +83,11 @@
 import org.apache.hyracks.storage.am.lsm.common.api.ILSMIOOperationCallbackFactory;
 import org.apache.hyracks.storage.am.lsm.common.api.ILSMMergePolicyFactory;
 import org.apache.hyracks.storage.am.lsm.common.api.ILSMOperationTrackerFactory;
-import org.apache.hyracks.storage.am.lsm.common.api.IVirtualBufferCache;
 import org.apache.hyracks.storage.am.lsm.common.impls.ConstantMergePolicyFactory;
 import org.apache.hyracks.storage.am.lsm.common.impls.NoMergePolicyFactory;
 import org.apache.hyracks.storage.am.lsm.common.impls.PrefixMergePolicyFactory;
 import org.apache.hyracks.storage.common.ILocalResourceRepository;
 import org.apache.hyracks.storage.common.LocalResource;
-import org.apache.hyracks.storage.common.buffercache.IBufferCache;
-import org.apache.hyracks.storage.common.file.IFileMapProvider;
 
 /**
  * Initializes the remote metadata storage facilities ("universe") using a
@@ -106,14 +102,10 @@
     public static final boolean IS_DEBUG_MODE = false;
     private static final Logger LOGGER = Logger.getLogger(MetadataBootstrap.class.getName());
     private static INcApplicationContext appContext;
-    private static IBufferCache bufferCache;
-    private static IFileMapProvider fileMapProvider;
-    private static IDatasetLifecycleManager dataLifecycleManager;
     private static ILocalResourceRepository localResourceRepository;
     private static IIOManager ioManager;
     private static String metadataNodeName;
     private static List<String> nodeNames;
-    private static String outputDir;
     private static boolean isNewUniverse;
     private static final IMetadataIndex[] PRIMARY_INDEXES =
             new IMetadataIndex[] { MetadataPrimaryIndexes.DATAVERSE_DATASET, MetadataPrimaryIndexes.DATASET_DATASET,
@@ -130,7 +122,6 @@
     /**
      * bootstrap metadata
      *
-     * @param asterixPropertiesProvider
      * @param ncServiceContext
      * @param isNewUniverse
      * @throws ACIDException
@@ -146,10 +137,7 @@
         MetadataProperties metadataProperties = appContext.getMetadataProperties();
         metadataNodeName = metadataProperties.getMetadataNodeName();
         nodeNames = metadataProperties.getNodeNames();
-        dataLifecycleManager = appContext.getDatasetLifecycleManager();
         localResourceRepository = appContext.getLocalResourceRepository();
-        bufferCache = appContext.getBufferCache();
-        fileMapProvider = appContext.getFileMapManager();
         ioManager = ncServiceContext.getIoManager();
 
         MetadataTransactionContext mdTxnCtx = MetadataManager.INSTANCE.beginTransaction();
@@ -260,10 +248,6 @@
         metadataGroupNodeNames.add(metadataNodeName);
         NodeGroup groupRecord = new NodeGroup(MetadataConstants.METADATA_NODEGROUP_NAME, metadataGroupNodeNames);
         MetadataManager.INSTANCE.addNodegroup(mdTxnCtx, groupRecord);
-        List<String> nodes = new ArrayList<>();
-        nodes.addAll(nodeNames);
-        NodeGroup defaultGroup = new NodeGroup(MetadataConstants.METADATA_DEFAULT_NODEGROUP_NAME, nodes);
-        MetadataManager.INSTANCE.addNodegroup(mdTxnCtx, defaultGroup);
     }
 
     private static void insertInitialAdapters(MetadataTransactionContext mdTxnCtx) throws MetadataException {
@@ -378,14 +362,6 @@
             indexHelper.open(); // Opening the index through the helper will ensure it gets instantiated
             indexHelper.close();
         }
-    }
-
-    public static String getOutputDir() {
-        return outputDir;
-    }
-
-    public static String getMetadataNodeName() {
-        return metadataNodeName;
     }
 
     /**
diff --git a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Dataset.java b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Dataset.java
index 4b31767..3aed427 100644
--- a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Dataset.java
+++ b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Dataset.java
@@ -56,11 +56,11 @@
 import org.apache.asterix.metadata.declared.BTreeResourceFactoryProvider;
 import org.apache.asterix.metadata.declared.MetadataProvider;
 import org.apache.asterix.metadata.lock.ExternalDatasetsRegistry;
+import org.apache.asterix.metadata.lock.MetadataLockManager;
 import org.apache.asterix.metadata.utils.DatasetUtil;
 import org.apache.asterix.metadata.utils.ExternalIndexingOperations;
 import org.apache.asterix.metadata.utils.IndexUtil;
 import org.apache.asterix.metadata.utils.InvertedIndexResourceFactoryProvider;
-import org.apache.asterix.metadata.utils.MetadataConstants;
 import org.apache.asterix.metadata.utils.MetadataUtil;
 import org.apache.asterix.metadata.utils.RTreeResourceFactoryProvider;
 import org.apache.asterix.om.types.ARecordType;
@@ -320,7 +320,7 @@
      */
     public void drop(MetadataProvider metadataProvider, MutableObject<MetadataTransactionContext> mdTxnCtx,
             List<JobSpecification> jobsToExecute, MutableBoolean bActiveTxn, MutableObject<ProgressState> progress,
-            IHyracksClientConnection hcc) throws Exception {
+            IHyracksClientConnection hcc, boolean dropCorrespondingNodeGroup) throws Exception {
         Map<FeedConnectionId, Pair<JobSpecification, Boolean>> disconnectJobList = new HashMap<>();
         if (getDatasetType() == DatasetType.INTERNAL) {
             // prepare job spec(s) that would disconnect any active feeds involving the dataset.
@@ -408,10 +408,11 @@
 
         // #. finally, delete the dataset.
         MetadataManager.INSTANCE.dropDataset(mdTxnCtx.getValue(), dataverseName, datasetName);
-        // Drop the associated nodegroup
-        String nodegroup = getNodeGroupName();
-        if (!nodegroup.equalsIgnoreCase(MetadataConstants.METADATA_DEFAULT_NODEGROUP_NAME)) {
-            MetadataManager.INSTANCE.dropNodegroup(mdTxnCtx.getValue(), nodegroup);
+
+        // Drops the associated nodegroup if it is no longer used by any other dataset.
+        if (dropCorrespondingNodeGroup) {
+            MetadataLockManager.INSTANCE.acquireNodeGroupWriteLock(metadataProvider.getLocks(), nodeGroupName);
+            MetadataManager.INSTANCE.dropNodegroup(mdTxnCtx.getValue(), nodeGroupName, true);
         }
     }
 
diff --git a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/lock/MetadataLockManager.java b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/lock/MetadataLockManager.java
index 6c8999a..9815f6f 100644
--- a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/lock/MetadataLockManager.java
+++ b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/lock/MetadataLockManager.java
@@ -108,8 +108,8 @@
         locks.add(IMetadataLock.Mode.READ, lock);
     }
 
-    public void acquireNodeGroupWriteLock(LockList locks, String dataverseName) throws AsterixException {
-        MetadataLock lock = nodeGroupsLocks.computeIfAbsent(dataverseName, LOCK_FUNCTION);
+    public void acquireNodeGroupWriteLock(LockList locks, String nodeGroupName) throws AsterixException {
+        MetadataLock lock = nodeGroupsLocks.computeIfAbsent(nodeGroupName, LOCK_FUNCTION);
         locks.add(IMetadataLock.Mode.WRITE, lock);
     }
 
@@ -174,7 +174,9 @@
                 && !metaItemTypeFullyQualifiedName.equals(itemTypeFullyQualifiedName)) {
             acquireDataTypeReadLock(locks, metaItemTypeFullyQualifiedName);
         }
-        acquireNodeGroupReadLock(locks, nodeGroupName);
+        if (nodeGroupName != null) {
+            acquireNodeGroupReadLock(locks, nodeGroupName);
+        }
         if (!isDefaultCompactionPolicy) {
             acquireCompactionPolicyReadLock(locks, compactionPolicyName);
         }
diff --git a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/DatasetUtil.java b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/DatasetUtil.java
index 6801427..154e1b5 100644
--- a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/DatasetUtil.java
+++ b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/DatasetUtil.java
@@ -20,8 +20,11 @@
 
 import java.io.DataOutput;
 import java.rmi.RemoteException;
+import java.util.ArrayList;
 import java.util.List;
 import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
 import java.util.logging.Logger;
 
 import org.apache.asterix.builders.IARecordBuilder;
@@ -47,6 +50,8 @@
 import org.apache.asterix.metadata.entities.Dataverse;
 import org.apache.asterix.metadata.entities.Index;
 import org.apache.asterix.metadata.entities.InternalDatasetDetails;
+import org.apache.asterix.metadata.entities.NodeGroup;
+import org.apache.asterix.metadata.lock.MetadataLockManager;
 import org.apache.asterix.om.base.AMutableString;
 import org.apache.asterix.om.base.AString;
 import org.apache.asterix.om.types.ARecordType;
@@ -505,4 +510,53 @@
         int idx = datasetName.indexOf('.');
         return datasetName.substring(0, idx);
     }
+
+    /***
+     * Creates a node group that is associated with a new dataset.
+     *
+     * @param dataverseName,
+     *            the dataverse name of the dataset.
+     * @param datasetName,
+     *            the name of the dataset.
+     * @param ncNames,
+     *            the set of node names.
+     * @param metadataProvider,
+     *            the metadata provider.
+     * @return the name of the created node group.
+     * @throws Exception
+     */
+    public static String createNodeGroupForNewDataset(String dataverseName, String datasetName, Set<String> ncNames,
+            MetadataProvider metadataProvider) throws Exception {
+        return createNodeGroupForNewDataset(dataverseName, datasetName, 0L, ncNames, metadataProvider);
+    }
+
+    /***
+     * Creates a node group that is associated with a new dataset.
+     *
+     * @param dataverseName,
+     *            the dataverse name of the dataset.
+     * @param datasetName,
+     *            the name of the dataset.
+     * @param rebalanceCount
+     *            , the rebalance count of the dataset.
+     * @param ncNames,
+     *            the set of node names.
+     * @param metadataProvider,
+     *            the metadata provider.
+     * @return the name of the created node group.
+     * @throws Exception
+     */
+    public static String createNodeGroupForNewDataset(String dataverseName, String datasetName, long rebalanceCount,
+            Set<String> ncNames, MetadataProvider metadataProvider) throws Exception {
+        String nodeGroup = dataverseName + "." + datasetName + (rebalanceCount == 0L ? "" : "_" + rebalanceCount);
+        MetadataTransactionContext mdTxnCtx = metadataProvider.getMetadataTxnContext();
+        MetadataLockManager.INSTANCE.acquireNodeGroupWriteLock(metadataProvider.getLocks(), nodeGroup);
+        NodeGroup ng = MetadataManager.INSTANCE.getNodegroup(mdTxnCtx, nodeGroup);
+        if (ng != null) {
+            nodeGroup = nodeGroup + "_" + UUID.randomUUID().toString();
+            MetadataLockManager.INSTANCE.acquireNodeGroupWriteLock(metadataProvider.getLocks(), nodeGroup);
+        }
+        MetadataManager.INSTANCE.addNodegroup(mdTxnCtx, new NodeGroup(nodeGroup, new ArrayList<>(ncNames)));
+        return nodeGroup;
+    }
 }
diff --git a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/MetadataConstants.java b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/MetadataConstants.java
index 1dc6657..6769770 100644
--- a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/MetadataConstants.java
+++ b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/MetadataConstants.java
@@ -30,10 +30,6 @@
     // Name of the node group where metadata is stored on.
     public static final String METADATA_NODEGROUP_NAME = "MetadataGroup";
 
-    // Name of the default nodegroup where internal/feed datasets will be partitioned
-    // if an explicit nodegroup is not specified at the time of creation of a dataset
-    public static final String METADATA_DEFAULT_NODEGROUP_NAME = "DEFAULT_NG_ALL_NODES";
-
     private MetadataConstants() {
     }
 }

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1799
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: If68dc6a7c1270ab1f5049c9334e3318425fd8287
Gerrit-PatchSet: 9
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Yingyi Bu <buyingyi@gmail.com>
Gerrit-Reviewer: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Michael Blow <mblow@apache.org>
Gerrit-Reviewer: Steven Jacobs <sjaco002@ucr.edu>
Gerrit-Reviewer: Till Westmann <tillw@apache.org>
Gerrit-Reviewer: Yingyi Bu <buyingyi@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <bamousaa@gmail.com>

Mime
View raw message