asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ian Maxon (Code Review)" <do-not-re...@asterixdb.incubator.apache.org>
Subject Change in asterixdb[master]: Files necessary for initial perf regression tests
Date Thu, 26 May 2016 01:44:28 GMT
Ian Maxon has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/884

Change subject: Files necessary for initial perf regression tests
......................................................................

Files necessary for initial perf regression tests

These are files that are run as part of the asterix-perf job
on our Jenkins CI server. Basically it is a set of queries that
are run using the LSM experiments framework.

Change-Id: I21e2d44ca46a4a28478d9ef256c4fa8d53f03dc9
---
A asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/action/derived/RunSQLPPFileAction.java
A asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/AbstractPerfLoadBuilder.java
A asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/Experiment1EBuilder.java
A asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/PerfTestAggBuilder.java
M asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/client/LSMExperimentSetRunner.java
A asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/client/LSMPerfConstants.java
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-3-months.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-6-months.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-day.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-hour.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-minute.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-month.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-week.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-3-months.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-6-months.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-day.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-hour.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-minute.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-month.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-week.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-3-months.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-6-months.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-day.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-hour.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-minute.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-month.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-week.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-3-months.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-6-months.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-day.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-hour.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-minute.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-month.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-week.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-3-months.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-6-months.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-day.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-hour.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-minute.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-month.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-week.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-3-months.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-6-months.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-day.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-hour.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-minute.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-month.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-week.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-3-months.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-6-months.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-day.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-hour.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-minute.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-month.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-week.sqlpp
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/base/asterix_base_4_ingest.aql
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/base/nuclear_base_4_ingest.aql
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/base/perf_types.aql
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/bench_3.aql
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/bench_3_load.aql
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/bench_count.aql
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/asterix-agg.xml
M asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/asterix-configuration.xml
M asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/managix-conf.xml
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/nuclear.xml
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/dgen/asterix-4.dgen
A asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/dgen/nuc.dgen
68 files changed, 1,442 insertions(+), 131 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/84/884/3

diff --git a/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/action/derived/RunSQLPPFileAction.java b/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/action/derived/RunSQLPPFileAction.java
new file mode 100644
index 0000000..e3969e4
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/action/derived/RunSQLPPFileAction.java
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.asterix.experiment.action.derived;
+
+import org.apache.asterix.experiment.action.base.AbstractAction;
+import org.apache.avro.generic.GenericData;
+import org.apache.commons.httpclient.HttpException;
+import org.apache.commons.httpclient.HttpStatus;
+import org.apache.commons.io.IOUtils;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpResponse;
+import org.apache.http.client.HttpClient;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.util.EntityUtils;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.ByteBuffer;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.text.MessageFormat;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.logging.Level;
+import java.util.logging.Logger;
+
+public class RunSQLPPFileAction extends AbstractAction {
+    private final Logger LOGGER = Logger.getLogger(RunSQLPPFileAction.class.getName());
+    private static final String REST_URI_TEMPLATE = "http://{0}:{1}/sqlpp";
+
+    private final HttpClient httpClient;
+
+    private final Path aqlFilePath;
+    private final List<Path> queriesToRun;
+
+    private final String restHost;
+
+    private final int restPort;
+
+    private final OutputStream os;
+
+    public RunSQLPPFileAction(HttpClient httpClient, String restHost, int restPort, Path aqlFilePath) {
+        this.httpClient = httpClient;
+        this.aqlFilePath = aqlFilePath;
+        this.restHost = restHost;
+        this.restPort = restPort;
+        os = null;
+        queriesToRun = new ArrayList<>();
+    }
+
+    public RunSQLPPFileAction(HttpClient httpClient, String restHost, int restPort, Path aqlFilePath, OutputStream os) {
+        this.httpClient = httpClient;
+        this.aqlFilePath = aqlFilePath;
+        this.restHost = restHost;
+        this.restPort = restPort;
+        this.os = os;
+        queriesToRun = new ArrayList<>();
+    }
+
+    @Override
+    public void doPerform() throws Exception {
+        if(aqlFilePath.toFile().isDirectory()){
+           for(File f: aqlFilePath.toFile().listFiles()){
+               queriesToRun.add(f.toPath());
+           }
+        }
+        else{
+            queriesToRun.add(aqlFilePath);
+        }
+        for(Path p: queriesToRun) {
+            String sqlpp = StandardCharsets.UTF_8.decode(ByteBuffer.wrap(Files.readAllBytes(p))).toString();
+            String uri = MessageFormat.format(REST_URI_TEMPLATE, restHost, String.valueOf(restPort));
+            HttpPost post = new HttpPost(uri);
+            post.setHeader(HttpHeaders.CONTENT_TYPE, "application/json");
+            post.setEntity(new StringEntity(sqlpp, StandardCharsets.UTF_8));
+            long start = System.currentTimeMillis();
+            HttpResponse resp = httpClient.execute(post);
+            HttpEntity entity = resp.getEntity();
+            if(resp.getStatusLine().getStatusCode() != HttpStatus.SC_OK){
+                throw new HttpException("Query returned error");
+            }
+            EntityUtils.consume(entity);
+            long end = System.currentTimeMillis();
+            long wallClock = end-start;
+            System.out.println(p.getFileName().toString() + ',' +wallClock );
+        }
+    }
+}
diff --git a/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/AbstractPerfLoadBuilder.java b/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/AbstractPerfLoadBuilder.java
new file mode 100644
index 0000000..4bf0c21
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/AbstractPerfLoadBuilder.java
@@ -0,0 +1,286 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.asterix.experiment.builder;
+
+import org.apache.asterix.event.schema.cluster.Cluster;
+import org.apache.asterix.experiment.action.base.ParallelActionSet;
+import org.apache.asterix.experiment.action.base.SequentialActionList;
+import org.apache.asterix.experiment.action.derived.*;
+import org.apache.asterix.experiment.action.derived.ManagixActions.CreateAsterixManagixAction;
+import org.apache.asterix.experiment.action.derived.ManagixActions.DeleteAsterixManagixAction;
+import org.apache.asterix.experiment.action.derived.ManagixActions.LogAsterixManagixAction;
+import org.apache.asterix.experiment.action.derived.ManagixActions.StopAsterixManagixAction;
+import org.apache.asterix.experiment.client.LSMExperimentConstants;
+import org.apache.asterix.experiment.client.LSMExperimentSetRunner.LSMExperimentSetRunnerConfig;
+import org.apache.asterix.experiment.client.LSMPerfConstants;
+import org.apache.http.client.HttpClient;
+import org.apache.http.impl.client.DefaultHttpClient;
+
+import javax.xml.bind.JAXBContext;
+import javax.xml.bind.Unmarshaller;
+import java.io.File;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.*;
+
+/**
+ * This class is used to create experiments for spatial index static data evaluation, that is, no ingestion is involved.
+ * Also, there is no orchestration server involved in this experiment builder.
+ */
+public abstract class AbstractPerfLoadBuilder extends AbstractExperimentBuilder {
+
+    private static final String ASTERIX_INSTANCE_NAME = "a1";
+
+    private final String logDirSuffix;
+
+    protected final HttpClient httpClient;
+
+    protected final String restHost;
+
+    protected final int restPort;
+
+    private final String managixHomePath;
+
+    protected final String javaHomePath;
+
+    protected final Path localExperimentRoot;
+
+    protected final String username;
+
+    protected final String sshKeyLocation;
+
+    private final String clusterConfigFileName;
+
+    protected final String dgenFileName;
+
+    private final String countFileName;
+
+    private final String statFile;
+
+    protected final SequentialActionList lsAction;
+
+    protected final String openStreetMapFilePath;
+
+    protected final int locationSampleInterval;
+
+    protected final String loadAQLFilePath;
+
+    protected final String querySQLPPFileName;
+
+    public AbstractPerfLoadBuilder(String name, LSMExperimentSetRunnerConfig config,
+                                   String clusterConfigFileName, String ingestFileName, String dgenFileName, String countFileName,
+                                   String loadAQLFileName, String querySQLPPFileName) {
+        super(name);
+        this.logDirSuffix = config.getLogDirSuffix();
+        this.httpClient = new DefaultHttpClient();
+        this.restHost = config.getRESTHost();
+        this.restPort = config.getRESTPort();
+        this.managixHomePath = config.getManagixHome();
+        this.javaHomePath = config.getJavaHome();
+        this.localExperimentRoot = Paths.get(config.getLocalExperimentRoot());
+        this.username = config.getUsername();
+        this.sshKeyLocation = config.getSSHKeyLocation();
+        this.clusterConfigFileName = clusterConfigFileName;
+        this.dgenFileName = dgenFileName;
+        this.countFileName = countFileName;
+        this.statFile = config.getStatFile();
+        this.lsAction = new SequentialActionList();
+        this.openStreetMapFilePath = config.getOpenStreetMapFilePath();
+        this.locationSampleInterval = config.getLocationSampleInterval();
+        this.loadAQLFilePath = loadAQLFileName;
+        this.querySQLPPFileName = querySQLPPFileName;
+    }
+
+    protected abstract void doBuildDDL(SequentialActionList seq);
+
+    protected void doPost(SequentialActionList seq) {
+    }
+
+    protected void doBuildDataGen(SequentialActionList seq, final Map<String, List<String>> dgenPairs) throws Exception {
+    }
+
+    @Override
+    protected void doBuild(Experiment e) throws Exception {
+        SequentialActionList execs = new SequentialActionList();
+
+        String clusterConfigPath = localExperimentRoot.resolve(LSMExperimentConstants.CONFIG_DIR)
+                .resolve(clusterConfigFileName).toString();
+        String asterixConfigPath = localExperimentRoot.resolve(LSMExperimentConstants.CONFIG_DIR)
+                .resolve(LSMExperimentConstants.ASTERIX_CONFIGURATION).toString();
+
+        //stop/delete/create instance
+        execs.add(new StopAsterixManagixAction(managixHomePath, ASTERIX_INSTANCE_NAME));
+        execs.add(new DeleteAsterixManagixAction(managixHomePath, ASTERIX_INSTANCE_NAME));
+        execs.add(new SleepAction(30000));
+        execs.add(new CreateAsterixManagixAction(managixHomePath, ASTERIX_INSTANCE_NAME, clusterConfigPath,
+                asterixConfigPath));
+
+        //ddl statements
+        execs.add(new SleepAction(15000));
+        // TODO: implement retry handler
+        execs.add(new RunAQLFileAction(httpClient, restHost, restPort, localExperimentRoot.resolve(
+                LSMExperimentConstants.AQL_DIR).resolve(LSMPerfConstants.BASE_TYPES)));
+        doBuildDDL(execs);
+
+        //prepare io state action in NC node(s)
+        Map<String, List<String>> dgenPairs = readDatagenPairs(localExperimentRoot.resolve(
+                LSMExperimentConstants.DGEN_DIR).resolve(dgenFileName));
+        final Set<String> ncHosts = new HashSet<>();
+        for (List<String> ncHostList : dgenPairs.values()) {
+            for (String ncHost : ncHostList) {
+                ncHosts.add(ncHost.split(":")[0]);
+            }
+        }
+        if (statFile != null) {
+            ParallelActionSet ioCountActions = new ParallelActionSet();
+            for (String ncHost : ncHosts) {
+                ioCountActions.add(new AbstractRemoteExecutableAction(ncHost, username, sshKeyLocation) {
+
+                    @Override
+                    protected String getCommand() {
+                        String cmd = "screen -d -m sh -c \"sar -b -u 1 > " + statFile + "\"";
+                        return cmd;
+                    }
+                });
+            }
+            execs.add(ioCountActions);
+        }
+
+        //prepare post ls action
+        SequentialActionList postLSAction = new SequentialActionList();
+        File file = new File(clusterConfigPath);
+        JAXBContext ctx = JAXBContext.newInstance(Cluster.class);
+        Unmarshaller unmarshaller = ctx.createUnmarshaller();
+        final Cluster cluster = (Cluster) unmarshaller.unmarshal(file);
+        String[] storageRoots = cluster.getIodevices().split(",");
+        for (String ncHost : ncHosts) {
+            for (final String sRoot : storageRoots) {
+                lsAction.add(new AbstractRemoteExecutableAction(ncHost, username, sshKeyLocation) {
+                    @Override
+                    protected String getCommand() {
+                        return "ls -Rl " + sRoot;
+                    }
+                });
+                postLSAction.add(new AbstractRemoteExecutableAction(ncHost, username, sshKeyLocation) {
+                    @Override
+                    protected String getCommand() {
+                        return "ls -Rl " + sRoot;
+                    }
+                });
+
+            }
+        }
+
+        //---------- main experiment body begins -----------
+
+        //load data into pidx 
+        execs.add(new TimedAction(new RunAQLFileAction(httpClient, restHost, restPort, localExperimentRoot.resolve(
+                LSMExperimentConstants.AQL_DIR).resolve(loadAQLFilePath))));
+
+        //load data into pidx
+        execs.add(new TimedAction(new RunSQLPPFileAction(httpClient, restHost, restPort, localExperimentRoot.resolve(
+                LSMExperimentConstants.AQL_DIR).resolve(querySQLPPFileName))));
+
+        //---------- main experiment body ends -----------
+
+        //kill io state action
+        if (statFile != null) {
+            ParallelActionSet ioCountKillActions = new ParallelActionSet();
+            for (String ncHost : ncHosts) {
+                ioCountKillActions.add(new AbstractRemoteExecutableAction(ncHost, username, sshKeyLocation) {
+
+                    @Override
+                    protected String getCommand() {
+                        String cmd = "screen -X -S `screen -list | grep Detached | awk '{print $1}'` quit";
+                        return cmd;
+                    }
+                });
+            }
+            execs.add(ioCountKillActions);
+        }
+
+        //total record count
+        execs.add(new SleepAction(10000));
+        if (countFileName != null) {
+            execs.add(new RunAQLFileAction(httpClient, restHost, restPort, localExperimentRoot.resolve(
+                    LSMExperimentConstants.AQL_DIR).resolve(countFileName)));
+        }
+
+        //add ls action
+        execs.add(postLSAction);
+
+        //kill asterix cc and nc
+        /*
+        ParallelActionSet killCmds = new ParallelActionSet();
+        for (String ncHost : ncHosts) {
+            killCmds.add(new RemoteAsterixDriverKill(ncHost, username, sshKeyLocation));
+        }
+        killCmds.add(new RemoteAsterixDriverKill(restHost, username, sshKeyLocation));
+        execs.add(killCmds);
+        */
+
+        //stop asterix instance
+        execs.add(new StopAsterixManagixAction(managixHomePath, ASTERIX_INSTANCE_NAME));
+
+        //prepare to collect io state by putting the state file into asterix log dir
+        if (statFile != null) {
+            ParallelActionSet collectIOActions = new ParallelActionSet();
+            for (String ncHost : ncHosts) {
+                collectIOActions.add(new AbstractRemoteExecutableAction(ncHost, username, sshKeyLocation) {
+
+                    @Override
+                    protected String getCommand() {
+                        String cmd = "cp " + statFile + " " + cluster.getLogDir();
+                        return cmd;
+                    }
+                });
+            }
+            execs.add(collectIOActions);
+        }
+
+        //collect cc and nc logs
+        execs.add(new LogAsterixManagixAction(managixHomePath, ASTERIX_INSTANCE_NAME, localExperimentRoot
+                .resolve(LSMExperimentConstants.LOG_DIR + "-" + logDirSuffix).resolve(getName()).toString()));
+
+        e.addBody(execs);
+    }
+
+    protected Map<String, List<String>> readDatagenPairs(Path p) throws IOException {
+        Map<String, List<String>> dgenPairs = new HashMap<>();
+        Scanner s = new Scanner(p, StandardCharsets.UTF_8.name());
+        try {
+            while (s.hasNextLine()) {
+                String line = s.nextLine();
+                String[] pair = line.split("\\s+");
+                List<String> vals = dgenPairs.get(pair[0]);
+                if (vals == null) {
+                    vals = new ArrayList<>();
+                    dgenPairs.put(pair[0], vals);
+                }
+                vals.add(pair[1]);
+            }
+        } finally {
+            s.close();
+        }
+        return dgenPairs;
+    }
+}
diff --git a/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/Experiment1EBuilder.java b/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/Experiment1EBuilder.java
new file mode 100644
index 0000000..c2d30a5
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/Experiment1EBuilder.java
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.asterix.experiment.builder;
+
+import org.apache.asterix.experiment.client.LSMExperimentSetRunner.LSMExperimentSetRunnerConfig;
+
+public class Experiment1EBuilder extends AbstractExperiment1Builder {
+
+    public Experiment1EBuilder(LSMExperimentSetRunnerConfig config) {
+        super("1E", config, "nuclear.xml", "nuclear_base_4_ingest.aql", "nuc.dgen");
+    }
+
+}
diff --git a/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/PerfTestAggBuilder.java b/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/PerfTestAggBuilder.java
new file mode 100644
index 0000000..ee5ba01
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/PerfTestAggBuilder.java
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.asterix.experiment.builder;
+
+import org.apache.asterix.experiment.action.base.SequentialActionList;
+import org.apache.asterix.experiment.action.derived.RunAQLFileAction;
+import org.apache.asterix.experiment.client.LSMExperimentConstants;
+import org.apache.asterix.experiment.client.LSMExperimentSetRunner.LSMExperimentSetRunnerConfig;
+
+public class PerfTestAggBuilder extends AbstractPerfLoadBuilder {
+
+    public PerfTestAggBuilder(LSMExperimentSetRunnerConfig config) {
+        super("PerfTestAggBuilder", config, "asterix-agg.xml", "asterix_base_4_ingest.aql", "asterix-4.dgen", "bench_count.aql", "bench_3_load.aql", "agg_bench");
+    }
+
+    @Override
+    protected void doBuildDDL(SequentialActionList seq) {
+        seq.add(new RunAQLFileAction(httpClient, restHost, restPort, localExperimentRoot.resolve(
+                LSMExperimentConstants.AQL_DIR).resolve("bench_3.aql")));
+    }
+
+}
diff --git a/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/client/LSMExperimentSetRunner.java b/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/client/LSMExperimentSetRunner.java
index 8b59449..dd7e855 100644
--- a/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/client/LSMExperimentSetRunner.java
+++ b/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/client/LSMExperimentSetRunner.java
@@ -26,48 +26,7 @@
 import java.util.regex.Pattern;
 
 import org.apache.asterix.experiment.action.base.SequentialActionList;
-import org.apache.asterix.experiment.builder.AbstractExperimentBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1ADhbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1ADhvbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1ARtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1AShbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1ASifBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1BDhbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1BDhvbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1BRtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1BShbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1BSifBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1CDhbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1CDhvbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1CRtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1CShbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1CSifBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1DDhbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1DDhvbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1DRtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1DShbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment1DSifBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment2DhbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment2DhvbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment2RtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment2ShbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment2SifBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment3DhbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment3DhvbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment3PIdxLoadBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment3RtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment3ShbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment3SifBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment4DhbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment4DhvbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment4RtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment4ShbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment4SifBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment5DhbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment5DhvbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment5RtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment5ShbtreeBuilder;
-import org.apache.asterix.experiment.builder.SpatialIndexExperiment5SifBuilder;
+import org.apache.asterix.experiment.builder.*;
 import org.kohsuke.args4j.CmdLineException;
 import org.kohsuke.args4j.CmdLineParser;
 import org.kohsuke.args4j.Option;
@@ -279,91 +238,52 @@
 
         Collection<AbstractExperimentBuilder> suite = new ArrayList<>();
 
-        suite.add(new SpatialIndexExperiment1ADhbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment1ADhvbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment1ARtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment1AShbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment1ASifBuilder(config));
-        suite.add(new SpatialIndexExperiment1BDhbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment1BDhvbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment1BRtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment1BShbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment1BSifBuilder(config));
-        suite.add(new SpatialIndexExperiment1CDhbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment1CDhvbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment1CRtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment1CShbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment1CSifBuilder(config));
-        suite.add(new SpatialIndexExperiment1DDhbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment1DDhvbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment1DRtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment1DShbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment1DSifBuilder(config));
-        suite.add(new SpatialIndexExperiment2DhbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment2DhvbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment2RtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment2ShbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment2SifBuilder(config));
-        suite.add(new SpatialIndexExperiment3PIdxLoadBuilder(config));
-        suite.add(new SpatialIndexExperiment3DhbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment3DhvbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment3RtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment3ShbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment3SifBuilder(config));
-        suite.add(new SpatialIndexExperiment4DhbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment4DhvbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment4RtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment4ShbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment4SifBuilder(config));
-        suite.add(new SpatialIndexExperiment5DhbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment5DhvbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment5RtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment5ShbtreeBuilder(config));
-        suite.add(new SpatialIndexExperiment5SifBuilder(config));
-        //        suite.add(new Experiment7BBuilder(config));
-        //        suite.add(new Experiment7DBuilder(config));
-        //        suite.add(new Experiment7ABuilder(config));
-        //        suite.add(new Experiment8DBuilder(config));
-        //        suite.add(new Experiment8ABuilder(config));
-        //        suite.add(new Experiment8BBuilder(config));
-        //        suite.add(new Experiment9ABuilder(config));
-        //        suite.add(new Experiment9DBuilder(config));
-        //        suite.add(new Experiment9BBuilder(config));
-        //        suite.add(new Experiment6ABuilder(config));
-        //        suite.add(new Experiment6BBuilder(config));
-        //        suite.add(new Experiment6CBuilder(config));
-        //        suite.add(new Experiment2D1Builder(config));
-        //        suite.add(new Experiment2D2Builder(config));
-        //        suite.add(new Experiment2D4Builder(config));
-        //        suite.add(new Experiment2D8Builder(config));
-        //        suite.add(new Experiment2C1Builder(config));
-        //        suite.add(new Experiment2C2Builder(config));
-        //        suite.add(new Experiment2C4Builder(config));
-        //        suite.add(new Experiment2C8Builder(config));
-        //        suite.add(new Experiment2A1Builder(config));
-        //        suite.add(new Experiment2A2Builder(config));
-        //        suite.add(new Experiment2A4Builder(config));
-        //        suite.add(new Experiment2A8Builder(config));
-        //        suite.add(new Experiment2B1Builder(config));
-        //        suite.add(new Experiment2B2Builder(config));
-        //        suite.add(new Experiment2B4Builder(config));
-        //        suite.add(new Experiment2B8Builder(config));
-        //        suite.add(new Experiment1ABuilder(config));
-        //        suite.add(new Experiment1BBuilder(config));
-        //        suite.add(new Experiment1CBuilder(config));
-        //        suite.add(new Experiment1DBuilder(config));
-        //        suite.add(new Experiment4ABuilder(config));
-        //        suite.add(new Experiment4BBuilder(config));
-        //        suite.add(new Experiment4CBuilder(config));
-        //        suite.add(new Experiment4DBuilder(config));
-        //        suite.add(new Experiment3ABuilder(config));
-        //        suite.add(new Experiment3BBuilder(config));
-        //        suite.add(new Experiment3CBuilder(config));
-        //        suite.add(new Experiment3DBuilder(config));
-        //        suite.add(new Experiment5ABuilder(config));
-        //        suite.add(new Experiment5BBuilder(config));
-        //        suite.add(new Experiment5CBuilder(config));
-        //        suite.add(new Experiment5DBuilder(config));
+                suite.add(new Experiment7BBuilder(config));
+                suite.add(new Experiment7DBuilder(config));
+                suite.add(new Experiment7ABuilder(config));
+                suite.add(new Experiment8DBuilder(config));
+                suite.add(new Experiment8ABuilder(config));
+                suite.add(new Experiment8BBuilder(config));
+                suite.add(new Experiment9ABuilder(config));
+                suite.add(new Experiment9DBuilder(config));
+                suite.add(new Experiment9BBuilder(config));
+                suite.add(new Experiment6ABuilder(config));
+                suite.add(new Experiment6BBuilder(config));
+                suite.add(new Experiment6CBuilder(config));
+                suite.add(new Experiment2D1Builder(config));
+                suite.add(new Experiment2D2Builder(config));
+                suite.add(new Experiment2D4Builder(config));
+                suite.add(new Experiment2D8Builder(config));
+                suite.add(new Experiment2C1Builder(config));
+                suite.add(new Experiment2C2Builder(config));
+                suite.add(new Experiment2C4Builder(config));
+                suite.add(new Experiment2C8Builder(config));
+                suite.add(new Experiment2A1Builder(config));
+                suite.add(new Experiment2A2Builder(config));
+                suite.add(new Experiment2A4Builder(config));
+                suite.add(new Experiment2A8Builder(config));
+                suite.add(new Experiment2B1Builder(config));
+                suite.add(new Experiment2B2Builder(config));
+                suite.add(new Experiment2B4Builder(config));
+                suite.add(new Experiment2B8Builder(config));
+                suite.add(new Experiment1ABuilder(config));
+                suite.add(new Experiment1BBuilder(config));
+                suite.add(new Experiment1CBuilder(config));
+                suite.add(new Experiment1DBuilder(config));
+                suite.add(new Experiment1EBuilder(config));
+                suite.add(new Experiment4ABuilder(config));
+                suite.add(new Experiment4BBuilder(config));
+                suite.add(new Experiment4CBuilder(config));
+                suite.add(new Experiment4DBuilder(config));
+                suite.add(new Experiment3ABuilder(config));
+                suite.add(new Experiment3BBuilder(config));
+                suite.add(new Experiment3CBuilder(config));
+                suite.add(new Experiment3DBuilder(config));
+                suite.add(new Experiment5ABuilder(config));
+                suite.add(new Experiment5BBuilder(config));
+                suite.add(new Experiment5CBuilder(config));
+                suite.add(new Experiment5DBuilder(config));
+                suite.add(new PerfTestAggBuilder(config));
 
         Pattern p = config.getRegex() == null ? null : Pattern.compile(config.getRegex());
 
diff --git a/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/client/LSMPerfConstants.java b/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/client/LSMPerfConstants.java
new file mode 100644
index 0000000..a136577
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/client/LSMPerfConstants.java
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.asterix.experiment.client;
+
+public class LSMPerfConstants {
+
+    public static final String CONFIG_DIR = "configs";
+
+    public static final String AQL_DIR = "aql";
+
+    public static final String BASE_DIR = "base";
+
+    public static final String DGEN_DIR = "dgen";
+
+    public static final String LOG_DIR = "log";
+
+    public static final String BASE_TYPES = "base/perf_types.aql";
+
+    public static final String ASTERIX_CONFIGURATION = "asterix-configuration.xml";
+}
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg.sqlpp
new file mode 100644
index 0000000..7583275
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg.sqlpp
@@ -0,0 +1,9 @@
+use experiments;
+
+select element coll_avg((
+select element "string-length"(g.message)
+from GleambookMessages as g
+where g.send_time > datetime('2013-07-16T14:31:31') and
+      g.send_time <= datetime('2014-01-12T14:31:31')
+));
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-3-months.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-3-months.sqlpp
new file mode 100644
index 0000000..f98608c
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-3-months.sqlpp
@@ -0,0 +1,9 @@
+use SocialNetworkData;
+
+select element coll_avg((
+select element "string-length"(g.message) 
+from GleambookMessages as g
+where g.send_time > datetime('2007-12-24T16:03:59') and
+      g.send_time <= datetime('2008-03-23T16:03:59')
+));
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-6-months.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-6-months.sqlpp
new file mode 100644
index 0000000..b0e9301
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-6-months.sqlpp
@@ -0,0 +1,9 @@
+use SocialNetworkData;
+
+select element coll_avg((
+select element "string-length"(g.message) 
+from GleambookMessages as g
+where g.send_time > datetime('2013-07-16T14:31:31') and
+      g.send_time <= datetime('2014-01-12T14:31:31')
+));
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-day.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-day.sqlpp
new file mode 100644
index 0000000..219468a
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-day.sqlpp
@@ -0,0 +1,9 @@
+use SocialNetworkData;
+
+select element coll_avg((
+select element "string-length"(g.message) 
+from GleambookMessages as g
+where g.send_time > datetime('2010-10-10T03:13:47') and
+      g.send_time <= datetime('2010-10-11T03:13:47')
+));
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-hour.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-hour.sqlpp
new file mode 100644
index 0000000..174920e
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-hour.sqlpp
@@ -0,0 +1,9 @@
+use SocialNetworkData;
+
+select element coll_avg((
+select element "string-length"(g.message) 
+from GleambookMessages as g
+where g.send_time > datetime('2007-05-26T08:58:39') and
+      g.send_time <= datetime('2007-05-26T09:58:39')
+));
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-minute.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-minute.sqlpp
new file mode 100644
index 0000000..7a06dc0
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-minute.sqlpp
@@ -0,0 +1,9 @@
+use SocialNetworkData;
+
+select element coll_avg((
+select element "string-length"(g.message) 
+from GleambookMessages as g
+where g.send_time > datetime('2001-05-10T12:16:12') and
+      g.send_time <= datetime('2001-05-10T12:17:12')
+));
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-month.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-month.sqlpp
new file mode 100644
index 0000000..6c0909e
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-month.sqlpp
@@ -0,0 +1,9 @@
+use SocialNetworkData;
+
+select element coll_avg((
+select element "string-length"(g.message) 
+from GleambookMessages as g
+where g.send_time > datetime('2007-03-01T21:43:22') and
+      g.send_time <= datetime('2007-03-31T21:43:22')
+));
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-week.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-week.sqlpp
new file mode 100644
index 0000000..22517d6
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/agg-week.sqlpp
@@ -0,0 +1,9 @@
+use SocialNetworkData;
+
+select element coll_avg((
+select element "string-length"(g.message) 
+from GleambookMessages as g
+where g.send_time > datetime('2008-10-10T21:33:13') and
+      g.send_time <= datetime('2008-10-17T21:33:13')
+));
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-3-months.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-3-months.sqlpp
new file mode 100644
index 0000000..2c706a7
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-3-months.sqlpp
@@ -0,0 +1,18 @@
+use SocialNetworkData;
+
+coll_count(
+ (
+  select element {
+  'uname': user.name,
+  'alias': user.alias,
+  'send_time': message.send_time,
+  'message': substring(message.message, 1, 31)
+  }
+  from  GleambookUsers as user,
+        GleambookMessages as message
+  where message.author_id = user.id and
+      message.send_time >= datetime('2007-03-16T01:34:15') and
+      message.send_time < datetime('2007-06-14T01:34:15')
+ )
+);
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-6-months.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-6-months.sqlpp
new file mode 100644
index 0000000..48f9832
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-6-months.sqlpp
@@ -0,0 +1,18 @@
+use SocialNetworkData;
+
+coll_count(
+ (
+  select element {
+  'uname': user.name,
+  'alias': user.alias,
+  'send_time': message.send_time,
+  'message': substring(message.message, 1, 31)
+  }
+  from  GleambookUsers as user,
+        GleambookMessages as message
+  where message.author_id = user.id and
+      message.send_time >= datetime('2009-10-31T05:05:19') and
+      message.send_time < datetime('2010-04-29T05:05:19')
+ )
+);
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-day.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-day.sqlpp
new file mode 100644
index 0000000..29ae25c
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-day.sqlpp
@@ -0,0 +1,18 @@
+use SocialNetworkData;
+
+coll_count(
+ (
+  select element {
+  'uname': user.name,
+  'alias': user.alias,
+  'send_time': message.send_time,
+  'message': substring(message.message, 1, 31)
+  }
+  from  GleambookUsers as user,
+        GleambookMessages as message
+  where message.author_id = user.id and
+      message.send_time >= datetime('2008-08-22T16:20:12') and
+      message.send_time < datetime('2008-08-23T16:20:12')
+ )
+);
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-hour.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-hour.sqlpp
new file mode 100644
index 0000000..3f4ef30
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-hour.sqlpp
@@ -0,0 +1,18 @@
+use SocialNetworkData;
+
+coll_count(
+ (
+  select element {
+  'uname': user.name,
+  'alias': user.alias,
+  'send_time': message.send_time,
+  'message': substring(message.message, 1, 31)
+  }
+  from  GleambookUsers as user,
+        GleambookMessages as message
+  where message.author_id = user.id and
+      message.send_time >= datetime('2012-04-21T12:40:53') and
+      message.send_time < datetime('2012-04-21T13:40:53')
+ )
+);
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-minute.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-minute.sqlpp
new file mode 100644
index 0000000..6e677ce
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-minute.sqlpp
@@ -0,0 +1,18 @@
+use SocialNetworkData;
+
+coll_count(
+ (
+  select element {
+  'uname': user.name,
+  'alias': user.alias,
+  'send_time': message.send_time,
+  'message': substring(message.message, 1, 31)
+  }
+  from  GleambookUsers as user,
+        GleambookMessages as message
+  where message.author_id = user.id and
+      message.send_time >= datetime('2006-04-15T23:53:14') and
+      message.send_time < datetime('2006-04-15T23:54:14')
+ )
+);
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-month.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-month.sqlpp
new file mode 100644
index 0000000..12183c8
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-month.sqlpp
@@ -0,0 +1,18 @@
+use SocialNetworkData;
+
+coll_count(
+ (
+  select element {
+  'uname': user.name,
+  'alias': user.alias,
+  'send_time': message.send_time,
+  'message': substring(message.message, 1, 31)
+  }
+  from  GleambookUsers as user,
+        GleambookMessages as message
+  where message.author_id = user.id and
+      message.send_time >= datetime('2003-02-22T05:05:02') and
+      message.send_time < datetime('2003-03-24T05:05:02')
+ )
+);
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-week.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-week.sqlpp
new file mode 100644
index 0000000..5529798
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin-week.sqlpp
@@ -0,0 +1,18 @@
+use SocialNetworkData;
+
+coll_count(
+ (
+  select element {
+  'uname': user.name,
+  'alias': user.alias,
+  'send_time': message.send_time,
+  'message': substring(message.message, 1, 31)
+  }
+  from  GleambookUsers as user,
+        GleambookMessages as message
+  where message.author_id = user.id and
+      message.send_time >= datetime('2008-08-08T15:24:25') and
+      message.send_time < datetime('2008-08-15T15:24:25')
+ )
+);
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-3-months.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-3-months.sqlpp
new file mode 100644
index 0000000..d44974d
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-3-months.sqlpp
@@ -0,0 +1,18 @@
+use SocialNetworkData;
+
+coll_count(
+ (
+  select element {
+  'uname': user.name,
+  'alias': user.alias,
+  'send_time': message.send_time,
+  'message': substring(message.message, 1, 31)
+  }
+  from  GleambookMessages as message,
+        GleambookUsers as user
+  where message.author_id /*+ indexnl */ = user.id and
+      message.send_time >= datetime('2007-03-16T01:34:15') and
+      message.send_time < datetime('2007-06-14T01:34:15')
+ )
+);
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-6-months.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-6-months.sqlpp
new file mode 100644
index 0000000..0eb1801
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-6-months.sqlpp
@@ -0,0 +1,18 @@
+use SocialNetworkData;
+
+coll_count(
+ (
+  select element {
+  'uname': user.name,
+  'alias': user.alias,
+  'send_time': message.send_time,
+  'message': substring(message.message, 1, 31)
+  }
+  from  GleambookMessages as message,
+        GleambookUsers as user
+  where message.author_id /*+ indexnl */ = user.id and
+      message.send_time >= datetime('2009-10-31T05:05:19') and
+      message.send_time < datetime('2010-04-29T05:05:19')
+ )
+);
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-day.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-day.sqlpp
new file mode 100644
index 0000000..5d8ce0c
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-day.sqlpp
@@ -0,0 +1,18 @@
+use SocialNetworkData;
+
+coll_count(
+ (
+  select element {
+  'uname': user.name,
+  'alias': user.alias,
+  'send_time': message.send_time,
+  'message': substring(message.message, 1, 31)
+  }
+  from  GleambookMessages as message,
+        GleambookUsers as user
+  where message.author_id /*+ indexnl */ = user.id and
+      message.send_time >= datetime('2008-08-22T16:20:12') and
+      message.send_time < datetime('2008-08-23T16:20:12')
+ )
+);
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-hour.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-hour.sqlpp
new file mode 100644
index 0000000..3e37d03
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-hour.sqlpp
@@ -0,0 +1,18 @@
+use SocialNetworkData;
+
+coll_count(
+ (
+  select element {
+  'uname': user.name,
+  'alias': user.alias,
+  'send_time': message.send_time,
+  'message': substring(message.message, 1, 31)
+  }
+  from  GleambookMessages as message,
+        GleambookUsers as user
+  where message.author_id /*+ indexnl */ = user.id and
+      message.send_time >= datetime('2012-04-21T12:40:53') and
+      message.send_time < datetime('2012-04-21T13:40:53')
+ )
+);
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-minute.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-minute.sqlpp
new file mode 100644
index 0000000..5c9faf4
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-minute.sqlpp
@@ -0,0 +1,18 @@
+use SocialNetworkData;
+
+coll_count(
+ (
+  select element {
+  'uname': user.name,
+  'alias': user.alias,
+  'send_time': message.send_time,
+  'message': substring(message.message, 1, 31)
+  }
+  from  GleambookMessages as message,
+        GleambookUsers as user
+  where message.author_id /*+ indexnl */ = user.id and
+      message.send_time >= datetime('2006-04-15T23:53:14') and
+      message.send_time < datetime('2006-04-15T23:54:14')
+ )
+);
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-month.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-month.sqlpp
new file mode 100644
index 0000000..18d780d
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-month.sqlpp
@@ -0,0 +1,18 @@
+use SocialNetworkData;
+
+coll_count(
+ (
+  select element {
+  'uname': user.name,
+  'alias': user.alias,
+  'send_time': message.send_time,
+  'message': substring(message.message, 1, 31)
+  }
+  from  GleambookMessages as message,
+        GleambookUsers as user
+  where message.author_id /*+ indexnl */ = user.id and
+      message.send_time >= datetime('2003-02-22T05:05:02') and
+      message.send_time < datetime('2003-03-24T05:05:02')
+ )
+);
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-week.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-week.sqlpp
new file mode 100644
index 0000000..4813f0b
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/countjoin_index-week.sqlpp
@@ -0,0 +1,18 @@
+use SocialNetworkData;
+
+coll_count(
+ (
+  select element {
+  'uname': user.name,
+  'alias': user.alias,
+  'send_time': message.send_time,
+  'message': substring(message.message, 1, 31)
+  }
+  from  GleambookMessages as message,
+        GleambookUsers as user
+  where message.author_id /*+ indexnl */ = user.id and
+      message.send_time >= datetime('2008-08-08T15:24:25') and
+      message.send_time < datetime('2008-08-15T15:24:25')
+ )
+);
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-3-months.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-3-months.sqlpp
new file mode 100644
index 0000000..5777a57
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-3-months.sqlpp
@@ -0,0 +1,14 @@
+use SocialNetworkData;
+
+select element {
+'uname': user.name,
+'alias': user.alias,
+'send_time': message.send_time,
+'message': substring(message.message, 1, 31)
+}
+from GleambookUsers as user,
+     GleambookMessages as message
+where message.author_id = user.id and
+      message.send_time >= datetime('2000-04-03T17:47:01') and
+      message.send_time < datetime('2000-07-02T17:47:01');
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-6-months.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-6-months.sqlpp
new file mode 100644
index 0000000..c0ef25f
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-6-months.sqlpp
@@ -0,0 +1,14 @@
+use SocialNetworkData;
+
+select element {
+'uname': user.name,
+'alias': user.alias,
+'send_time': message.send_time,
+'message': substring(message.message, 1, 31)
+}
+from GleambookUsers as user,
+     GleambookMessages as message
+where message.author_id = user.id and
+      message.send_time >= datetime('2001-10-22T09:18:11') and
+      message.send_time < datetime('2002-04-20T09:18:11');
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-day.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-day.sqlpp
new file mode 100644
index 0000000..5433272
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-day.sqlpp
@@ -0,0 +1,14 @@
+use SocialNetworkData;
+
+select element {
+'uname': user.name,
+'alias': user.alias,
+'send_time': message.send_time,
+'message': substring(message.message, 1, 31)
+}
+from GleambookUsers as user,
+     GleambookMessages as message
+where message.author_id = user.id and
+      message.send_time >= datetime('2002-08-25T10:53:14') and
+      message.send_time < datetime('2002-08-26T10:53:14');
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-hour.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-hour.sqlpp
new file mode 100644
index 0000000..516dd23
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-hour.sqlpp
@@ -0,0 +1,14 @@
+use SocialNetworkData;
+
+select element {
+'uname': user.name,
+'alias': user.alias,
+'send_time': message.send_time,
+'message': substring(message.message, 1, 31)
+}
+from GleambookUsers as user,
+     GleambookMessages as message
+where message.author_id = user.id and
+      message.send_time >= datetime('2000-03-28T13:29:53') and
+      message.send_time < datetime('2000-03-28T14:29:53');
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-minute.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-minute.sqlpp
new file mode 100644
index 0000000..8dac96c
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-minute.sqlpp
@@ -0,0 +1,14 @@
+use SocialNetworkData;
+
+select element {
+'uname': user.name,
+'alias': user.alias,
+'send_time': message.send_time,
+'message': substring(message.message, 1, 31)
+}
+from GleambookUsers as user,
+     GleambookMessages as message
+where message.author_id = user.id and
+      message.send_time >= datetime('2013-08-31T04:21:21') and
+      message.send_time < datetime('2013-08-31T04:22:21');
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-month.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-month.sqlpp
new file mode 100644
index 0000000..830fae8
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-month.sqlpp
@@ -0,0 +1,14 @@
+use SocialNetworkData;
+
+select element {
+'uname': user.name,
+'alias': user.alias,
+'send_time': message.send_time,
+'message': substring(message.message, 1, 31)
+}
+from GleambookUsers as user,
+     GleambookMessages as message
+where message.author_id = user.id and
+      message.send_time >= datetime('2014-01-27T06:38:19') and
+      message.send_time < datetime('2014-02-26T06:38:19');
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-week.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-week.sqlpp
new file mode 100644
index 0000000..281492d
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join-week.sqlpp
@@ -0,0 +1,14 @@
+use SocialNetworkData;
+
+select element {
+'uname': user.name,
+'alias': user.alias,
+'send_time': message.send_time,
+'message': substring(message.message, 1, 31)
+}
+from GleambookUsers as user,
+     GleambookMessages as message
+where message.author_id = user.id and
+      message.send_time >= datetime('2012-08-03T11:02:50') and
+      message.send_time < datetime('2012-08-10T11:02:50');
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-3-months.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-3-months.sqlpp
new file mode 100644
index 0000000..dbd61a0
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-3-months.sqlpp
@@ -0,0 +1,14 @@
+use SocialNetworkData;
+
+select element {
+'uname': user.name,
+'alias': user.alias,
+'send_time': message.send_time,
+'message': substring(message.message, 1, 31)
+}
+from GleambookMessages as message,
+     GleambookUsers as user
+where message.author_id /*+ indexnl */= user.id and
+      message.send_time >= datetime('2000-04-03T17:47:01') and
+      message.send_time < datetime('2000-07-02T17:47:01');
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-6-months.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-6-months.sqlpp
new file mode 100644
index 0000000..6b0ab33
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-6-months.sqlpp
@@ -0,0 +1,14 @@
+use SocialNetworkData;
+
+select element {
+'uname': user.name,
+'alias': user.alias,
+'send_time': message.send_time,
+'message': substring(message.message, 1, 31)
+}
+from GleambookMessages as message,
+     GleambookUsers as user
+where message.author_id /*+ indexnl */= user.id and
+      message.send_time >= datetime('2001-10-22T09:18:11') and
+      message.send_time < datetime('2002-04-20T09:18:11');
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-day.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-day.sqlpp
new file mode 100644
index 0000000..9eb2e40
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-day.sqlpp
@@ -0,0 +1,14 @@
+use SocialNetworkData;
+
+select element {
+'uname': user.name,
+'alias': user.alias,
+'send_time': message.send_time,
+'message': substring(message.message, 1, 31)
+}
+from GleambookMessages as message,
+     GleambookUsers as user
+where message.author_id /*+ indexnl */= user.id and
+      message.send_time >= datetime('2002-08-25T10:53:14') and
+      message.send_time < datetime('2002-08-26T10:53:14');
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-hour.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-hour.sqlpp
new file mode 100644
index 0000000..f540c6b
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-hour.sqlpp
@@ -0,0 +1,14 @@
+use SocialNetworkData;
+
+select element {
+'uname': user.name,
+'alias': user.alias,
+'send_time': message.send_time,
+'message': substring(message.message, 1, 31)
+}
+from GleambookMessages as message,
+     GleambookUsers as user
+where message.author_id /*+ indexnl */= user.id and
+      message.send_time >= datetime('2000-03-28T13:29:53') and
+      message.send_time < datetime('2000-03-28T14:29:53');
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-minute.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-minute.sqlpp
new file mode 100644
index 0000000..2f448ad
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-minute.sqlpp
@@ -0,0 +1,14 @@
+use SocialNetworkData;
+
+select element {
+'uname': user.name,
+'alias': user.alias,
+'send_time': message.send_time,
+'message': substring(message.message, 1, 31)
+}
+from GleambookMessages as message,
+     GleambookUsers as user
+where message.author_id /*+ indexnl */= user.id and
+      message.send_time >= datetime('2013-08-31T04:21:21') and
+      message.send_time < datetime('2013-08-31T04:22:21');
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-month.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-month.sqlpp
new file mode 100644
index 0000000..8112a57
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-month.sqlpp
@@ -0,0 +1,14 @@
+use SocialNetworkData;
+
+select element {
+'uname': user.name,
+'alias': user.alias,
+'send_time': message.send_time,
+'message': substring(message.message, 1, 31)
+}
+from GleambookMessages as message,
+     GleambookUsers as user
+where message.author_id /*+ indexnl */= user.id and
+      message.send_time >= datetime('2014-01-27T06:38:19') and
+      message.send_time < datetime('2014-02-26T06:38:19');
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-week.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-week.sqlpp
new file mode 100644
index 0000000..322f06a
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/join_indexnl-week.sqlpp
@@ -0,0 +1,14 @@
+use SocialNetworkData;
+
+select element {
+'uname': user.name,
+'alias': user.alias,
+'send_time': message.send_time,
+'message': substring(message.message, 1, 31)
+}
+from GleambookMessages as message,
+     GleambookUsers as user
+where message.author_id /*+ indexnl */= user.id and
+      message.send_time >= datetime('2012-08-03T11:02:50') and
+      message.send_time < datetime('2012-08-10T11:02:50');
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-3-months.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-3-months.sqlpp
new file mode 100644
index 0000000..343f704
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-3-months.sqlpp
@@ -0,0 +1,13 @@
+use SocialNetworkData;
+
+select uid as uid, c as coll_count
+from GleambookUsers as user,
+     GleambookMessages as message
+where message.author_id = user.id and
+      message.send_time >= datetime('2004-04-21T04:24:26') and
+      message.send_time < datetime('2004-07-20T04:24:26')
+group by user.id AS uid
+let c = coll_count(message)
+order by c desc
+limit 10;
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-6-months.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-6-months.sqlpp
new file mode 100644
index 0000000..fea8f3f
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-6-months.sqlpp
@@ -0,0 +1,13 @@
+use SocialNetworkData;
+
+select uid as uid, c as coll_count
+from GleambookUsers as user,
+     GleambookMessages as message
+where message.author_id = user.id and
+      message.send_time >= datetime('2003-06-04T05:29:58') and
+      message.send_time < datetime('2003-12-01T05:29:58')
+group by user.id AS uid
+let c = coll_count(message)
+order by c desc
+limit 10;
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-day.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-day.sqlpp
new file mode 100644
index 0000000..0872cda
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-day.sqlpp
@@ -0,0 +1,13 @@
+use SocialNetworkData;
+
+select uid as uid, c as coll_count
+from GleambookUsers as user,
+     GleambookMessages as message
+where message.author_id = user.id and
+      message.send_time >= datetime('2011-08-06T12:22:42') and
+      message.send_time < datetime('2011-08-07T12:22:42')
+group by user.id AS uid
+let c = coll_count(message)
+order by c desc
+limit 10;
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-hour.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-hour.sqlpp
new file mode 100644
index 0000000..cfacbd1
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-hour.sqlpp
@@ -0,0 +1,13 @@
+use SocialNetworkData;
+
+select uid as uid, c as coll_count
+from GleambookUsers as user,
+     GleambookMessages as message
+where message.author_id = user.id and
+      message.send_time >= datetime('2004-03-25T01:06:02') and
+      message.send_time < datetime('2004-03-25T02:06:02')
+group by user.id AS uid
+let c = coll_count(message)
+order by c desc
+limit 10;
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-minute.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-minute.sqlpp
new file mode 100644
index 0000000..a19bd4e
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-minute.sqlpp
@@ -0,0 +1,13 @@
+use SocialNetworkData;
+
+select uid as uid, c as coll_count
+from GleambookUsers as user,
+     GleambookMessages as message
+where message.author_id = user.id and
+      message.send_time >= datetime('2003-07-04T21:31:33') and
+      message.send_time < datetime('2003-07-04T21:32:33')
+group by user.id AS uid
+let c = coll_count(message)
+order by c desc
+limit 10;
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-month.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-month.sqlpp
new file mode 100644
index 0000000..7d6d7a0
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-month.sqlpp
@@ -0,0 +1,13 @@
+use SocialNetworkData;
+
+select uid as uid, c as coll_count
+from GleambookUsers as user,
+     GleambookMessages as message
+where message.author_id = user.id and
+      message.send_time >= datetime('2003-07-11T10:56:04') and
+      message.send_time < datetime('2003-08-10T10:56:04')
+group by user.id AS uid
+let c = coll_count(message)
+order by c desc
+limit 10;
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-week.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-week.sqlpp
new file mode 100644
index 0000000..a2087ae
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby-week.sqlpp
@@ -0,0 +1,13 @@
+use SocialNetworkData;
+
+select uid as uid, c as coll_count
+from GleambookUsers as user,
+     GleambookMessages as message
+where message.author_id = user.id and
+      message.send_time >= datetime('2010-04-10T18:10:27') and
+      message.send_time < datetime('2010-04-17T18:10:27')
+group by user.id AS uid
+let c = coll_count(message)
+order by c desc
+limit 10;
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-3-months.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-3-months.sqlpp
new file mode 100644
index 0000000..cba114b
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-3-months.sqlpp
@@ -0,0 +1,13 @@
+use SocialNetworkData;
+
+select uid as uid, c as coll_count
+from  GleambookMessages as message,
+      GleambookUsers as user
+where message.author_id /*+ indexnl */ = user.id and
+      message.send_time >= datetime('2004-04-21T04:24:26') and
+      message.send_time < datetime('2004-07-20T04:24:26')
+group by user.id AS uid
+let c = coll_count(message)
+order by c desc
+limit 10;
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-6-months.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-6-months.sqlpp
new file mode 100644
index 0000000..3b45502
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-6-months.sqlpp
@@ -0,0 +1,13 @@
+use SocialNetworkData;
+
+select uid as uid, c as coll_count
+from  GleambookMessages as message,
+      GleambookUsers as user
+where message.author_id /*+ indexnl */ = user.id and
+      message.send_time >= datetime('2003-06-04T05:29:58') and
+      message.send_time < datetime('2003-12-01T05:29:58')
+group by user.id AS uid
+let c = coll_count(message)
+order by c desc
+limit 10;
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-day.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-day.sqlpp
new file mode 100644
index 0000000..3633815
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-day.sqlpp
@@ -0,0 +1,13 @@
+use SocialNetworkData;
+
+select uid as uid, c as coll_count
+from  GleambookMessages as message,
+      GleambookUsers as user
+where message.author_id /*+ indexnl */ = user.id and
+      message.send_time >= datetime('2011-08-06T12:22:42') and
+      message.send_time < datetime('2011-08-07T12:22:42')
+group by user.id AS uid
+let c = coll_count(message)
+order by c desc
+limit 10;
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-hour.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-hour.sqlpp
new file mode 100644
index 0000000..d075df9
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-hour.sqlpp
@@ -0,0 +1,13 @@
+use SocialNetworkData;
+
+select uid as uid, c as coll_count
+from  GleambookMessages as message,
+      GleambookUsers as user
+where message.author_id /*+ indexnl */ = user.id and
+      message.send_time >= datetime('2004-03-25T01:06:02') and
+      message.send_time < datetime('2004-03-25T02:06:02')
+group by user.id AS uid
+let c = coll_count(message)
+order by c desc
+limit 10;
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-minute.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-minute.sqlpp
new file mode 100644
index 0000000..b0ee72e
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-minute.sqlpp
@@ -0,0 +1,13 @@
+use SocialNetworkData;
+
+select uid as uid, c as coll_count
+from  GleambookMessages as message,
+      GleambookUsers as user
+where message.author_id /*+ indexnl */ = user.id and
+      message.send_time >= datetime('2003-07-04T21:31:33') and
+      message.send_time < datetime('2003-07-04T21:32:33')
+group by user.id AS uid
+let c = coll_count(message)
+order by c desc
+limit 10;
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-month.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-month.sqlpp
new file mode 100644
index 0000000..1a80aad
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-month.sqlpp
@@ -0,0 +1,13 @@
+use SocialNetworkData;
+
+select uid as uid, c as coll_count
+from  GleambookMessages as message,
+      GleambookUsers as user
+where message.author_id /*+ indexnl */ = user.id and
+      message.send_time >= datetime('2003-07-11T10:56:04') and
+      message.send_time < datetime('2003-08-10T10:56:04')
+group by user.id AS uid
+let c = coll_count(message)
+order by c desc
+limit 10;
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-week.sqlpp b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-week.sqlpp
new file mode 100644
index 0000000..b7e0057
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/joingby_index-week.sqlpp
@@ -0,0 +1,13 @@
+use SocialNetworkData;
+
+select uid as uid, c as coll_count
+from  GleambookMessages as message,
+      GleambookUsers as user
+where message.author_id /*+ indexnl */ = user.id and
+      message.send_time >= datetime('2010-04-10T18:10:27') and
+      message.send_time < datetime('2010-04-17T18:10:27')
+group by user.id AS uid
+let c = coll_count(message)
+order by c desc
+limit 10;
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/base/asterix_base_4_ingest.aql b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/base/asterix_base_4_ingest.aql
new file mode 100644
index 0000000..818baa3
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/base/asterix_base_4_ingest.aql
@@ -0,0 +1,2 @@
+use dataverse SocialNetworkData;
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/base/nuclear_base_4_ingest.aql b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/base/nuclear_base_4_ingest.aql
new file mode 100644
index 0000000..9ca69c8
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/base/nuclear_base_4_ingest.aql
@@ -0,0 +1,14 @@
+use dataverse experiments;
+
+create feed TweetFeed1 using socket_adapter
+(
+    ("sockets"="radium.ics.uci.edu:10001,promethium.ics.uci.edu:10001,actinium.ics.uci.edu:10001"),
+    ("address-type"="IP"),
+    ("type-name"="TweetMessageType"),
+    ("format"="adm"),
+    ("duration"="1200")
+);
+
+set wait-for-completion-feed "false";
+
+connect feed TweetFeed1 to dataset Tweets;
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/base/perf_types.aql b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/base/perf_types.aql
new file mode 100644
index 0000000..789b600
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/base/perf_types.aql
@@ -0,0 +1,45 @@
+drop dataverse SocialNetworkData if exists;
+create dataverse SocialNetworkData;
+use dataverse SocialNetworkData;
+
+  create type EmploymentType as {
+    organization: string,
+    start_date: date,
+    end_date: date?
+  }
+
+  create type GleambookUserType as {
+    id: string,
+    alias: string,
+    name: string,
+    user_since: datetime,
+    friend_ids: {{ string }},
+    employment: [EmploymentType]
+  }
+
+  create type GleambookMessageType as {
+    message_id: string,
+    author_id: string,
+    in_response_to: string?,
+    sender_location: point,
+    send_time: datetime,
+    message: string
+  }
+
+  create type ChirpUserType as {
+    screen_name: string,
+    lang: string,
+    friends_count: int32,
+    statuses_count: int32,
+    name: string,
+    followers_count: int32
+  }
+
+  create type ChirpMessageType as {
+    chirpid: string,
+    user: ChirpUserType,
+    sender_location: point,
+    send_time: datetime,
+    referred_topics: {{ string }},
+    message_text: string
+  }
\ No newline at end of file
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/bench_3.aql b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/bench_3.aql
new file mode 100644
index 0000000..b51bcae
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/bench_3.aql
@@ -0,0 +1,8 @@
+use dataverse SocialNetworkData;
+
+create dataset GleambookUsers(GleambookUserType)
+primary key id;
+
+create dataset GleambookMessages(GleambookMessageType)
+primary key message_id;
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/bench_3_load.aql b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/bench_3_load.aql
new file mode 100644
index 0000000..9710ddd
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/bench_3_load.aql
@@ -0,0 +1,8 @@
+use dataverse SocialNetworkData;
+
+load dataset GleambookMessages using
+localfs(("path"="asterix-6:///mnt/data/sdf/imaxon/datagen/gbook_messages.adm,asterix-7:///mnt/data/sdf/imaxon/datagen/gbook_messages.adm,asterix-8:///mnt/data/sdf/imaxon/datagen/gbook_messages.adm,asterix-9:///mnt/data/sdf/imaxon/datagen/gbook_messages.adm"),("format"="adm"));
+
+load dataset GleambookUsers using
+localfs(("path"="asterix-6:///mnt/data/sdf/imaxon/datagen/gbook_users.adm,asterix-7:///mnt/data/sdf/imaxon/datagen/gbook_users.adm,asterix-8:///mnt/data/sdf/imaxon/datagen/gbook_users.adm,asterix-9:///mnt/data/sdf/imaxon/datagen/gbook_users.adm"),("format"="adm"));
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/bench_count.aql b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/bench_count.aql
new file mode 100644
index 0000000..f4eab3b
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/bench_count.aql
@@ -0,0 +1,4 @@
+use dataverse SocialNetworkData;
+
+let $count := count(for $t in dataset GleambookMessages return $t.id)
+return $count;
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/asterix-agg.xml b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/asterix-agg.xml
new file mode 100644
index 0000000..0445da2
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/asterix-agg.xml
@@ -0,0 +1,51 @@
+<!--
+ ! Copyright 2009-2013 by The Regents of the University of California
+ ! Licensed under the Apache License, Version 2.0 (the "License");
+ ! you may not use this file except in compliance with the License.
+ ! you may obtain a copy of the License from
+ !
+ !     http://www.apache.org/licenses/LICENSE-2.0
+ !
+ ! Unless required by applicable law or agreed to in writing, software
+ ! distributed under the License is distributed on an "AS IS" BASIS,
+ ! WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ! See the License for the specific language governing permissions and
+ ! limitations under the License.
+ !-->
+<cluster xmlns="cluster">
+  <name>4node</name>
+  <username>asterix</username>
+  <working_dir>
+    <dir>/home/asterix/asterixdb/working</dir>
+    <NFS>true</NFS>
+  </working_dir>
+  <log_dir>/home/asterix/logs</log_dir>
+  <txn_log_dir>/mnt/data/sdh/scratch/asterix/asterix/txnlogs</txn_log_dir>
+  <iodevices>/mnt/data/sde/asterix/asterix/</iodevices>
+  <store>storage</store>
+  <java_home>/usr/lib/jvm/jre-1.8.0/</java_home>
+  <master_node>
+    <id>3</id>
+    <cluster_ip>asterix-3</cluster_ip>
+    <cluster_port>1099</cluster_port>
+    <client_ip>asterix-3</client_ip>
+    <client_port>1098</client_port>
+    <http_port>8888</http_port>
+  </master_node>
+  <node>
+    <id>4</id>
+    <cluster_ip>asterix-6</cluster_ip>
+  </node>
+  <node>
+    <id>5</id>
+    <cluster_ip>asterix-7</cluster_ip>
+  </node>
+  <node>
+    <id>6</id>
+    <cluster_ip>asterix-8</cluster_ip>
+  </node>
+  <node>
+    <id>7</id>
+    <cluster_ip>asterix-9</cluster_ip>
+  </node>
+</cluster>
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/asterix-configuration.xml b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/asterix-configuration.xml
index cc289bd..af37d0b 100644
--- a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/asterix-configuration.xml
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/asterix-configuration.xml
@@ -232,7 +232,7 @@
 
   <property>
     <name>log.level</name>
-    <value>ALL</value>
+    <value>INFO</value>
     <description>The minimum log level to be displayed. (Default = INFO)
     </description>
   </property>
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/managix-conf.xml b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/managix-conf.xml
index 720770f..6bfb383 100644
--- a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/managix-conf.xml
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/managix-conf.xml
@@ -5,13 +5,13 @@
       <version>0.20.2</version>
       <url></url>
     </hdfs>
-    <backupDir>/home/youngsk2/managix/clusters/local/working_dir/backup</backupDir>
+    <backupDir>/home/asterixdb/asterixdb/.installer/zookeeper_home</backupDir>
   </backup>
   <zookeeper>
-    <homeDir>/home/youngsk2/managix/.installer/zookeeper_home</homeDir>
+    <homeDir>/home/asterixdb/asterixdb/.installer/zookeeper_home</homeDir>
     <clientPort>2900</clientPort>
     <servers>
-      <java_home>/home/youngsk2/jdk1.7.0_65</java_home>
+      <java_home>/usr/lib/jvm/jre-1.8.0/</java_home>
       <server>128.195.9.21</server>
     </servers>
   </zookeeper>
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/nuclear.xml b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/nuclear.xml
new file mode 100644
index 0000000..dc6981f
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/configs/nuclear.xml
@@ -0,0 +1,48 @@
+<!--
+ ! Copyright 2009-2013 by The Regents of the University of California
+ ! Licensed under the Apache License, Version 2.0 (the "License");
+ ! you may not use this file except in compliance with the License.
+ ! you may obtain a copy of the License from
+ !
+ !     http://www.apache.org/licenses/LICENSE-2.0
+ !
+ ! Unless required by applicable law or agreed to in writing, software
+ ! distributed under the License is distributed on an "AS IS" BASIS,
+ ! WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ! See the License for the specific language governing permissions and
+ ! limitations under the License.
+ !-->
+<cluster xmlns="cluster">
+  <name>nuclear</name>
+  <username>asterix</username>
+  <working_dir>
+    <dir>/home/asterixdb/asterixdb/working</dir>
+    <NFS>false</NFS>
+  </working_dir>
+  <log_dir>/home/asterixdb/asterixdb/logs/</log_dir>
+  <txn_log_dir>/home/asterixdb/asterixdb/txnLogs/</txn_log_dir>
+  <iodevices>/home/asterixdb/asterixdb/</iodevices>
+  <store>storage</store>
+  <java_home>/usr/lib/jvm/jre-1.8.0</java_home>
+  <master_node>
+    <id>master</id>
+    <cluster_ip>technetium.ics.uci.edu</cluster_ip>
+    <cluster_port>1099</cluster_port>
+    <client_ip>technetium.ics.uci.edu</client_ip>
+    <client_port>1098</client_port>
+    <http_port>8888</http_port>
+  </master_node>
+  <node>
+    <id>technetium</id>
+    <cluster_ip>technetium.ics.uci.edu</cluster_ip>
+  </node>
+  <node>
+    <id>polonium</id>
+    <cluster_ip>polonium.ics.uci.edu</cluster_ip>
+  </node>
+  <node>
+    <id>radon</id>
+    <cluster_ip>radon.ics.uci.edu</cluster_ip>
+  </node>
+</cluster>
+
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/dgen/asterix-4.dgen b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/dgen/asterix-4.dgen
new file mode 100644
index 0000000..5bd95d7
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/dgen/asterix-4.dgen
@@ -0,0 +1,4 @@
+asterix-4.ics.uci.edu   asterix-8.ics.uci.edu:10001
+asterix-5.ics.uci.edu   asterix-9.ics.uci.edu:10001
+asterix-6.ics.uci.edu   asterix-10.ics.uci.edu:10001
+asterix-7.ics.uci.edu   asterix-2.ics.uci.edu:10001
diff --git a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/dgen/nuc.dgen b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/dgen/nuc.dgen
new file mode 100644
index 0000000..142b656
--- /dev/null
+++ b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/dgen/nuc.dgen
@@ -0,0 +1,3 @@
+technetium.ics.uci.edu   radium.ics.uci.edu:10001
+polonium.ics.uci.edu   promethium.ics.uci.edu:10001
+radon.ics.uci.edu   actinium.ics.uci.edu:10001

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/884
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I21e2d44ca46a4a28478d9ef256c4fa8d53f03dc9
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon <imaxon@apache.org>

Mime
View raw message