asterixdb-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From amo...@apache.org
Subject [13/13] incubator-asterixdb git commit: Improve Error Handling in Local Directory Feeds
Date Sat, 26 Mar 2016 20:27:44 GMT
Improve Error Handling in Local Directory Feeds

This change improves handling of two error types for filesystem
based feeds. The first one is the handling of IO Errors which
causes the input stream to be closed, and the second one is
reacting to missed filesystem events. In both cases, we scan the
directory and compare it with the history we have in order to
resume from where we last left off.

In addition, this change includes some refactoring in external
data. Particularly, we get rid of the stream provider layer and
instead, stream factories create input streams directly. This
is consistent with record reader factories which create readers
directly without reader providers.

Change-Id: I08d89229e33c91532b1038ba9f7a372f7ca1fdb5
Reviewed-on: https://asterix-gerrit.ics.uci.edu/720
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>


Project: http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/commit/121e1d9a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/tree/121e1d9a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/diff/121e1d9a

Branch: refs/heads/master
Commit: 121e1d9abd54c60acf595b678955b38b63d38100
Parents: cf39bc9
Author: Abdullah Alamoudi <bamousaa@gmail.com>
Authored: Sat Mar 26 23:02:39 2016 +0300
Committer: abdullah alamoudi <bamousaa@gmail.com>
Committed: Sat Mar 26 13:21:49 2016 -0700

----------------------------------------------------------------------
 .gitattributes                                  |   3 +
 .gitignore                                      |   1 +
 .../optimizer/rules/UnnestToDataScanRule.java   |   4 +-
 .../apache/asterix/app/external/FeedJoint.java  |  19 +-
 .../asterix/aql/translator/QueryTranslator.java |  16 +-
 .../queries/feeds/feeds_08/feeds_08.1.ddl.aql   |  10 +-
 .../feeds/feeds_08/feeds_08.2.update.aql        |   8 +-
 .../queries/feeds/feeds_08/feeds_08.3.query.aql |   8 +-
 .../queries/feeds/feeds_08/feeds_08.4.sleep.aql |  31 ++
 .../classad-parser/classad-parser.1.adm         | 200 +++----
 .../feed-with-external-parser.1.adm             | 198 +++----
 .../src/test/resources/runtimets/testsuite.xml  |  54 +-
 ...erixLSMInsertDeleteOperatorNodePushable.java |   7 +-
 .../external/api/AsterixInputStream.java        |  44 ++
 .../asterix/external/api/IExternalIndexer.java  |   2 +-
 .../external/api/IIndexingDatasource.java       |  27 +
 .../external/api/IInputStreamFactory.java       |  32 ++
 .../external/api/IInputStreamProvider.java      |  29 -
 .../api/IInputStreamProviderFactory.java        |  33 --
 .../asterix/external/api/IRecordReader.java     |   8 +-
 .../dataflow/FeedRecordDataFlowController.java  |  20 +-
 .../dataflow/FeedStreamDataFlowController.java  |   9 +-
 .../external/dataflow/FeedTupleForwarder.java   |   2 +-
 .../external/feed/api/IFeedFrameHandler.java    |   4 +-
 .../asterix/external/feed/api/IFeedJoint.java   |   4 +-
 .../feed/api/IFeedLifecycleListener.java        |   6 -
 .../external/feed/api/IMessageReceiver.java     |   2 +-
 .../dataflow/DistributeFeedFrameWriter.java     |   4 +-
 .../FeedCollectRuntimeInputHandler.java         |  13 +-
 .../feed/dataflow/FeedFrameHandlers.java        |   6 +-
 .../feed/dataflow/FeedRuntimeInputHandler.java  |   7 +-
 .../feed/dataflow/FrameDistributor.java         | 123 +++--
 .../feed/management/FeedConnectionRequest.java  |   9 +-
 .../external/feed/message/MessageReceiver.java  |  12 +-
 .../external/feed/runtime/IngestionRuntime.java |   2 +-
 .../external/indexing/FileOffsetIndexer.java    |  16 +-
 .../indexing/RecordColumnarIndexer.java         |   4 +-
 .../external/input/HDFSDataSourceFactory.java   |  16 +-
 .../external/input/record/CharArrayRecord.java  |  45 +-
 ...questToRecordWithMetadataAndPKConverter.java |   2 +-
 .../record/reader/TestAsterixMembersReader.java |   9 +-
 .../hdfs/AbstractCharRecordLookupReader.java    |   2 +-
 .../record/reader/hdfs/HDFSRecordReader.java    |  14 +-
 .../input/record/reader/kv/KVReader.java        |  18 +-
 .../input/record/reader/kv/KVTestReader.java    |   9 +-
 .../record/reader/rss/RSSRecordReader.java      |   9 +-
 .../stream/AbstractStreamRecordReader.java      |  41 +-
 .../AbstractStreamRecordReaderFactory.java      |  17 +-
 .../stream/EmptyLineSeparatedRecordReader.java  |   4 +-
 .../EmptyLineSeparatedRecordReaderFactory.java  |   4 +-
 .../record/reader/stream/LineRecordReader.java  |   4 +-
 .../reader/stream/LineRecordReaderFactory.java  |   4 +-
 .../reader/stream/QuotedLineRecordReader.java   |   6 +-
 .../stream/SemiStructuredRecordReader.java      |  16 +-
 .../SemiStructuredRecordReaderFactory.java      |   4 +-
 .../reader/twitter/TwitterPullRecordReader.java |   9 +-
 .../reader/twitter/TwitterPushRecordReader.java |   9 +-
 .../external/input/stream/AInputStream.java     |  36 --
 .../input/stream/AInputStreamReader.java        | 114 ----
 .../input/stream/AsterixInputStreamReader.java  | 120 ++++
 .../external/input/stream/BasicInputStream.java |  17 +-
 .../external/input/stream/HDFSInputStream.java  | 237 ++++++++
 .../input/stream/LocalFSInputStream.java        | 182 +++++++
 .../stream/LocalFileSystemInputStream.java      | 137 -----
 .../input/stream/SocketClientInputStream.java   |  61 +++
 .../input/stream/SocketServerInputStream.java   |  28 +-
 .../stream/TwitterFirehoseInputStream.java      | 158 ++++++
 .../factory/LocalFSInputStreamFactory.java      | 158 ++++++
 .../LocalFSInputStreamProviderFactory.java      | 152 ------
 .../factory/SocketClientInputStreamFactory.java |  84 +++
 .../SocketClientInputStreamProviderFactory.java |  80 ---
 .../factory/SocketServerInputStreamFactory.java | 141 +++++
 .../SocketServerInputStreamProviderFactory.java | 141 -----
 .../factory/TwitterFirehoseStreamFactory.java   | 102 ++++
 .../TwitterFirehoseStreamProviderFactory.java   |  98 ----
 .../provider/HDFSInputStreamProvider.java       | 127 -----
 .../provider/LocalFSInputStreamProvider.java    |  58 --
 .../SocketClientInputStreamProvider.java        |  98 ----
 .../SocketServerInputStreamProvider.java        |  43 --
 .../TwitterFirehoseInputStreamProvider.java     | 202 -------
 .../FeedCollectOperatorDescriptor.java          |  52 +-
 .../FeedCollectOperatorNodePushable.java        |  17 +-
 .../external/parser/AbstractDataParser.java     |   2 +-
 .../provider/DataflowControllerProvider.java    |  15 +-
 .../provider/DatasourceFactoryProvider.java     |  22 +-
 .../external/util/ExternalDataConstants.java    |   5 +-
 .../external/util/ExternalDataUtils.java        |   6 +-
 .../external/util/FileSystemWatcher.java        |  52 +-
 .../apache/asterix/external/util/HDFSUtils.java |   4 +-
 .../external/classad/AttributeReference.java    | 149 ++---
 .../classad/BuiltinClassAdFunctions.java        | 543 ++++++++++--------
 .../external/classad/CaseInsensitiveString.java |  12 +-
 .../asterix/external/classad/ClassAd.java       | 411 +++++++-------
 .../asterix/external/classad/ClassAdFunc.java   |   4 +-
 .../asterix/external/classad/ClassAdTime.java   |  12 +
 .../external/classad/ClassAdUnParser.java       |  76 +--
 .../asterix/external/classad/EvalState.java     |  30 +-
 .../asterix/external/classad/ExprList.java      |  81 +--
 .../asterix/external/classad/ExprTree.java      |  56 +-
 .../external/classad/ExprTreeHolder.java        |  29 +-
 .../asterix/external/classad/FunctionCall.java  | 119 ++--
 .../apache/asterix/external/classad/Lexer.java  |   6 +-
 .../asterix/external/classad/Literal.java       | 143 ++---
 .../asterix/external/classad/Operation.java     | 545 ++++++++++---------
 .../asterix/external/classad/PrettyPrint.java   |   9 +-
 .../apache/asterix/external/classad/Value.java  | 149 ++---
 .../pool/AMutableCharArrayStringPool.java       |  35 ++
 .../object/pool/AttributeReferencePool.java     |   9 +-
 .../object/pool/CaseInsensitiveStringPool.java  |  23 +-
 .../classad/object/pool/ClassAdObjectPool.java  | 103 ++++
 .../classad/object/pool/ClassAdParserPool.java  |  39 ++
 .../classad/object/pool/ClassAdPool.java        |   8 +-
 .../classad/object/pool/ClassAdTimePool.java    |  35 ++
 .../classad/object/pool/DoublePool.java         |  34 ++
 .../classad/object/pool/EvalStatePool.java      |  41 ++
 .../classad/object/pool/ExprHolderPool.java     |   9 +-
 .../classad/object/pool/ExprListPool.java       |   8 +-
 .../classad/object/pool/FunctionCallPool.java   |  41 ++
 .../classad/object/pool/HashMapPool.java        |  34 ++
 .../external/classad/object/pool/Int32Pool.java |  35 ++
 .../external/classad/object/pool/Int64Pool.java |  34 ++
 .../classad/object/pool/LiteralPool.java        |   8 +-
 .../classad/object/pool/MutableBooleanPool.java |  35 ++
 .../object/pool/MutableNumberFactorPool.java    |  36 ++
 .../classad/object/pool/OperationPool.java      |   8 +-
 .../classad/object/pool/PrettyPrintPool.java    |  40 ++
 .../object/pool/StringArrayListPool.java        |  35 ++
 .../classad/object/pool/TreeSetPool.java        |  35 ++
 .../external/classad/object/pool/ValuePool.java |  11 +-
 .../classad/test/ClassAdFunctionalTest.java     |   9 +-
 .../classad/test/ClassAdParserTest.java         |  12 +-
 .../external/classad/test/ClassAdToADMTest.java |  20 +-
 .../external/classad/test/ClassAdUnitTest.java  |   7 +-
 .../classad/test/ClassAdUnitTester.java         |  80 +--
 .../external/classad/test/FunctionalTester.java | 170 +++---
 .../asterix/external/library/ClassAdParser.java | 277 +++++-----
 .../external/library/ClassAdParserFactory.java  |  17 +-
 .../parser/test/RecordWithMetaTest.java         |  10 +-
 .../src/test/resources/functional_tests.txt     |  22 +-
 .../record-parser/record-parser.1.adm           | 200 +++----
 .../metadata/declared/AqlMetadataProvider.java  |  14 +-
 .../metadata/declared/FeedDataSource.java       |   8 +-
 .../serde/AStringSerializerDeserializer.java    |  15 +-
 .../AqlSerializerDeserializerProvider.java      |   9 +
 .../serde/SimpleSerializerDeserializerTest.java |   3 +-
 145 files changed, 4395 insertions(+), 3516 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/121e1d9a/.gitattributes
----------------------------------------------------------------------
diff --git a/.gitattributes b/.gitattributes
index 2451378..5eb0c00 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -18,3 +18,6 @@
 
 # Explicitly declare binary files
 *testdata.txt binary
+
+#Unset sample_01.cvs
+asterix-app/data/csv/sample_01.csv text=unset

http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/121e1d9a/.gitignore
----------------------------------------------------------------------
diff --git a/.gitignore b/.gitignore
index 1b129d6..8d0e894 100644
--- a/.gitignore
+++ b/.gitignore
@@ -16,6 +16,7 @@ asterix-app/opttest/
 asterix-app/parserts/
 asterix-app/opt_parserts/
 asterix-app/runtime_parserts/
+asterix-app/data/csv/beer.csv
 asterix-installer/ittest/
 asterix-installer/repliationtest/
 asterix-installer/src/test/resources/clusterts/asterix-installer-0.8.9-SNAPSHOT-binary-assembly/

http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/121e1d9a/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java
----------------------------------------------------------------------
diff --git a/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java
b/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java
index c3062ed..a0060b4 100644
--- a/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java
+++ b/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java
@@ -22,7 +22,7 @@ import java.util.ArrayList;
 import java.util.List;
 
 import org.apache.asterix.common.config.DatasetConfig.DatasetType;
-import org.apache.asterix.external.feed.api.IFeedLifecycleListener.ConnectionLocation;
+import org.apache.asterix.external.feed.api.IFeedRuntime.FeedRuntimeType;
 import org.apache.asterix.external.feed.watch.FeedActivity.FeedActivityDetails;
 import org.apache.asterix.external.util.ExternalDataUtils;
 import org.apache.asterix.external.util.FeedUtils;
@@ -268,7 +268,7 @@ public class UnnestToDataScanRule implements IAlgebraicRewriteRule {
         }
         FeedDataSource feedDataSource = new FeedDataSource(sourceFeed, aqlId, targetDataset,
feedOutputType, metaType,
                 pkTypes, partitioningKeys, keyAccessScalarFunctionCallExpression, sourceFeed.getFeedId(),
-                sourceFeed.getFeedType(), ConnectionLocation.valueOf(subscriptionLocation),
locations.split(","),
+                sourceFeed.getFeedType(), FeedRuntimeType.valueOf(subscriptionLocation),
locations.split(","),
                 context.getComputationNodeDomain());
         feedDataSource.getProperties().put(BuiltinFeedPolicies.CONFIG_FEED_POLICY_KEY, feedPolicy);
         return feedDataSource;

http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/121e1d9a/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJoint.java
----------------------------------------------------------------------
diff --git a/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJoint.java b/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJoint.java
index e650a5b..3f0bc54 100644
--- a/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJoint.java
+++ b/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJoint.java
@@ -24,7 +24,7 @@ import java.util.logging.Level;
 import java.util.logging.Logger;
 
 import org.apache.asterix.external.feed.api.IFeedJoint;
-import org.apache.asterix.external.feed.api.IFeedLifecycleListener.ConnectionLocation;
+import org.apache.asterix.external.feed.api.IFeedRuntime.FeedRuntimeType;
 import org.apache.asterix.external.feed.management.FeedConnectionId;
 import org.apache.asterix.external.feed.management.FeedConnectionRequest;
 import org.apache.asterix.external.feed.management.FeedId;
@@ -49,13 +49,13 @@ public class FeedJoint implements IFeedJoint {
     /** A list of feed subscription requests submitted for subscribing to the FeedPoint's
data **/
     private final List<FeedConnectionRequest> connectionRequests;
 
-    private final ConnectionLocation connectionLocation;
+    private final FeedRuntimeType connectionLocation;
 
     private final FeedJointType type;
 
     private FeedConnectionId provider;
 
-    public FeedJoint(FeedJointKey key, FeedId ownerFeedId, ConnectionLocation subscriptionLocation,
FeedJointType type,
+    public FeedJoint(FeedJointKey key, FeedId ownerFeedId, FeedRuntimeType subscriptionLocation,
FeedJointType type,
             FeedConnectionId provider) {
         this.key = key;
         this.ownerFeedId = ownerFeedId;
@@ -72,14 +72,17 @@ public class FeedJoint implements IFeedJoint {
         return key.hashCode();
     }
 
+    @Override
     public void addReceiver(FeedConnectionId connectionId) {
         receivers.add(connectionId);
     }
 
+    @Override
     public void removeReceiver(FeedConnectionId connectionId) {
         receivers.remove(connectionId);
     }
 
+    @Override
     public synchronized void addConnectionRequest(FeedConnectionRequest request) {
         connectionRequests.add(request);
         if (state.equals(State.ACTIVE)) {
@@ -87,6 +90,7 @@ public class FeedJoint implements IFeedJoint {
         }
     }
 
+    @Override
     public synchronized void setState(State state) {
         if (this.state.equals(state)) {
             return;
@@ -121,6 +125,7 @@ public class FeedJoint implements IFeedJoint {
         connectionRequests.clear();
     }
 
+    @Override
     public FeedConnectionId getReceiver(FeedConnectionId connectionId) {
         for (FeedConnectionId cid : receivers) {
             if (cid.equals(connectionId)) {
@@ -149,18 +154,22 @@ public class FeedJoint implements IFeedJoint {
         return ((FeedJoint) o).getFeedJointKey().equals(this.key);
     }
 
+    @Override
     public FeedId getOwnerFeedId() {
         return ownerFeedId;
     }
 
+    @Override
     public List<FeedConnectionRequest> getConnectionRequests() {
         return connectionRequests;
     }
 
-    public ConnectionLocation getConnectionLocation() {
+    @Override
+    public FeedRuntimeType getConnectionLocation() {
         return connectionLocation;
     }
 
+    @Override
     public FeedJointType getType() {
         return type;
     }
@@ -170,6 +179,7 @@ public class FeedJoint implements IFeedJoint {
         return provider;
     }
 
+    @Override
     public List<FeedConnectionId> getReceivers() {
         return receivers;
     }
@@ -178,6 +188,7 @@ public class FeedJoint implements IFeedJoint {
         return key;
     }
 
+    @Override
     public synchronized State getState() {
         return state;
     }

http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/121e1d9a/asterix-app/src/main/java/org/apache/asterix/aql/translator/QueryTranslator.java
----------------------------------------------------------------------
diff --git a/asterix-app/src/main/java/org/apache/asterix/aql/translator/QueryTranslator.java
b/asterix-app/src/main/java/org/apache/asterix/aql/translator/QueryTranslator.java
index 67bf3bb..3786413 100644
--- a/asterix-app/src/main/java/org/apache/asterix/aql/translator/QueryTranslator.java
+++ b/asterix-app/src/main/java/org/apache/asterix/aql/translator/QueryTranslator.java
@@ -63,7 +63,7 @@ import org.apache.asterix.external.feed.api.IFeedJoint;
 import org.apache.asterix.external.feed.api.IFeedJoint.FeedJointType;
 import org.apache.asterix.external.feed.api.IFeedLifecycleEventSubscriber;
 import org.apache.asterix.external.feed.api.IFeedLifecycleEventSubscriber.FeedLifecycleEvent;
-import org.apache.asterix.external.feed.api.IFeedLifecycleListener.ConnectionLocation;
+import org.apache.asterix.external.feed.api.IFeedRuntime.FeedRuntimeType;
 import org.apache.asterix.external.feed.management.FeedConnectionId;
 import org.apache.asterix.external.feed.management.FeedConnectionRequest;
 import org.apache.asterix.external.feed.management.FeedId;
@@ -863,7 +863,7 @@ public class QueryTranslator extends AbstractLangTranslator {
                         if (subType.isOpen()) {
                             isOpen = true;
                             break;
-                        };
+                        } ;
                     }
                 }
                 if (fieldExpr.second == null) {
@@ -1951,7 +1951,7 @@ public class QueryTranslator extends AbstractLangTranslator {
 
     private JobSpecification rewriteCompileQuery(AqlMetadataProvider metadataProvider, Query
query,
             ICompiledDmlStatement stmt)
-                    throws AsterixException, RemoteException, AlgebricksException, JSONException,
ACIDException {
+            throws AsterixException, RemoteException, AlgebricksException, JSONException,
ACIDException {
 
         //Query Rewriting (happens under the same ongoing metadata transaction)
         Pair<Query, Integer> reWrittenQuery = apiFramework.reWriteQuery(declaredFunctions,
metadataProvider, query,
@@ -2257,7 +2257,7 @@ public class QueryTranslator extends AbstractLangTranslator {
      */
     private Triple<FeedConnectionRequest, Boolean, List<IFeedJoint>> getFeedConnectionRequest(String
dataverse,
             Feed feed, String dataset, FeedPolicyEntity feedPolicy, MetadataTransactionContext
mdTxnCtx)
-                    throws MetadataException {
+            throws MetadataException {
         IFeedJoint sourceFeedJoint = null;
         FeedConnectionRequest request = null;
         List<String> functionsToApply = new ArrayList<String>();
@@ -2265,13 +2265,13 @@ public class QueryTranslator extends AbstractLangTranslator {
         List<IFeedJoint> jointsToRegister = new ArrayList<IFeedJoint>();
         FeedConnectionId connectionId = new FeedConnectionId(feed.getFeedId(), dataset);
 
-        ConnectionLocation connectionLocation = null;
+        FeedRuntimeType connectionLocation = null;
         FeedJointKey feedJointKey = getFeedJointKey(feed, mdTxnCtx);
         boolean isFeedJointAvailable = FeedLifecycleListener.INSTANCE.isFeedJointAvailable(feedJointKey);
         if (!isFeedJointAvailable) {
             sourceFeedJoint = FeedLifecycleListener.INSTANCE.getAvailableFeedJoint(feedJointKey);
             if (sourceFeedJoint == null) { //the feed is currently not being ingested, i.e.,
it is unavailable.
-                connectionLocation = ConnectionLocation.SOURCE_FEED_INTAKE_STAGE;
+                connectionLocation = FeedRuntimeType.INTAKE;
                 FeedId sourceFeedId = feedJointKey.getFeedId(); // the root/primary feedId
                 Feed primaryFeed = MetadataManager.INSTANCE.getFeed(mdTxnCtx, dataverse,
sourceFeedId.getFeedName());
                 FeedJointKey intakeFeedJointKey = new FeedJointKey(sourceFeedId, new ArrayList<String>());
@@ -2295,7 +2295,7 @@ public class QueryTranslator extends AbstractLangTranslator {
             if (!functionsToApply.isEmpty()) {
                 FeedJointKey computeFeedJointKey = new FeedJointKey(feed.getFeedId(), functionsToApply);
                 IFeedJoint computeFeedJoint = new FeedJoint(computeFeedJointKey, feed.getFeedId(),
-                        ConnectionLocation.SOURCE_FEED_COMPUTE_STAGE, FeedJointType.COMPUTE,
connectionId);
+                        FeedRuntimeType.COMPUTE, FeedJointType.COMPUTE, connectionId);
                 jointsToRegister.add(computeFeedJoint);
             }
         } else {
@@ -2924,7 +2924,7 @@ public class QueryTranslator extends AbstractLangTranslator {
     private void prepareRunExternalRuntime(AqlMetadataProvider metadataProvider, IHyracksClientConnection
hcc,
             RunStatement pregelixStmt, String dataverseNameFrom, String dataverseNameTo,
String datasetNameFrom,
             String datasetNameTo, MetadataTransactionContext mdTxnCtx)
-                    throws AlgebricksException, AsterixException, Exception {
+            throws AlgebricksException, AsterixException, Exception {
         //Validates the source/sink dataverses and datasets.
         Dataset fromDataset = metadataProvider.findDataset(dataverseNameFrom, datasetNameFrom);
         if (fromDataset == null) {

http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/121e1d9a/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.1.ddl.aql
----------------------------------------------------------------------
diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.1.ddl.aql
b/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.1.ddl.aql
index af15e99..658487b 100644
--- a/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.1.ddl.aql
+++ b/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.1.ddl.aql
@@ -17,12 +17,14 @@
  * under the License.
  */
 /*
- * Description  : Create a feed using the synthetic feed simulator adapter. 
+ * Description  : Create a feed using the synthetic feed simulator adapter.
                   Create a dataset that has an associated ngram index.
-                  The synthetic feed simulator uses the Social-Data generator to generate
data and simulate a feed. 
-                  The feed lasts a configured duration with data arriving at a configured
rate (tweets per second). 
+                  The synthetic feed simulator uses the Social-Data generator
+                  to generate data and simulate a feed.
+                  The feed lasts a configured duration with data arriving at
+                  a configured rate (tweets per second).
                   Verify the existence of data after the feed finishes.
- * Issue        : 711                 
+ * Issue        : 711
  * Expected Res : Success
  * Date         : 8th Feb 2014
  */

http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/121e1d9a/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.2.update.aql
----------------------------------------------------------------------
diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.2.update.aql
b/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.2.update.aql
index b821bc7..4a1a6a4 100644
--- a/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.2.update.aql
+++ b/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.2.update.aql
@@ -17,10 +17,12 @@
  * under the License.
  */
 /*
- * Description  : Create a feed using the synthetic feed simulator adapter.             
     
+ * Description  : Create a feed using the synthetic feed simulator adapter.
                   Create a dataset that has an associated ngram index.
-                  The synthetic feed simulator uses the Social-Data generator to generate
data and simulate a feed.
-                  The feed lasts a configured duration with data arriving at a configured
rate (tweets per second).
+                  The synthetic feed simulator uses the Social-Data generator
+                  to generate data and simulate a feed.
+                  The feed lasts a configured duration with data arriving at
+                  a configured rate (tweets per second).
                   Verify the existence of data after the feed finishes.
  * Issue        : 711
  * Expected Res : Success

http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/121e1d9a/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.3.query.aql
----------------------------------------------------------------------
diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.3.query.aql
b/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.3.query.aql
index 57b8ecd..177bd98 100644
--- a/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.3.query.aql
+++ b/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.3.query.aql
@@ -17,10 +17,12 @@
  * under the License.
  */
 /*
- * Description  : Create a feed using the synthetic feed simulator adapter.             
     
+ * Description  : Create a feed using the synthetic feed simulator adapter.
                   Create a dataset that has an associated ngram index.
-                  The synthetic feed simulator uses the Social-Data generator to generate
data and simulate a feed.
-                  The feed lasts a configured duration with data arriving at a configured
rate (tweets per second).
+                  The synthetic feed simulator uses the Social-Data generator
+                  to generate data and simulate a feed.
+                  The feed lasts a configured duration with data arriving at
+                  a configured rate (tweets per second).
                   Verify the existence of data after the feed finishes.
  * Issue        : 711
  * Expected Res : Success

http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/121e1d9a/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.4.sleep.aql
----------------------------------------------------------------------
diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.4.sleep.aql
b/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.4.sleep.aql
new file mode 100644
index 0000000..6b7f502
--- /dev/null
+++ b/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.4.sleep.aql
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description  : Create a feed using the synthetic feed simulator adapter.
+                  Create a dataset that has an associated ngram index.
+                  The synthetic feed simulator uses the Social-Data generator
+                  to generate data and simulate a feed.
+                  The feed lasts a configured duration with data arriving at
+                  a configured rate (tweets per second).
+                  Verify the existence of data after the feed finishes.
+ * Issue        : 711
+ * Expected Res : Success
+ * Date         : 8th Feb 2014
+ */
+2000
\ No newline at end of file


Mime
View raw message