Return-Path: X-Original-To: apmail-asterixdb-commits-archive@minotaur.apache.org Delivered-To: apmail-asterixdb-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7801519E3C for ; Wed, 23 Mar 2016 19:03:43 +0000 (UTC) Received: (qmail 95184 invoked by uid 500); 23 Mar 2016 19:03:43 -0000 Delivered-To: apmail-asterixdb-commits-archive@asterixdb.apache.org Received: (qmail 95145 invoked by uid 500); 23 Mar 2016 19:03:43 -0000 Mailing-List: contact commits-help@asterixdb.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@asterixdb.incubator.apache.org Delivered-To: mailing list commits@asterixdb.incubator.apache.org Received: (qmail 95116 invoked by uid 99); 23 Mar 2016 19:03:43 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Mar 2016 19:03:43 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id AFD78C8C4A for ; Wed, 23 Mar 2016 19:03:42 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -3.23 X-Spam-Level: X-Spam-Status: No, score=-3.23 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled Received: from mx2-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id r3Wiey1C8pS9 for ; Wed, 23 Mar 2016 19:03:37 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx2-lw-us.apache.org (ASF Mail Server at mx2-lw-us.apache.org) with SMTP id C34475FAC2 for ; Wed, 23 Mar 2016 19:03:35 +0000 (UTC) Received: (qmail 94874 invoked by uid 99); 23 Mar 2016 19:03:35 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Mar 2016 19:03:35 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 08A29DFF10; Wed, 23 Mar 2016 19:03:35 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: amoudi@apache.org To: commits@asterixdb.incubator.apache.org Date: Wed, 23 Mar 2016 19:03:36 -0000 Message-Id: In-Reply-To: <2cd44d36600d47fd9a339cbec78f8485@git.apache.org> References: <2cd44d36600d47fd9a339cbec78f8485@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [2/2] incubator-asterixdb git commit: Fixed Feed Connect Statement Fixed Feed Connect Statement This change includes two fixes: 1. Feed connect doesn't return until the connection is complete. 2. When using wait for completion, it waits until all the jobs complete. Change-Id: I416bf4917b1f5cea687d1202c435f7183136cf1f Reviewed-on: https://asterix-gerrit.ics.uci.edu/726 Tested-by: Jenkins Reviewed-by: Murtadha Hubail Project: http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/commit/b2a5170c Tree: http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/tree/b2a5170c Diff: http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/diff/b2a5170c Branch: refs/heads/master Commit: b2a5170c6f09780f6bd697450b98bb0f07c563ab Parents: 9486133 Author: Abdullah Alamoudi Authored: Wed Mar 23 12:59:25 2016 +0300 Committer: abdullah alamoudi Committed: Wed Mar 23 11:58:02 2016 -0700 ---------------------------------------------------------------------- .../external/FeedJobNotificationHandler.java | 154 ++++++++++++------- .../app/external/FeedLifecycleListener.java | 35 +++-- .../asterix/aql/translator/QueryTranslator.java | 6 +- .../asterix/messaging/CCMessageBroker.java | 12 +- ...change-feed-with-meta-pk-in-meta.3.query.aql | 29 ++++ ...change-feed-with-meta-pk-in-meta.3.sleep.aql | 24 --- ...change-feed-with-meta-pk-in-meta.4.query.aql | 5 +- ...change-feed-with-meta-pk-in-meta.5.query.aql | 12 +- .../change-feed-with-meta-pk-in-meta.6.ddl.aql | 24 +++ ...change-feed-with-meta-pk-in-meta.6.query.aql | 34 ---- .../change-feed-with-meta-pk-in-meta.7.ddl.aql | 24 --- ...ataverse-with-disconnected-feed.2.update.aql | 1 + ...p-dataverse-with-disconnected-feed.3.ddl.aql | 26 ++++ ...dataverse-with-disconnected-feed.3.sleep.aql | 24 --- ...ataverse-with-disconnected-feed.4.update.aql | 26 ---- ...p-dataverse-with-disconnected-feed.5.ddl.aql | 26 ---- .../feed-push-socket.3.server.aql | 26 ++++ .../feed-push-socket.3.sleep.aql | 26 ---- .../feed-push-socket.4.server.aql | 26 ---- .../feed-push-socket.4.sleep.aql | 26 ++++ .../feed-push-socket.5.sleep.aql | 26 ---- .../feed-push-socket.5.update.aql | 28 ++++ .../feed-push-socket.6.query.aql | 31 ++++ .../feed-push-socket.6.update.aql | 28 ---- .../feed-push-socket.7.query.aql | 31 ---- .../feed-push-socket.7.server.aql | 27 ++++ .../feed-push-socket/feed-push-socket.8.ddl.aql | 28 ++++ .../feed-push-socket.8.server.aql | 27 ---- .../feed-push-socket/feed-push-socket.9.ddl.aql | 28 ---- .../feed-with-meta-pk-in-meta.3.query.aql | 29 ++++ .../feed-with-meta-pk-in-meta.3.sleep.aql | 24 --- .../feed-with-meta-pk-in-meta.4.query.aql | 8 +- .../feed-with-meta-pk-in-meta.5.ddl.aql | 24 +++ .../feed-with-meta-pk-in-meta.5.query.aql | 29 ---- .../feed-with-meta-pk-in-meta.6.ddl.aql | 24 --- .../messaging/api/IApplicationMessage.java | 3 +- .../external/feed/api/FeedOperationCounter.java | 65 ++++++++ .../feed/message/FeedProviderReadyMessage.java | 48 ++++++ .../FeedCollectOperatorDescriptor.java | 17 +- .../FeedIntakeOperatorNodePushable.java | 4 + 40 files changed, 567 insertions(+), 528 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJobNotificationHandler.java ---------------------------------------------------------------------- diff --git a/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJobNotificationHandler.java b/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJobNotificationHandler.java index d729680..04f20fb 100644 --- a/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJobNotificationHandler.java +++ b/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJobNotificationHandler.java @@ -32,15 +32,15 @@ import java.util.concurrent.LinkedBlockingQueue; import java.util.logging.Level; import java.util.logging.Logger; -import org.apache.commons.lang3.StringUtils; -import org.apache.asterix.app.external.FeedLifecycleListener.Message; +import org.apache.asterix.app.external.FeedLifecycleListener.FeedEvent; import org.apache.asterix.app.external.FeedWorkCollection.SubscribeFeedWork; import org.apache.asterix.common.exceptions.ACIDException; +import org.apache.asterix.external.feed.api.FeedOperationCounter; import org.apache.asterix.external.feed.api.IFeedJoint; -import org.apache.asterix.external.feed.api.IFeedLifecycleEventSubscriber; -import org.apache.asterix.external.feed.api.IIntakeProgressTracker; import org.apache.asterix.external.feed.api.IFeedJoint.State; +import org.apache.asterix.external.feed.api.IFeedLifecycleEventSubscriber; import org.apache.asterix.external.feed.api.IFeedLifecycleEventSubscriber.FeedLifecycleEvent; +import org.apache.asterix.external.feed.api.IIntakeProgressTracker; import org.apache.asterix.external.feed.management.FeedConnectionId; import org.apache.asterix.external.feed.management.FeedConnectionRequest; import org.apache.asterix.external.feed.management.FeedId; @@ -59,6 +59,7 @@ import org.apache.asterix.external.operators.FeedIntakeOperatorDescriptor; import org.apache.asterix.external.operators.FeedMetaOperatorDescriptor; import org.apache.asterix.metadata.feeds.BuiltinFeedPolicies; import org.apache.asterix.om.util.AsterixAppContextInfo; +import org.apache.commons.lang3.StringUtils; import org.apache.hyracks.algebricks.common.utils.Pair; import org.apache.hyracks.algebricks.runtime.base.IPushRuntimeFactory; import org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor; @@ -78,37 +79,43 @@ public class FeedJobNotificationHandler implements Runnable { private static final Logger LOGGER = Logger.getLogger(FeedJobNotificationHandler.class.getName()); - private final LinkedBlockingQueue inbox; + private final LinkedBlockingQueue inbox; private final Map> eventSubscribers; private final Map jobInfos; private final Map intakeJobInfos; private final Map connectJobInfos; - private final Map> feedPipeline; + private final Map>> feedPipeline; private final Map> feedIntakeProgressTrackers; - public FeedJobNotificationHandler(LinkedBlockingQueue inbox) { + public FeedJobNotificationHandler(LinkedBlockingQueue inbox) { this.inbox = inbox; this.jobInfos = new HashMap(); this.intakeJobInfos = new HashMap(); this.connectJobInfos = new HashMap(); - this.feedPipeline = new HashMap>(); + this.feedPipeline = new HashMap>>(); this.eventSubscribers = new HashMap>(); this.feedIntakeProgressTrackers = new HashMap>(); } @Override public void run() { - Message mesg; + FeedEvent event; while (true) { try { - mesg = inbox.take(); - switch (mesg.messageKind) { + event = inbox.take(); + switch (event.eventKind) { case JOB_START: - handleJobStartMessage(mesg); + handleJobStartEvent(event); break; case JOB_FINISH: - handleJobFinishMessage(mesg); + handleJobFinishEvent(event); + break; + case PROVIDER_READY: + handleProviderReady(event); + break; + default: + LOGGER.log(Level.WARNING, "Unknown Feed Event"); break; } } catch (Exception e) { @@ -121,11 +128,11 @@ public class FeedJobNotificationHandler implements Runnable { public void registerFeedIntakeProgressTracker(FeedConnectionId connectionId, IIntakeProgressTracker feedIntakeProgressTracker) { if (feedIntakeProgressTrackers.get(connectionId) == null) { - this.feedIntakeProgressTrackers.put(connectionId, new Pair( - feedIntakeProgressTracker, 0L)); + this.feedIntakeProgressTrackers.put(connectionId, + new Pair(feedIntakeProgressTracker, 0L)); } else { - throw new IllegalStateException(" Progress tracker for connection " + connectionId - + " is alreader registered"); + throw new IllegalStateException( + " Progress tracker for connection " + connectionId + " is alreader registered"); } } @@ -149,29 +156,35 @@ public class FeedJobNotificationHandler implements Runnable { return connectJobInfos.values(); } - public void registerFeedJoint(IFeedJoint feedJoint) { - List feedJointsOnPipeline = feedPipeline.get(feedJoint.getOwnerFeedId()); + public synchronized void registerFeedJoint(IFeedJoint feedJoint, int numOfPrividers) { + Pair> feedJointsOnPipeline = feedPipeline + .get(feedJoint.getOwnerFeedId()); + if (feedJointsOnPipeline == null) { - feedJointsOnPipeline = new ArrayList(); + feedJointsOnPipeline = new Pair>( + new FeedOperationCounter(numOfPrividers, 1), new ArrayList()); feedPipeline.put(feedJoint.getOwnerFeedId(), feedJointsOnPipeline); - feedJointsOnPipeline.add(feedJoint); + feedJointsOnPipeline.second.add(feedJoint); } else { - if (!feedJointsOnPipeline.contains(feedJoint)) { - feedJointsOnPipeline.add(feedJoint); + if (!feedJointsOnPipeline.second.contains(feedJoint)) { + feedJointsOnPipeline.first.setJobsCount(feedJointsOnPipeline.first.getJobsCount() + 1); + feedJointsOnPipeline.second.add(feedJoint); } else { throw new IllegalArgumentException("Feed joint " + feedJoint + " already registered"); } } } - public void registerFeedIntakeJob(FeedId feedId, JobId jobId, JobSpecification jobSpec) throws HyracksDataException { + public void registerFeedIntakeJob(FeedId feedId, JobId jobId, JobSpecification jobSpec) + throws HyracksDataException { if (jobInfos.get(jobId) != null) { throw new IllegalStateException("Feed job already registered"); } - List joints = feedPipeline.get(feedId); + Pair> pair = feedPipeline.containsKey(feedId) ? feedPipeline.get(feedId) + : null; IFeedJoint intakeJoint = null; - for (IFeedJoint joint : joints) { + for (IFeedJoint joint : pair.second) { if (joint.getType().equals(IFeedJoint.FeedJointType.INTAKE)) { intakeJoint = joint; break; @@ -181,6 +194,7 @@ public class FeedJobNotificationHandler implements Runnable { if (intakeJoint != null) { FeedIntakeInfo intakeJobInfo = new FeedIntakeInfo(jobId, FeedJobState.CREATED, FeedJobInfo.JobType.INTAKE, feedId, intakeJoint, jobSpec); + pair.first.setFeedJobInfo(intakeJobInfo); intakeJobInfos.put(feedId, intakeJobInfo); jobInfos.put(jobId, intakeJobInfo); @@ -188,8 +202,8 @@ public class FeedJobNotificationHandler implements Runnable { LOGGER.info("Registered feed intake [" + jobId + "]" + " for feed " + feedId); } } else { - throw new HyracksDataException("Could not register feed intake job [" + jobId + "]" + " for feed " - + feedId); + throw new HyracksDataException( + "Could not register feed intake job [" + jobId + "]" + " for feed " + feedId); } } @@ -199,7 +213,7 @@ public class FeedJobNotificationHandler implements Runnable { throw new IllegalStateException("Feed job already registered"); } - List feedJoints = feedPipeline.get(sourceFeedId); + List feedJoints = feedPipeline.get(sourceFeedId).second; FeedConnectionId cid = null; IFeedJoint sourceFeedJoint = null; for (IFeedJoint joint : feedJoints) { @@ -238,7 +252,7 @@ public class FeedJobNotificationHandler implements Runnable { intakeJobInfos.remove(info.getFeedId()); if (!info.getState().equals(FeedJobState.UNDER_RECOVERY)) { - List joints = feedPipeline.get(info.getFeedId()); + List joints = feedPipeline.get(info.getFeedId()).second; joints.remove(info.getIntakeFeedJoint()); if (LOGGER.isLoggable(Level.INFO)) { @@ -252,7 +266,7 @@ public class FeedJobNotificationHandler implements Runnable { } - private void handleJobStartMessage(Message message) throws Exception { + private void handleJobStartEvent(FeedEvent message) throws Exception { FeedJobInfo jobInfo = jobInfos.get(message.jobId); switch (jobInfo.getJobType()) { case INTAKE: @@ -265,7 +279,7 @@ public class FeedJobNotificationHandler implements Runnable { } - private void handleJobFinishMessage(Message message) throws Exception { + private void handleJobFinishEvent(FeedEvent message) throws Exception { FeedJobInfo jobInfo = jobInfos.get(message.jobId); switch (jobInfo.getJobType()) { case INTAKE: @@ -276,12 +290,22 @@ public class FeedJobNotificationHandler implements Runnable { break; case FEED_CONNECT: if (LOGGER.isLoggable(Level.INFO)) { - LOGGER.info("Collect Job finished for " + (FeedConnectJobInfo) jobInfo); + LOGGER.info("Collect Job finished for " + jobInfo); } handleFeedCollectJobFinishMessage((FeedConnectJobInfo) jobInfo); break; } + } + private void handleProviderReady(FeedEvent message) { + FeedIntakeInfo jobInfo = (FeedIntakeInfo) jobInfos.get(message.jobId); + Pair> feedCounter = feedPipeline.get(message.feedId); + feedCounter.first.setProvidersCount(feedCounter.first.getProvidersCount() - 1);; + if (feedCounter.first.getProvidersCount() == 0) { + jobInfo.getIntakeFeedJoint().setState(State.ACTIVE); + jobInfo.setState(FeedJobState.ACTIVE); + notifyFeedEventSubscribers(jobInfo, FeedLifecycleEvent.FEED_INTAKE_STARTED); + } } private synchronized void handleIntakeJobStartMessage(FeedIntakeInfo intakeJobInfo) throws Exception { @@ -306,19 +330,16 @@ public class FeedJobNotificationHandler implements Runnable { } // intakeLocations is an ordered list; element at position i corresponds to location of i'th instance of operator intakeJobInfo.setIntakeLocation(intakeLocations); - intakeJobInfo.getIntakeFeedJoint().setState(State.ACTIVE); - intakeJobInfo.setState(FeedJobState.ACTIVE); - - // notify event listeners - notifyFeedEventSubscribers(intakeJobInfo, FeedLifecycleEvent.FEED_INTAKE_STARTED); } private void handleCollectJobStartMessage(FeedConnectJobInfo cInfo) throws RemoteException, ACIDException { // set locations of feed sub-operations (intake, compute, store) setLocations(cInfo); + Pair> pair = feedPipeline.get(cInfo.getConnectionId().getFeedId()); + pair.first.setJobsCount(pair.first.getJobsCount() + 1); // activate joints - List joints = feedPipeline.get(cInfo.getConnectionId().getFeedId()); + List joints = pair.second; for (IFeedJoint joint : joints) { if (joint.getProvider().equals(cInfo.getConnectionId())) { joint.setState(State.ACTIVE); @@ -328,7 +349,6 @@ public class FeedJobNotificationHandler implements Runnable { } } cInfo.setState(FeedJobState.ACTIVE); - // register activity in metadata registerFeedActivity(cInfo); // notify event listeners @@ -413,20 +433,25 @@ public class FeedJobNotificationHandler implements Runnable { return connectJobInfos.get(connectionId).getState(); } - private void handleFeedIntakeJobFinishMessage(FeedIntakeInfo intakeInfo, Message message) throws Exception { + private void handleFeedIntakeJobFinishMessage(FeedIntakeInfo intakeInfo, FeedEvent message) throws Exception { IHyracksClientConnection hcc = AsterixAppContextInfo.getInstance().getHcc(); JobInfo info = hcc.getJobInfo(message.jobId); JobStatus status = info.getStatus(); - FeedLifecycleEvent event; - event = status.equals(JobStatus.FAILURE) ? FeedLifecycleEvent.FEED_INTAKE_FAILURE - : FeedLifecycleEvent.FEED_ENDED; - + FeedId feedId = intakeInfo.getFeedId(); + Pair> pair = feedPipeline.get(feedId); + pair.first.setJobsCount(pair.first.getJobsCount() - 1); + if (status.equals(JobStatus.FAILURE)) { + pair.first.setFailedIngestion(true); + } // remove feed joints deregisterFeedIntakeJob(message.jobId); // notify event listeners - notifyFeedEventSubscribers(intakeInfo, event); - + if (pair.first.getJobsCount() == 0) { + feedPipeline.remove(feedId); + notifyFeedEventSubscribers(intakeInfo, pair.first.isFailedIngestion() + ? FeedLifecycleEvent.FEED_INTAKE_FAILURE : FeedLifecycleEvent.FEED_ENDED); + } } private void handleFeedCollectJobFinishMessage(FeedConnectJobInfo cInfo) throws Exception { @@ -446,7 +471,8 @@ public class FeedJobNotificationHandler implements Runnable { IFeedJoint feedJoint = cInfo.getSourceFeedJoint(); feedJoint.removeReceiver(connectionId); if (LOGGER.isLoggable(Level.INFO)) { - LOGGER.info("Subscription " + cInfo.getConnectionId() + " completed successfully. Removed subscription"); + LOGGER.info( + "Subscription " + cInfo.getConnectionId() + " completed successfully. Removed subscription"); } removeFeedJointsPostPipelineTermination(cInfo.getConnectionId()); } @@ -458,6 +484,15 @@ public class FeedJobNotificationHandler implements Runnable { } deregisterFeedActivity(cInfo); + Pair> pair = feedPipeline + .get(cInfo.getSourceFeedJoint().getFeedJointKey().getFeedId()); + pair.first.setJobsCount(pair.first.getJobsCount() - 1); + if (pair.first.getJobsCount() == 0) { + notifyFeedEventSubscribers(pair.first.getFeedJobInfo(), pair.first.isFailedIngestion() + ? FeedLifecycleEvent.FEED_INTAKE_FAILURE : FeedLifecycleEvent.FEED_ENDED); + feedPipeline.remove(cInfo.getSourceFeedJoint().getFeedJointKey().getFeedId()); + } + // notify event listeners FeedLifecycleEvent event = failure ? FeedLifecycleEvent.FEED_COLLECT_FAILURE : FeedLifecycleEvent.FEED_ENDED; notifyFeedEventSubscribers(cInfo, event); @@ -486,11 +521,11 @@ public class FeedJobNotificationHandler implements Runnable { feedActivityDetails.put(FeedActivity.FeedActivityDetails.FEED_CONNECT_TIMESTAMP, (new Date()).toString()); try { - FeedActivity feedActivity = new FeedActivity(cInfo.getConnectionId().getFeedId().getDataverse(), cInfo - .getConnectionId().getFeedId().getFeedName(), cInfo.getConnectionId().getDatasetName(), + FeedActivity feedActivity = new FeedActivity(cInfo.getConnectionId().getFeedId().getDataverse(), + cInfo.getConnectionId().getFeedId().getFeedName(), cInfo.getConnectionId().getDatasetName(), feedActivityDetails); - CentralFeedManager.getInstance().getFeedLoadManager() - .reportFeedActivity(cInfo.getConnectionId(), feedActivity); + CentralFeedManager.getInstance().getFeedLoadManager().reportFeedActivity(cInfo.getConnectionId(), + feedActivity); } catch (Exception e) { e.printStackTrace(); @@ -514,7 +549,7 @@ public class FeedJobNotificationHandler implements Runnable { public void removeFeedJointsPostPipelineTermination(FeedConnectionId connectionId) { FeedConnectJobInfo cInfo = connectJobInfos.get(connectionId); - List feedJoints = feedPipeline.get(connectionId.getFeedId()); + List feedJoints = feedPipeline.get(connectionId.getFeedId()).second; IFeedJoint sourceJoint = cInfo.getSourceFeedJoint(); List all = sourceJoint.getReceivers(); @@ -534,7 +569,7 @@ public class FeedJobNotificationHandler implements Runnable { } public List getFeedComputeLocations(FeedId feedId) { - List feedJoints = feedPipeline.get(feedId); + List feedJoints = feedPipeline.get(feedId).second; for (IFeedJoint joint : feedJoints) { if (joint.getFeedJointKey().getFeedId().equals(feedId)) { return connectJobInfos.get(joint.getProvider()).getComputeLocations(); @@ -578,7 +613,8 @@ public class FeedJobNotificationHandler implements Runnable { //============================ public boolean isFeedPointAvailable(FeedJointKey feedJointKey) { - List joints = feedPipeline.get(feedJointKey.getFeedId()); + List joints = feedPipeline.containsKey(feedJointKey.getFeedId()) + ? feedPipeline.get(feedJointKey.getFeedId()).second : null; if (joints != null && !joints.isEmpty()) { for (IFeedJoint joint : joints) { if (joint.getFeedJointKey().equals(feedJointKey)) { @@ -598,7 +634,8 @@ public class FeedJobNotificationHandler implements Runnable { } public IFeedJoint getFeedJoint(FeedJointKey feedPointKey) { - List joints = feedPipeline.get(feedPointKey.getFeedId()); + List joints = feedPipeline.containsKey(feedPointKey.getFeedId()) + ? feedPipeline.get(feedPointKey.getFeedId()).second : null; if (joints != null && !joints.isEmpty()) { for (IFeedJoint joint : joints) { if (joint.getFeedJointKey().equals(feedPointKey)) { @@ -615,7 +652,8 @@ public class FeedJobNotificationHandler implements Runnable { return feedJoint; } else { String jointKeyString = feedJointKey.getStringRep(); - List jointsOnPipeline = feedPipeline.get(feedJointKey.getFeedId()); + List jointsOnPipeline = feedPipeline.containsKey(feedJointKey.getFeedId()) + ? feedPipeline.get(feedJointKey.getFeedId()).second : null; IFeedJoint candidateJoint = null; if (jointsOnPipeline != null) { for (IFeedJoint joint : jointsOnPipeline) { @@ -638,7 +676,7 @@ public class FeedJobNotificationHandler implements Runnable { } public IFeedJoint getFeedPoint(FeedId sourceFeedId, IFeedJoint.FeedJointType type) { - List joints = feedPipeline.get(sourceFeedId); + List joints = feedPipeline.get(sourceFeedId).second; for (IFeedJoint joint : joints) { if (joint.getType().equals(type)) { return joint; http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/main/java/org/apache/asterix/app/external/FeedLifecycleListener.java ---------------------------------------------------------------------- diff --git a/asterix-app/src/main/java/org/apache/asterix/app/external/FeedLifecycleListener.java b/asterix-app/src/main/java/org/apache/asterix/app/external/FeedLifecycleListener.java index 8e44af4..d7129b8 100644 --- a/asterix-app/src/main/java/org/apache/asterix/app/external/FeedLifecycleListener.java +++ b/asterix-app/src/main/java/org/apache/asterix/app/external/FeedLifecycleListener.java @@ -88,7 +88,7 @@ public class FeedLifecycleListener implements IFeedLifecycleListener { public static FeedLifecycleListener INSTANCE = new FeedLifecycleListener(); private static final ILangCompilationProvider compilationProvider = new AqlCompilationProvider(); - private final LinkedBlockingQueue jobEventInbox; + private final LinkedBlockingQueue jobEventInbox; private final LinkedBlockingQueue responseInbox; private final Map> dependentFeeds = new HashMap>(); private final Map> feedReportQueue; @@ -99,7 +99,7 @@ public class FeedLifecycleListener implements IFeedLifecycleListener { private ClusterState state; private FeedLifecycleListener() { - this.jobEventInbox = new LinkedBlockingQueue(); + this.jobEventInbox = new LinkedBlockingQueue(); this.feedJobNotificationHandler = new FeedJobNotificationHandler(jobEventInbox); this.responseInbox = new LinkedBlockingQueue(); this.feedWorkRequestResponseHandler = new FeedWorkRequestResponseHandler(responseInbox); @@ -114,14 +114,14 @@ public class FeedLifecycleListener implements IFeedLifecycleListener { @Override public void notifyJobStart(JobId jobId) throws HyracksException { if (feedJobNotificationHandler.isRegisteredFeedJob(jobId)) { - jobEventInbox.add(new Message(jobId, Message.MessageKind.JOB_START)); + jobEventInbox.add(new FeedEvent(jobId, FeedEvent.EventKind.JOB_START)); } } @Override public void notifyJobFinish(JobId jobId) throws HyracksException { if (feedJobNotificationHandler.isRegisteredFeedJob(jobId)) { - jobEventInbox.add(new Message(jobId, Message.MessageKind.JOB_FINISH)); + jobEventInbox.add(new FeedEvent(jobId, FeedEvent.EventKind.JOB_FINISH)); } else { if (LOGGER.isLoggable(Level.INFO)) { LOGGER.info("NO NEED TO NOTIFY JOB FINISH!"); @@ -178,19 +178,26 @@ public class FeedLifecycleListener implements IFeedLifecycleListener { return feedJobNotificationHandler.getFeedJobState(connectionId); } - public static class Message { + public static class FeedEvent { public JobId jobId; + public FeedId feedId; - public enum MessageKind { + public enum EventKind { JOB_START, - JOB_FINISH + JOB_FINISH, + PROVIDER_READY } - public MessageKind messageKind; + public EventKind eventKind; - public Message(JobId jobId, MessageKind msgKind) { + public FeedEvent(JobId jobId, EventKind eventKind) { + this(jobId, eventKind, null); + } + + public FeedEvent(JobId jobId, EventKind eventKind, FeedId feedId) { this.jobId = jobId; - this.messageKind = msgKind; + this.eventKind = eventKind; + this.feedId = feedId; } } @@ -469,8 +476,8 @@ public class FeedLifecycleListener implements IFeedLifecycleListener { return feedJobNotificationHandler.isFeedPointAvailable(feedJointKey); } - public void registerFeedJoint(IFeedJoint feedJoint) { - feedJobNotificationHandler.registerFeedJoint(feedJoint); + public void registerFeedJoint(IFeedJoint feedJoint, int numOfPrividers) { + feedJobNotificationHandler.registerFeedJoint(feedJoint, numOfPrividers); } public IFeedJoint getFeedJoint(FeedJointKey feedJointKey) { @@ -496,4 +503,8 @@ public class FeedLifecycleListener implements IFeedLifecycleListener { return feedJobNotificationHandler.getFeedCollectJobId(connectionId); } + public void notifyProviderReady(FeedId feedId, JobId jobId) { + jobEventInbox.add(new FeedEvent(jobId, FeedEvent.EventKind.PROVIDER_READY, feedId)); + } + } http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/main/java/org/apache/asterix/aql/translator/QueryTranslator.java ---------------------------------------------------------------------- diff --git a/asterix-app/src/main/java/org/apache/asterix/aql/translator/QueryTranslator.java b/asterix-app/src/main/java/org/apache/asterix/aql/translator/QueryTranslator.java index 213b090..67bf3bb 100644 --- a/asterix-app/src/main/java/org/apache/asterix/aql/translator/QueryTranslator.java +++ b/asterix-app/src/main/java/org/apache/asterix/aql/translator/QueryTranslator.java @@ -26,7 +26,6 @@ import java.io.InputStream; import java.io.InputStreamReader; import java.rmi.RemoteException; import java.util.ArrayList; -import java.util.Collections; import java.util.Date; import java.util.HashMap; import java.util.Iterator; @@ -2205,8 +2204,9 @@ public class QueryTranslator extends AbstractLangTranslator { metadataProvider, policyAccessor); //adapter configuration are valid at this stage //register the feed joints (these are auto-de-registered) + int numOfPrividers = pair.second.getPartitionConstraint().getLocations().length; for (IFeedJoint fj : triple.third) { - FeedLifecycleListener.INSTANCE.registerFeedJoint(fj); + FeedLifecycleListener.INSTANCE.registerFeedJoint(fj, numOfPrividers); } JobUtils.runJob(hcc, pair.first, false); /* @@ -2220,7 +2220,7 @@ public class QueryTranslator extends AbstractLangTranslator { eventSubscriber.assertEvent(FeedLifecycleEvent.FEED_INTAKE_STARTED); } else { for (IFeedJoint fj : triple.third) { - FeedLifecycleListener.INSTANCE.registerFeedJoint(fj); + FeedLifecycleListener.INSTANCE.registerFeedJoint(fj, 0); } } MetadataManager.INSTANCE.commitTransaction(mdTxnCtx); http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/main/java/org/apache/asterix/messaging/CCMessageBroker.java ---------------------------------------------------------------------- diff --git a/asterix-app/src/main/java/org/apache/asterix/messaging/CCMessageBroker.java b/asterix-app/src/main/java/org/apache/asterix/messaging/CCMessageBroker.java index 258bc35..2378032 100644 --- a/asterix-app/src/main/java/org/apache/asterix/messaging/CCMessageBroker.java +++ b/asterix-app/src/main/java/org/apache/asterix/messaging/CCMessageBroker.java @@ -25,9 +25,10 @@ import java.util.concurrent.atomic.AtomicLong; import java.util.logging.Level; import java.util.logging.Logger; +import org.apache.asterix.app.external.FeedLifecycleListener; import org.apache.asterix.common.messaging.AbstractApplicationMessage; -import org.apache.asterix.common.messaging.PreparePartitionsFailbackResponseMessage; import org.apache.asterix.common.messaging.CompleteFailbackResponseMessage; +import org.apache.asterix.common.messaging.PreparePartitionsFailbackResponseMessage; import org.apache.asterix.common.messaging.ReportMaxResourceIdMessage; import org.apache.asterix.common.messaging.ReportMaxResourceIdRequestMessage; import org.apache.asterix.common.messaging.ResourceIdRequestMessage; @@ -36,6 +37,7 @@ import org.apache.asterix.common.messaging.TakeoverMetadataNodeResponseMessage; import org.apache.asterix.common.messaging.TakeoverPartitionsResponseMessage; import org.apache.asterix.common.messaging.api.IApplicationMessage; import org.apache.asterix.common.messaging.api.ICCMessageBroker; +import org.apache.asterix.external.feed.message.FeedProviderReadyMessage; import org.apache.asterix.om.util.AsterixClusterProperties; import org.apache.hyracks.api.messages.IMessage; import org.apache.hyracks.api.util.JavaSerializationUtils; @@ -79,12 +81,20 @@ public class CCMessageBroker implements ICCMessageBroker { case COMPLETE_FAILBACK_RESPONSE: handleCompleteFailbcakResponse(message); break; + case FEED_PROVIDER_READY: + handleFeedProviderReady(message); + break; default: LOGGER.warning("Unknown message: " + absMessage.getMessageType()); break; } } + private void handleFeedProviderReady(IMessage message) { + FeedProviderReadyMessage msg = (FeedProviderReadyMessage) message; + FeedLifecycleListener.INSTANCE.notifyProviderReady(msg.getFeedId(), msg.getJobId()); + } + private synchronized void handleResourceIdRequest(IMessage message, String nodeId) throws Exception { ResourceIdRequestMessage msg = (ResourceIdRequestMessage) message; ResourceIdRequestResponseMessage reponse = new ResourceIdRequestResponseMessage(); http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.3.query.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.3.query.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.3.query.aql new file mode 100644 index 0000000..b696974 --- /dev/null +++ b/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.3.query.aql @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* + * Description : Create a change feed and test ingestion of records + * Expected Res : Success + * Date : 24th Feb 2016 + */ +use dataverse KeyVerse; + +count( + for $d in dataset KVStore + return $d +); http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.3.sleep.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.3.sleep.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.3.sleep.aql deleted file mode 100644 index db6954e..0000000 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.3.sleep.aql +++ /dev/null @@ -1,24 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -/* - * Description : Create a change feed and test ingestion of records - * Expected Res : Success - * Date : 24th Feb 2016 - */ -4000 \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.4.query.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.4.query.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.4.query.aql index b696974..28e9a15 100644 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.4.query.aql +++ b/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.4.query.aql @@ -24,6 +24,7 @@ use dataverse KeyVerse; count( - for $d in dataset KVStore - return $d +for $d in dataset KVStore +distinct by meta()."key" +return 1 ); http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.5.query.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.5.query.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.5.query.aql index 28e9a15..7423399 100644 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.5.query.aql +++ b/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.5.query.aql @@ -23,8 +23,12 @@ */ use dataverse KeyVerse; -count( + for $d in dataset KVStore -distinct by meta()."key" -return 1 -); +group by $vb := meta().vbucket with $d +order by $vb +limit 5 +return { + "vbucket": $vb, + "tuple_count": count($d) +}; http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.6.ddl.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.6.ddl.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.6.ddl.aql new file mode 100644 index 0000000..d282e66 --- /dev/null +++ b/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.6.ddl.aql @@ -0,0 +1,24 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* + * Description : Create a change feed and test ingestion of records + * Expected Res : Success + * Date : 24th Feb 2016 + */ +drop dataverse KeyVerse; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.6.query.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.6.query.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.6.query.aql deleted file mode 100644 index 7423399..0000000 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.6.query.aql +++ /dev/null @@ -1,34 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -/* - * Description : Create a change feed and test ingestion of records - * Expected Res : Success - * Date : 24th Feb 2016 - */ -use dataverse KeyVerse; - - -for $d in dataset KVStore -group by $vb := meta().vbucket with $d -order by $vb -limit 5 -return { - "vbucket": $vb, - "tuple_count": count($d) -}; http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.7.ddl.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.7.ddl.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.7.ddl.aql deleted file mode 100644 index d282e66..0000000 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.7.ddl.aql +++ /dev/null @@ -1,24 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -/* - * Description : Create a change feed and test ingestion of records - * Expected Res : Success - * Date : 24th Feb 2016 - */ -drop dataverse KeyVerse; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.2.update.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.2.update.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.2.update.aql index 4d2f9c4..4c1635f 100644 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.2.update.aql +++ b/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.2.update.aql @@ -26,3 +26,4 @@ use dataverse experiments; set wait-for-completion-feed "false"; connect feed TweetFeed to dataset Tweets; +disconnect feed TweetFeed from dataset Tweets; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.3.ddl.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.3.ddl.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.3.ddl.aql new file mode 100644 index 0000000..5684b1c --- /dev/null +++ b/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.3.ddl.aql @@ -0,0 +1,26 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* + * Description : Drop a dataverse with disconnected feed + * Expected Res : Success + * Date : 24th Feb 2016 + */ + +use dataverse experiments; +drop dataverse experiments; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.3.sleep.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.3.sleep.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.3.sleep.aql deleted file mode 100644 index e70df33..0000000 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.3.sleep.aql +++ /dev/null @@ -1,24 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -/* - * Description : Drop a dataverse with disconnected feed - * Expected Res : Success - * Date : 24th Feb 2016 - */ -3000 \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.4.update.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.4.update.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.4.update.aql deleted file mode 100644 index 34d6285..0000000 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.4.update.aql +++ /dev/null @@ -1,26 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -/* - * Description : Drop a dataverse with disconnected feed - * Expected Res : Success - * Date : 24th Feb 2016 - */ - -use dataverse experiments; -disconnect feed TweetFeed from dataset Tweets; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.5.ddl.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.5.ddl.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.5.ddl.aql deleted file mode 100644 index 5684b1c..0000000 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/drop-dataverse-with-disconnected-feed/drop-dataverse-with-disconnected-feed.5.ddl.aql +++ /dev/null @@ -1,26 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -/* - * Description : Drop a dataverse with disconnected feed - * Expected Res : Success - * Date : 24th Feb 2016 - */ - -use dataverse experiments; -drop dataverse experiments; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.3.server.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.3.server.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.3.server.aql new file mode 100644 index 0000000..5b610de --- /dev/null +++ b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.3.server.aql @@ -0,0 +1,26 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* + * Description : Create a socket feed with a client that pushes + * 10 records. The feed is connected to a dataset that is then + * queried for the data. + * Expected Res : Success + * Date : 24th Feb 2016 + */ +start client 10001 file-client localhost ../asterix-app/data/twitter/tw_messages.adm 500 50 1000 \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.3.sleep.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.3.sleep.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.3.sleep.aql deleted file mode 100644 index eb18795..0000000 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.3.sleep.aql +++ /dev/null @@ -1,26 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -/* - * Description : Create a socket feed with a client that pushes - * 10 records. The feed is connected to a dataset that is then - * queried for the data. - * Expected Res : Success - * Date : 24th Feb 2016 - */ -3000 \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.4.server.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.4.server.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.4.server.aql deleted file mode 100644 index 5b610de..0000000 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.4.server.aql +++ /dev/null @@ -1,26 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -/* - * Description : Create a socket feed with a client that pushes - * 10 records. The feed is connected to a dataset that is then - * queried for the data. - * Expected Res : Success - * Date : 24th Feb 2016 - */ -start client 10001 file-client localhost ../asterix-app/data/twitter/tw_messages.adm 500 50 1000 \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.4.sleep.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.4.sleep.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.4.sleep.aql new file mode 100644 index 0000000..18bbbbc --- /dev/null +++ b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.4.sleep.aql @@ -0,0 +1,26 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* + * Description : Create a socket feed with a client that pushes + * 10 records. The feed is connected to a dataset that is then + * queried for the data. + * Expected Res : Success + * Date : 24th Feb 2016 + */ +10000 \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.5.sleep.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.5.sleep.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.5.sleep.aql deleted file mode 100644 index 18bbbbc..0000000 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.5.sleep.aql +++ /dev/null @@ -1,26 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -/* - * Description : Create a socket feed with a client that pushes - * 10 records. The feed is connected to a dataset that is then - * queried for the data. - * Expected Res : Success - * Date : 24th Feb 2016 - */ -10000 \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.5.update.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.5.update.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.5.update.aql new file mode 100644 index 0000000..0862bae --- /dev/null +++ b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.5.update.aql @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* + * Description : Create a socket feed with a client that pushes + * 10 records. The feed is connected to a dataset that is then + * queried for the data. + * Expected Res : Success + * Date : 24th Feb 2016 + */ + +use dataverse experiments; +disconnect feed TweetFeed from dataset Tweets; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.6.query.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.6.query.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.6.query.aql new file mode 100644 index 0000000..fd8926b --- /dev/null +++ b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.6.query.aql @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* + * Description : Create a socket feed with a client that pushes + * 10 records. The feed is connected to a dataset that is then + * queried for the data. + * Expected Res : Success + * Date : 24th Feb 2016 + */ + +use dataverse experiments; + +for $x in dataset Tweets +order by $x.tweetid +return $x; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.6.update.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.6.update.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.6.update.aql deleted file mode 100644 index 0862bae..0000000 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.6.update.aql +++ /dev/null @@ -1,28 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -/* - * Description : Create a socket feed with a client that pushes - * 10 records. The feed is connected to a dataset that is then - * queried for the data. - * Expected Res : Success - * Date : 24th Feb 2016 - */ - -use dataverse experiments; -disconnect feed TweetFeed from dataset Tweets; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.7.query.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.7.query.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.7.query.aql deleted file mode 100644 index fd8926b..0000000 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.7.query.aql +++ /dev/null @@ -1,31 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -/* - * Description : Create a socket feed with a client that pushes - * 10 records. The feed is connected to a dataset that is then - * queried for the data. - * Expected Res : Success - * Date : 24th Feb 2016 - */ - -use dataverse experiments; - -for $x in dataset Tweets -order by $x.tweetid -return $x; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.7.server.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.7.server.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.7.server.aql new file mode 100644 index 0000000..6753868 --- /dev/null +++ b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.7.server.aql @@ -0,0 +1,27 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* + * Description : Create a socket feed with a client that pushes + * 10 records. The feed is connected to a dataset that is then + * queried for the data. + * Expected Res : Success + * Date : 24th Feb 2016 + */ + +stop 10001 \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.8.ddl.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.8.ddl.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.8.ddl.aql new file mode 100644 index 0000000..1295b97 --- /dev/null +++ b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.8.ddl.aql @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* + * Description : Create a socket feed with a client that pushes + * 10 records. The feed is connected to a dataset that is then + * queried for the data. + * Expected Res : Success + * Date : 24th Feb 2016 + */ + +use dataverse experiments; +drop dataverse experiments; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.8.server.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.8.server.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.8.server.aql deleted file mode 100644 index 6753868..0000000 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.8.server.aql +++ /dev/null @@ -1,27 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -/* - * Description : Create a socket feed with a client that pushes - * 10 records. The feed is connected to a dataset that is then - * queried for the data. - * Expected Res : Success - * Date : 24th Feb 2016 - */ - -stop 10001 \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.9.ddl.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.9.ddl.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.9.ddl.aql deleted file mode 100644 index 1295b97..0000000 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.9.ddl.aql +++ /dev/null @@ -1,28 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -/* - * Description : Create a socket feed with a client that pushes - * 10 records. The feed is connected to a dataset that is then - * queried for the data. - * Expected Res : Success - * Date : 24th Feb 2016 - */ - -use dataverse experiments; -drop dataverse experiments; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.3.query.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.3.query.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.3.query.aql new file mode 100644 index 0000000..b696974 --- /dev/null +++ b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.3.query.aql @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* + * Description : Create a change feed and test ingestion of records + * Expected Res : Success + * Date : 24th Feb 2016 + */ +use dataverse KeyVerse; + +count( + for $d in dataset KVStore + return $d +); http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.3.sleep.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.3.sleep.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.3.sleep.aql deleted file mode 100644 index db6954e..0000000 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.3.sleep.aql +++ /dev/null @@ -1,24 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -/* - * Description : Create a change feed and test ingestion of records - * Expected Res : Success - * Date : 24th Feb 2016 - */ -4000 \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.4.query.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.4.query.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.4.query.aql index b696974..864ce01 100644 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.4.query.aql +++ b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.4.query.aql @@ -23,7 +23,7 @@ */ use dataverse KeyVerse; -count( - for $d in dataset KVStore - return $d -); +for $d in dataset KVStore +order by meta().id +limit 5 +return meta().id; http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.5.ddl.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.5.ddl.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.5.ddl.aql new file mode 100644 index 0000000..d282e66 --- /dev/null +++ b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.5.ddl.aql @@ -0,0 +1,24 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* + * Description : Create a change feed and test ingestion of records + * Expected Res : Success + * Date : 24th Feb 2016 + */ +drop dataverse KeyVerse; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.5.query.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.5.query.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.5.query.aql deleted file mode 100644 index 864ce01..0000000 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.5.query.aql +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -/* - * Description : Create a change feed and test ingestion of records - * Expected Res : Success - * Date : 24th Feb 2016 - */ -use dataverse KeyVerse; - -for $d in dataset KVStore -order by meta().id -limit 5 -return meta().id; http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.6.ddl.aql ---------------------------------------------------------------------- diff --git a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.6.ddl.aql b/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.6.ddl.aql deleted file mode 100644 index d282e66..0000000 --- a/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.6.ddl.aql +++ /dev/null @@ -1,24 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -/* - * Description : Create a change feed and test ingestion of records - * Expected Res : Success - * Date : 24th Feb 2016 - */ -drop dataverse KeyVerse; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-asterixdb/blob/b2a5170c/asterix-common/src/main/java/org/apache/asterix/common/messaging/api/IApplicationMessage.java ---------------------------------------------------------------------- diff --git a/asterix-common/src/main/java/org/apache/asterix/common/messaging/api/IApplicationMessage.java b/asterix-common/src/main/java/org/apache/asterix/common/messaging/api/IApplicationMessage.java index 5d2e263..fba74e8 100644 --- a/asterix-common/src/main/java/org/apache/asterix/common/messaging/api/IApplicationMessage.java +++ b/asterix-common/src/main/java/org/apache/asterix/common/messaging/api/IApplicationMessage.java @@ -35,7 +35,8 @@ public interface IApplicationMessage extends IMessage { PREPARE_PARTITIONS_FAILBACK_RESPONSE, COMPLETE_FAILBACK_REQUEST, COMPLETE_FAILBACK_RESPONSE, - REPLICA_EVENT + REPLICA_EVENT, + FEED_PROVIDER_READY } public abstract ApplicationMessageType getMessageType();