Return-Path: X-Original-To: apmail-usergrid-commits-archive@minotaur.apache.org Delivered-To: apmail-usergrid-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 51A281730F for ; Wed, 7 Oct 2015 21:14:27 +0000 (UTC) Received: (qmail 78756 invoked by uid 500); 7 Oct 2015 21:14:27 -0000 Delivered-To: apmail-usergrid-commits-archive@usergrid.apache.org Received: (qmail 78729 invoked by uid 500); 7 Oct 2015 21:14:27 -0000 Mailing-List: contact commits-help@usergrid.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@usergrid.apache.org Delivered-To: mailing list commits@usergrid.apache.org Received: (qmail 78691 invoked by uid 99); 7 Oct 2015 21:14:27 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 07 Oct 2015 21:14:27 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 1EDCCE042D; Wed, 7 Oct 2015 21:14:27 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: sfeldman@apache.org To: commits@usergrid.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: usergrid git commit: buffer size fix Date: Wed, 7 Oct 2015 21:14:27 +0000 (UTC) Repository: usergrid Updated Branches: refs/heads/2.1-release 79caa09d4 -> f9f082531 buffer size fix Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/f9f08253 Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/f9f08253 Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/f9f08253 Branch: refs/heads/2.1-release Commit: f9f0825315c22ecfa1375189bf70020cde868661 Parents: 79caa09 Author: Shawn Feldman Authored: Wed Oct 7 15:14:22 2015 -0600 Committer: Shawn Feldman Committed: Wed Oct 7 15:14:22 2015 -0600 ---------------------------------------------------------------------- .../asyncevents/AmazonAsyncEventService.java | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/usergrid/blob/f9f08253/stack/core/src/main/java/org/apache/usergrid/corepersistence/asyncevents/AmazonAsyncEventService.java ---------------------------------------------------------------------- diff --git a/stack/core/src/main/java/org/apache/usergrid/corepersistence/asyncevents/AmazonAsyncEventService.java b/stack/core/src/main/java/org/apache/usergrid/corepersistence/asyncevents/AmazonAsyncEventService.java index a5342ea..a9e2459 100644 --- a/stack/core/src/main/java/org/apache/usergrid/corepersistence/asyncevents/AmazonAsyncEventService.java +++ b/stack/core/src/main/java/org/apache/usergrid/corepersistence/asyncevents/AmazonAsyncEventService.java @@ -255,6 +255,7 @@ public class AmazonAsyncEventService implements AsyncEventService { logger.debug("handleMessages with {} message", messages.size()); } + final int bufferSize = messages.size(); Observable masterObservable = Observable.from(messages).flatMap(message -> { AsyncEvent event = null; try{ @@ -305,10 +306,8 @@ public class AmazonAsyncEventService implements AsyncEventService { //filter for success, send to the index(optional), ack return masterObservable - //remove unsuccessful - .filter(indexEventResult -> indexEventResult.getQueueMessage().isPresent()) //take the max - .buffer( MAX_TAKE ) + .buffer(250, TimeUnit.MILLISECONDS, bufferSize) //map them to index results and return them .flatMap(indexEventResults -> { IndexOperationMessage combined = new IndexOperationMessage(); @@ -322,8 +321,13 @@ public class AmazonAsyncEventService implements AsyncEventService { //ack after successful completion of the operation. return indexProducer.put(combined) + //change observable type .flatMap(indexOperationMessage -> Observable.from(indexEventResults)) + //remove unsuccessful + .filter(indexEventResult -> indexEventResult.getQueueMessage().isPresent()) + //measure .doOnNext(indexEventResult -> messageCycle.update(System.currentTimeMillis() - indexEventResult.getCreationTime())) + //return the queue messages to ack .map(result -> result.getQueueMessage().get()); });