From commits-return-473-archive-asf-public=cust-asf.ponee.io@heron.incubator.apache.org Fri Jul 6 02:23:14 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id C1FF8180674 for ; Fri, 6 Jul 2018 02:23:13 +0200 (CEST) Received: (qmail 93082 invoked by uid 500); 6 Jul 2018 00:23:12 -0000 Mailing-List: contact commits-help@heron.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@heron.incubator.apache.org Delivered-To: mailing list commits@heron.incubator.apache.org Received: (qmail 93073 invoked by uid 99); 6 Jul 2018 00:23:12 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Jul 2018 00:23:12 +0000 From: GitBox To: commits@heron.apache.org Subject: [GitHub] kalimfaria commented on a change in pull request #2951: Counting pending tuples Message-ID: <153083659232.19841.10893498739418705101.gitbox@gitbox.apache.org> Date: Fri, 06 Jul 2018 00:23:12 -0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit kalimfaria commented on a change in pull request #2951: Counting pending tuples URL: https://github.com/apache/incubator-heron/pull/2951#discussion_r200522300 ########## File path: heron/stmgr/src/cpp/manager/instance-server.cpp ########## @@ -74,7 +74,10 @@ const sp_string METRIC_TIME_SPENT_BACK_PRESSURE_AGGR = "__server/__time_spent_ba const sp_string METRIC_TIME_SPENT_BACK_PRESSURE_COMPID = "__time_spent_back_pressure_by_compid/"; // Prefix for connection buffer's metrics const sp_string CONNECTION_BUFFER_BY_INSTANCEID = "__connection_buffer_by_instanceid/"; -// Prefix for connection buffer's length metrics +// Prefix for connection buffer's length metrics. This is different +// from METRIC_DATA_TUPLES_TO_INSTANCES as that counts +// the tuples when they are sent to the instance -- this metric +// will be used to count the tuples as they are received Review comment: From what I understand, tuples can be dropped when they are being removed from the queue because of backpressure. https://github.com/apache/incubator-heron/blob/571a90b7c8452f7922cdc47b91e52de66f5edd70/heron/stmgr/src/cpp/manager/instance-server.cpp#L432 However, only tuples in the form of `HeronTupleSet` are dropped and the ones in the form of `TupleStreamMessage` are not dropped. In addition, I'd like to measure arrival rate of tuples at the stream manager and I think that because tuples are only drained when a certain number of tuples have arrived, that may not translate accurately? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org With regards, Apache Git Services