heron-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] kalimfaria commented on a change in pull request #2951: Counting pending tuples
Date Fri, 06 Jul 2018 00:23:12 GMT
kalimfaria commented on a change in pull request #2951: Counting pending tuples
URL: https://github.com/apache/incubator-heron/pull/2951#discussion_r200522300
 
 

 ##########
 File path: heron/stmgr/src/cpp/manager/instance-server.cpp
 ##########
 @@ -74,7 +74,10 @@ const sp_string METRIC_TIME_SPENT_BACK_PRESSURE_AGGR = "__server/__time_spent_ba
 const sp_string METRIC_TIME_SPENT_BACK_PRESSURE_COMPID = "__time_spent_back_pressure_by_compid/";
 // Prefix for connection buffer's metrics
 const sp_string CONNECTION_BUFFER_BY_INSTANCEID = "__connection_buffer_by_instanceid/";
-// Prefix for connection buffer's length metrics
+// Prefix for connection buffer's length metrics. This is different
+// from METRIC_DATA_TUPLES_TO_INSTANCES as that counts
+// the tuples when they are sent to the instance -- this metric
+// will be used to count the tuples as they are received
 
 Review comment:
   From what I understand, tuples can be dropped when they are being removed from the queue
because of backpressure. 
   https://github.com/apache/incubator-heron/blob/571a90b7c8452f7922cdc47b91e52de66f5edd70/heron/stmgr/src/cpp/manager/instance-server.cpp#L432
   However, only tuples in the form of `HeronTupleSet` are dropped and the ones in the form
of `TupleStreamMessage` are not dropped. 
   In addition, I'd like to measure arrival rate of tuples at the stream manager and I think
that because tuples are only drained when a certain number of tuples have arrived, that may
not translate accurately?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message