apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Yan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (APEXCORE-379) Latency is stuck when an operator is stuck (before the 1000 window threshold)
Date Sat, 08 Oct 2016 00:47:21 GMT

     [ https://issues.apache.org/jira/browse/APEXCORE-379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

David Yan updated APEXCORE-379:
-------------------------------
    Description: 
When the operator suddenly gets stuck, the latency value will also gets stuck, until it is
more than THROUGHPUT_CALCULATION_MAX_SAMPLES (default 1000) windows behind, at which point
STRAM estimates the latency by window width * windowsBehind.  But for 1000 windows, the latency
for that operator is stuck and hence the application latency is also based on stale value.

The way to fix this is for STRAM to estimate the latency when STRAM has not received the end
window stats for that operator for a certain amount of time.
 

  was:
When the operator suddenly gets stuck, the latency value will also gets stuck, until it is
more than THROUGHPUT_CALCULATION_MAX_SAMPLES (default 1000) windows behind, at which point
STRAM estimates the latency by window width * windowsBehind.  But for 1000 windows, the latency
for that operator is stuck and hence the application latency is also based on stale value.

The way to fix this is for STRAM to estimate the latency when STRAM has not received the end
window stats for that operator for a certain amount of time.

Also, we should add to the REST API whether an operator latency is estimated.  


> Latency is stuck when an operator is stuck (before the 1000 window threshold)
> -----------------------------------------------------------------------------
>
>                 Key: APEXCORE-379
>                 URL: https://issues.apache.org/jira/browse/APEXCORE-379
>             Project: Apache Apex Core
>          Issue Type: Improvement
>            Reporter: David Yan
>            Assignee: David Yan
>
> When the operator suddenly gets stuck, the latency value will also gets stuck, until
it is more than THROUGHPUT_CALCULATION_MAX_SAMPLES (default 1000) windows behind, at which
point STRAM estimates the latency by window width * windowsBehind.  But for 1000 windows,
the latency for that operator is stuck and hence the application latency is also based on
stale value.
> The way to fix this is for STRAM to estimate the latency when STRAM has not received
the end window stats for that operator for a certain amount of time.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message