aurora-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zameer Manji (JIRA)" <>
Subject [jira] [Commented] (AURORA-1890) Job Update Pulse History is not durably stored
Date Mon, 13 Feb 2017 22:00:43 GMT


Zameer Manji commented on AURORA-1890:

The scheduler does the right thing on first pulse. However on failover, any coordinated updates
are immediately sent to BLOCKED_AWAITING_PULSE. This is because on scheduler startup pulse
state is reset to no pulse received. The code sets the timestamp to the last pulse received
to 0L:

    synchronized void initializePulseState(IJobUpdate update, JobUpdateStatus status) {
      pulseStates.put(update.getSummary().getKey(), new PulseState(

Would it be ok to set the timestamp to the first event after the most recent {{BLOCKED_AWAITING_PULSE}}?
We know for sure at that point in time that a pulse was received because of the state transition
from {{BLCOKED_AWAITING_PULSE}} to some other event.

Also could you describe "significant" write volume? I can imagine if the pulse interval was
in the seconds and there are thousands of updates perhaps it would be too much. However we
could prevent excessively small pulse intervals.

> Job Update Pulse History is not durably stored
> ----------------------------------------------
>                 Key: AURORA-1890
>                 URL:
>             Project: Aurora
>          Issue Type: Bug
>            Reporter: Zameer Manji
> I have experienced the following problem with pulse updates. To reproduce:
> 1. Create an update with a pulse timeout of 1h
> 2. Send a pulse to get the update going.
> 3. Failover the scheduler immediately after.
> 4. Observe that the update is awaiting another pulse right after the failover.
> This is because the {{JobUpdateControllerImpl}} stores pulse history and state in memory
in {{PulseHandler}}. On scheduler startup, the pulse state is reset to no pulse received.
> We can solve this by durably storing the timestamp of the last pulse received in storage.

This message was sent by Atlassian JIRA

View raw message