aurora-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zameer Manji (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AURORA-1890) Job Update Pulse History is initialized to no pulses on scheduler recovery
Date Wed, 15 Feb 2017 02:24:41 GMT

     [ https://issues.apache.org/jira/browse/AURORA-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zameer Manji updated AURORA-1890:
---------------------------------
    Description: 
I have experienced the following problem with pulse updates. To reproduce:
1. Create an update with a pulse timeout of 1h
2. Send a pulse to get the update going.
3. Failover the scheduler immediately after.
4. Observe that the update is awaiting another pulse right after the failover.

This is because the {{JobUpdateControllerImpl}} stores pulse history and state in memory in
{{PulseHandler}}. On scheduler startup, the pulse state is reset to no pulse received.

We can solve this by inferring the timestamp of the last pulse by inspecting the job update
events.

  was:
I have experienced the following problem with pulse updates. To reproduce:
1. Create an update with a pulse timeout of 1h
2. Send a pulse to get the update going.
3. Failover the scheduler immediately after.
4. Observe that the update is awaiting another pulse right after the failover.

This is because the {{JobUpdateControllerImpl}} stores pulse history and state in memory in
{{PulseHandler}}. On scheduler startup, the pulse state is reset to no pulse received.

We can solve this by durably storing the timestamp of the last pulse received in storage.


> Job Update Pulse History is initialized to no pulses on scheduler recovery
> --------------------------------------------------------------------------
>
>                 Key: AURORA-1890
>                 URL: https://issues.apache.org/jira/browse/AURORA-1890
>             Project: Aurora
>          Issue Type: Bug
>            Reporter: Zameer Manji
>
> I have experienced the following problem with pulse updates. To reproduce:
> 1. Create an update with a pulse timeout of 1h
> 2. Send a pulse to get the update going.
> 3. Failover the scheduler immediately after.
> 4. Observe that the update is awaiting another pulse right after the failover.
> This is because the {{JobUpdateControllerImpl}} stores pulse history and state in memory
in {{PulseHandler}}. On scheduler startup, the pulse state is reset to no pulse received.
> We can solve this by inferring the timestamp of the last pulse by inspecting the job
update events.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message