aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Santhosh Kumar Shanmugham <sshanmug...@twitter.com.INVALID>
Subject Re: [jira] [Comment Edited] (AURORA-1791) Commit ca683 is not backwards compatible.
Date Wed, 12 Oct 2016 08:00:40 GMT
FYI - some of the end to end tests have started to become very flaky,
probably due to this commit. I am not sure if others have seen this
behavior.

On Oct 12, 2016 12:51 AM, "David McLaughlin (JIRA)" <jira@apache.org> wrote:

>
>     [ https://issues.apache.org/jira/browse/AURORA-1791?page=
> com.atlassian.jira.plugin.system.issuetabpanels:comment-
> tabpanel&focusedCommentId=15567954#comment-15567954 ]
>
> David McLaughlin edited comment on AURORA-1791 at 10/12/16 7:50 AM:
> --------------------------------------------------------------------
>
> Given the lack of test coverage I've found just looking at a single
> function, I would seriously recommend we roll back the commit (or will it
> be commits?) rather than rush a patch in order to fix master. Any
> objections? cc/ [~zmanji] and [~joshua.cohen]]
>
>
> was (Author: davmclau):
> Given the lack of test coverage I've found just looking at a single
> function, I would seriously recommend we roll back the commit (or will it
> be commits?) rather than rush a patch in order to fix master. Any
> objections? cc/ [~zmanji] and [~jcohen]
>
> > Commit ca683 is not backwards compatible.
> > -----------------------------------------
> >
> >                 Key: AURORA-1791
> >                 URL: https://issues.apache.org/jira/browse/AURORA-1791
> >             Project: Aurora
> >          Issue Type: Bug
> >            Reporter: Zameer Manji
> >            Assignee: Kai Huang
> >            Priority: Blocker
> >
> > The commit [ca683cb9e27bae76424a687bc6c3af5a73c501b9 |
> https://github.com/apache/aurora/commit/ca683cb9e27bae76424a687bc6c3af
> 5a73c501b9] is not backwards compatible. The last section of the commit
> > {quote}
> > 4. Modified the Health Checker and redefined the meaning
> initial_interval_secs.
> > {quote}
> > has serious, unintended consequences.
> > Consider the following health check config:
> > {noformat}
> >       initial_interval_secs: 10
> >       interval_secs: 5
> >       max_consecutive_failures: 1
> > {noformat}
> > On the 0.16.0 executor, no health checking will occur for the first 10
> seconds. Here the earliest a task can cause failure is at the 10th second.
> > On master, health checking starts right away which means the task can
> fail at the first second since {{max_consecutive_failures}} is set to 1.
> > This is not backwards compatible and needs to be fixed.
> > I think a good solution would be to revert the meaning change to
> initial_interval_secs and have the task transition into RUNNING when
> {{max_consecutive_successes}} is met.
> > An investigation shows {{initial_interval_secs}} was set to 5 but the
> task failed health checks right away:
> > {noformat}
> > D1011 19:52:13.295877 6 health_checker.py:107] Health checks enabled.
> Performing health check.
> > D1011 19:52:13.306816 6 health_checker.py:126] Reset consecutive
> failures counter.
> > D1011 19:52:13.307032 6 health_checker.py:132] Initial interval expired.
> > W1011 19:52:13.307130 6 health_checker.py:135] Failed to reach minimum
> consecutive successes.
> > {noformat}
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message