aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Sweeney <kswee...@twitter.com.INVALID>
Subject Re: [jira] [Created] (AURORA-1258) Improve procedure for adding instances to a job
Date Tue, 07 Apr 2015 19:45:21 GMT
How about a scale command somewhere that will check that the only operation
it's doing is "scaling" and not introducing new code. This could be
accomplished by diffing the config for the new instances with that of the
old instances. For cases where it's not a pure scale operation, maybe have
an override flag like --allow-heterogenous. Also, the existing updater
should handle this case fine if the config hasn't changed - it won't
attempt to modify unchanged instances. Maybe we need to update diff to give
an easy way to ensure that a proposed change is a pure scale operation and
not a combination of scaling and a code deploy.

On Tuesday, April 7, 2015, Joe Smith (JIRA) <jira@apache.org> wrote:

> Joe Smith created AURORA-1258:
> ---------------------------------
>
>              Summary: Improve procedure for adding instances to a job
>                  Key: AURORA-1258
>                  URL: https://issues.apache.org/jira/browse/AURORA-1258
>              Project: Aurora
>           Issue Type: Story
>           Components: Reliability, Usability
>             Reporter: Joe Smith
>
>
> The current process for adding instances to a job is highly manual, and
> potentially dangerous.
>
> 1. Take a config for a job with 10 instances, update it to 20 instances.
> 2. The batch size will be increased, and users will need to specify shards
> 10 to 19.
> 3. After this update is complete, users will need to manually update
> shards 0-9 again.
>
> There may be other changes pulled in as part of this update other than
> just increasing the number of instances, which could further complicate
> things.
>
> One possible improvement would be to change the updater from
> 'under-provision' where it kills instances first, then schedules new
> instances, to an 'over-provision' where it adds on new instances, then
> backpedals and kills the old instances.
>
> Overall, a single command or process for a user to take an
> already-existing job and increase the number of instances would reduce
> overhead and fat-fingering.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>


-- 
Sent from Gmail Mobile

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message