couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joan Touzet <woh...@apache.org>
Subject Re: Cloudbees Operations Center and Client Masters upgrade this weekend.
Date Sun, 05 Apr 2020 19:06:46 GMT
On 2020-04-05 2:22 p.m., Paul Davis wrote:
> Howdy Gavin,
> 
> I've updated everything besides the two FreeBSD nodes that Joan
> manages. Is there a trick to having Jenkins realize a node has
> upgraded its slave.jar? Took me an extra twenty minutes fumbling
> around before realizing that slave.jar is now actually agent.jar and
> that even though Jenkins realized the jar had actually been updated. I
> ended up realizing that I had to go through each node in the web UI to
> re-enable each worker.
> 
> Joan, for the FreeBSD ones I'm not sure how you set up the run
> scripts. For me it was just a `sudo sv restart jenkins` on each node
> because I download the agent.jar as part of `/etc/sv/jenkins/run`.
> Then it was just a random Googling till I happened to notice that was
> enough to upgrade the jar and needed to click the button.

Yup, that was it here too, though I had to sv stop && killall java 
because it didn't seem to be stopping right.

Sorry for the delay, I've been ill this weekend and only got to this now.

-Joan

> 
> Paul
> 
> On Sun, Apr 5, 2020 at 11:32 AM Gavin McDonald <gmcdonald@apache.org> wrote:
>>
>> Hi All,
>>
>> This is now completed.
>>
>> Overall time spent was around 3 hours on the Operations Center and 5
>> masters.
>>
>> When the OC was being upgraded, there was no downtime for the masters.
>>
>> Each master had downtime of around 20 minutes to 30 minutes, including
>> multiple restarts
>> , backups and plugin upgrades.
>>
>> CouchDB folks - most of your nodes are connected via 'node connects to the
>> master' meaning that you'll
>> have to update each slave.jar manually. The preferred method is ssh i/o
>> non-blocking, all nodes connected
>> in this fashion all came back on their own without intervention.
>>
>> Thanks all
>>
>> Gavin McDonald (ASF Infra)
>>
>>
>> On Sat, Apr 4, 2020 at 3:20 PM Gavin McDonald <gmcdonald@apache.org> wrote:
>>
>>> Hi All,
>>>
>>> I'll be upgrading the Cloudbees Core Operations Center and all attached
>>> Masters on Sunday 5th April at around 10am UTC until completed.
>>>
>>> I'll turn off new builds around 7am UTC.
>>>
>>> I'm 'expecting' a couple of hours downtime assuming all goes well,
>>> including upgrading all
>>> associated plugins, performing backups etc. But, this is the first time
>>> I'm upgrading this setup so there may be the odd gotcha that surprises me
>>> who knows! - I'm as prepared as possible and I am not expecting any
>>> problems.
>>>
>>> Thanks
>>>
>>> Gavin McDonald (ASF Infra)
>>>
>>>

Mime
View raw message