infra-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "yifan zou (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (INFRA-17672) Repuppetize Beam Jenkins nodes 1 and 14
Date Wed, 16 Jan 2019 18:57:00 GMT

    [ https://issues.apache.org/jira/browse/INFRA-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744357#comment-16744357
] 

yifan zou commented on INFRA-17672:
-----------------------------------

After rebooting, the beam14 was offline, but the beam1 was still accepting jobs and failed
them quickly. 

ERROR: Issue with creating launcher for agent beam1. The agent is being disconnected
10:45:01 [EnvInject] - Loading node environment variables.
10:45:01 ERROR: SEVERE ERROR occurs
10:45:01 org.jenkinsci.lib.envinject.EnvInjectException: hudson.remoting.ChannelClosedException:
Channel "unknown": Remote call on beam1 failed. The channel is closing down or has closed
down
10:45:01 	at org.jenkinsci.plugins.envinject.service.EnvironmentVariablesNodeLoader.gatherEnvVarsForNode(EnvironmentVariablesNodeLoader.java:91)
10:45:01 	at org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:80)
10:45:01 	at org.jenkinsci.plugins.envinject.EnvInjectListener.setUpEnvironment(EnvInjectListener.java:44)
10:45:01 	at hudson.model.AbstractBuild$AbstractBuildExecution.createLauncher(AbstractBuild.java:542)
10:45:01 	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:462)
10:45:01 	at hudson.model.Run.execute(Run.java:1810)
10:45:01 	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
10:45:01 	at hudson.model.ResourceController.execute(ResourceController.java:97)
10:45:01 	at hudson.model.Executor.run(Executor.java:429)
10:45:01 Caused by: hudson.remoting.ChannelClosedException: Channel "unknown": Remote call
on beam1 failed. The channel is closing down or has closed down
10:45:01 	at hudson.remoting.Channel.call(Channel.java:948)
10:45:01 	at hudson.FilePath.act(FilePath.java:1162)
10:45:01 	at org.jenkinsci.plugins.envinject.service.EnvironmentVariablesNodeLoader.gatherEnvVarsForNode(EnvironmentVariablesNodeLoader.java:64)
10:45:01 	... 8 more
10:45:01 Caused by: java.io.IOException: Unexpected termination of the channel
10:45:01 	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:77)
10:45:01 Caused by: java.io.EOFException
10:45:01 	at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2681)
10:45:01 	at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3156)
10:45:01 	at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:862)
10:45:01 	at java.io.ObjectInputStream.<init>(ObjectInputStream.java:358)
10:45:01 	at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
10:45:01 	at hudson.remoting.Command.readFrom(Command.java:140)
10:45:01 	at hudson.remoting.Command.readFrom(Command.java:126)
10:45:01 	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:36)
10:45:01 	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
10:45:03 [EnvInject] - [ERROR] - SEVERE ERROR occurs: Channel "unknown": Remote call on beam1
failed. The channel is closing down or has closed down
10:45:03 Finished: FAILURE

> Repuppetize Beam Jenkins nodes 1 and 14
> ---------------------------------------
>
>                 Key: INFRA-17672
>                 URL: https://issues.apache.org/jira/browse/INFRA-17672
>             Project: Infrastructure
>          Issue Type: Bug
>          Components: Jenkins
>            Reporter: yifan zou
>            Priority: Major
>
> The beam1 (35.225.86.178) and beam14 (104.154.226.17) were in bad status and reset this
morning. The 'jenkins' user is missing from those VMs. Might need to recreate the jenkins
user and repuppetize them.
> I've checked the host ssh keys (/etc/ssh/ssh_host_rsa_key.pub) of those two nodes in
case of the key changed after reset.
> beam1:
> ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/A18ntDJ7fiFeh/XKh1IEC2mrY7x6kg7CiSU3kjU6hh/v/X2Yn8nH3s7coqSkr0Yp258LeE45UiAYJGtPP5dlHU6Am4DbUMR6tvfozXRe7GI3dgkHagGSP0UrwZW1Ot8yfrkyn9mOWVqWinBTWDCNjUh+o1Rdff9aJkf4tMyWV1CvkL5RmSK/Up5mukdxPqP0GrvnyGv7K7kBdgKTKFnf/UpNCNkCpAnlWeKv9awOJsLv8z9dzfwncMQWoQDIqDI3RDlzFLET6keCsu+EpQdz9AXF9e7sv3WgqgtiNhkRoodtWIArcELwRBnBRlcT2ByBAdogmeT+wlUqjqjtxU5P
root@apache-beam-jenkins-slave-group-ndfp
> beam14:
> ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYDZHclvED06MfZQTXegxx7TiJ6QWqMPIH6MHIckMSTMRfTwMCvAPC8Ev0C06olnSsrQD9qhMj8UMx5tMc9DL8fME5QmicJkcoNzHYJwtFxu9x0FKgYaTP3iQsL6AknB9syqW7T1dkc3Yjm1Adq9Dbv2ilze96tVdEu0zXRv7PkiAY+WETH/nSJAKvaOrmObGHGjtp2ugs42MrkDYiv9RK4iYSesulx6UkG2koAiyuYBUJp3wQFO/AI4BUZ/bUUbRLzDiSzoR+JrnX7hZgaVmdGExhm+xmBoqONCzRgcwxVb4AHriYQwarWcV1UFxIaLtIATcWy0Hxqxp4dl3VVnT
root@apache-beam-jenkins-slave-group-k24j
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message