mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Peach (JIRA)" <>
Subject [jira] [Commented] (MESOS-8253) Mesos CI docker rmi conflict
Date Tue, 21 Nov 2017 19:25:00 GMT


James Peach commented on MESOS-8253:

Discussed with [~pono] on hip chat. Seems like the following error message is correlated:

write /dev/stdout: resource temporarily unavailable

The working theory is that the Jenkins worker sets up a pipe for the shell script that is
propagated all the way through the {{docker run}}. However it is not able to drain the pipe
fast enough, so it gets full. Then, the {{docker run}} process fails and detached from the
container leaving the tests running in the background docker daemon. The build script then
exits, and the trap for {{docker rmi}} fails because we didn't force it.

So we probably should have 2 workarounds to try to make this better:
#  {{docker rmi --force}} in the trap to clean up better
# Spool the output to a log file so that jenkins can keep up

> Mesos CI docker rmi conflict
> ----------------------------
>                 Key: MESOS-8253
>                 URL:
>             Project: Mesos
>          Issue Type: Bug
>          Components: build, docker
>            Reporter: James Peach
> We are seeing a lot of docker build jobs failing when they try to clean up there docker
> {noformat}
> + docker rmi mesos-1511286604-15916
> Error response from daemon: conflict: unable to remove repository reference "mesos-1511286604-15916"
(must force) - container 1aabf0225a43 is using its referenced image 23292073f88f
> Build step 'Execute shell' marked build as failure
> {noformat}
> The full Jenkins log is [here|,COMPILER=clang,CONFIGURATION=--verbose%20--disable-libtool-wrappers%20--enable-libevent%20--enable-ssl,ENVIRONMENT=GLOG_v=1%20MESOS_VERBOSE=1,OS=ubuntu%3A14.04,label_exp=(ubuntu)&&(!ubuntu-us1)&&(!ubuntu-eu2)&&(!qnode3)&&(!H23)/4486/console]

This message was sent by Atlassian JIRA

View raw message