mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gilbert Song (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MESOS-6004) Tasks fail when provisioning multiple containers with large docker images using copy backend
Date Mon, 08 Aug 2016 17:52:20 GMT

    [ https://issues.apache.org/jira/browse/MESOS-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15412159#comment-15412159
] 

Gilbert Song commented on MESOS-6004:
-------------------------------------

Thanks [~mito]. We need to fix this issue. Most likely this is because the image size is too
large, and it takes time to download/copy. Could you please:

1. Just for curious, could you test using localpuller&&overlay backend (`--docker_registry=/path/to/your/image/tarballs/folder`
and `--image_provider_backend=overlay`). Want to know whether you still have the scheduling
issue.

2. Attach the GLOG_v=1 log, should be fine in size if you are using `noformat`.

>  Tasks fail when provisioning multiple containers with large docker images using copy
backend
> ---------------------------------------------------------------------------------------------
>
>                 Key: MESOS-6004
>                 URL: https://issues.apache.org/jira/browse/MESOS-6004
>             Project: Mesos
>          Issue Type: Bug
>          Components: containerization, docker
>    Affects Versions: 1.0.0
>         Environment: h4. Agent Platform
> - Ubuntu 16.04
> - AWS g2.x2large instance
> - Nvidia support enabled
> h4. Agent Configuration
> -{noformat}
> --containerizers=mesos,docker
> --docker_config=<docker auth json>
> --docker_store_dir=/mnt/mesos/store/docker
> --executor_registration_timeout=3mins
> --hostname=<aws public dns>
> --image_providers=docker
> --image_provisioner_backend=copy
> --isolation=filesystem/linux,docker/runtime,cgroups/devices,gpu/nvidia
> --switch_user=false
> --work_dir=/mnt/mesos
> {noformat}
> h4. Framework
> - custom framework written in python
> - using unified containerizer with docker images
> h4. Test Setup
> * 1 master
> * 1 agent
> * 5 tasks scheduled at the same time:
> ** resources: cpus: 0.1, mem: 128
> ** command: `echo test`
> ** docker image: custom docker image, based on nvidia/cuda ~5gb
> ** the same docker image was for all tasks, already pulled.
>            Reporter: Michael Thomas
>              Labels: containerizer, docker, performance
>
> When scheduling more than one task on the same agent, all tasks fail a as containers
seem to be destroyed during provisioning.
> Specifically, the errors on the agent logs are:
> {noformat}
>  E0808 15:53:09.691315 30996 slave.cpp:3976] Container 'eb20f642-bb90-4293-8eec-6f1576ccaeb1'
for executor '3' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 failed to start: Container
is being destroyed during provisioning
> {noformat}
> and 
> {noformat}
> I0808 15:52:32.510210 30999 slave.cpp:4539] Terminating executor ''2' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001'
because it did not register within 3mins
> {noformat}
> As the default provisioning method {{copy}} is being used, I assume this is due to the
provisioning of multiple containers taking too long and the agent will not wait. For large
images, this method is simply not performant.
> The issue did not occur, when only one tasks was scheduled.
> Increasing the {{executor_registration_timeout}} parameter, seemed to help a bit as it
allowed to schedule at least 2 tasks at the same time. But still fails with more (5 in this
case)
> h4. Complete logs
> (with GLOG_v=0, as with 1 it was too long)
> {noformat}
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.661067 30961 main.cpp:434]
Starting Mesos agent
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.661551 30961 slave.cpp:198]
Agent started on 1)@172.31.23.17:5051
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.661578 30961 slave.cpp:199]
Flags at startup: --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc"
--authenticate_http_readonly="false" --authenticate_http_readwrite="false" --authenticatee="crammd5"
--authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false"
--cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false"
--cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos,docker"
--default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_config="{"auths":{"https:\/\/index.docker.io\/v1\/":{"auth":"dGVycmFsb3VwZTpUYWxFWUFOSXR5","email":"sebastian.gerke@terraloupe.com"}}}"
--docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs"
--docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/mnt/mesos/store/docker"
--do
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: cker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume"
--enforce_container_disk_quota="false" --executor_registration_timeout="3mins" --executor_shutdown_grace_period="5secs"
--fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks"
--gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname="ec2-52-59-113-0.eu-central-1.compute.amazonaws.com"
--hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --image_providers="docker"
--image_provisioner_backend="copy" --initialize_driver_logging="true" --isolation="filesystem/linux,docker/runtime,cgroups/devices,gpu/nvidia"
--launcher_dir="/usr/libexec/mesos" --log_dir="/var/log/mesos" --logbufsecs="0" --logging_level="INFO"
--master="zk://172.31.19.240:2181/mesos" --oversubscribed_resources_interval="15secs" --perf_duration="10secs"
--perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns" --quiet="false"
--recov
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: er="reconnect" --recovery_timeout="15mins"
--registration_backoff_factor="1secs" --revocable_cpu_low_priority="true" --sandbox_directory="/mnt/mesos/sandbox"
--strict="true" --switch_user="false" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system"
--version="false" --work_dir="/mnt/mesos"
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.662147 30961 slave.cpp:519]
Agent resources: gpus(*):1; cpus(*):8; mem(*):14014; disk(*):60257; ports(*):[31000-32000]
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.662211 30961 slave.cpp:527]
Agent attributes: [  ]
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.662230 30961 slave.cpp:532]
Agent hostname: ec2-52-59-113-0.eu-central-1.compute.amazonaws.com
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.663354 31000 state.cpp:57]
Recovering state from '/mnt/mesos/meta'
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.663918 30995 status_update_manager.cpp:200]
Recovering status update manager
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.664131 30996 containerizer.cpp:522]
Recovering containerizer
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.664136 31000 docker.cpp:775]
Recovering Docker containers
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: 2016-08-08 15:48:32,665:30961(0x7fce36077700):ZOO_INFO@check_events@1728:
initiated connection to server [172.31.19.240:2181]
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: 2016-08-08 15:48:32,667:30961(0x7fce36077700):ZOO_INFO@check_events@1775:
session establishment complete on server [172.31.19.240:2181], sessionId=0x1566a66ab9b000a,
negotiated timeout=10000
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.667783 31000 group.cpp:349]
Group process (group(1)@172.31.23.17:5051) connected to ZooKeeper
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.667835 31000 group.cpp:837]
Syncing group operations: queue size (joins, cancels, datas) = (0, 0, 0)
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.667857 31000 group.cpp:427]
Trying to create path '/mesos' in ZooKeeper
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.668104 30994 metadata_manager.cpp:251]
Successfully loaded 1 Docker images
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.668200 30993 provisioner.cpp:253]
Provisioner recovery complete
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.671417 31000 detector.cpp:152]
Detected a new leader: (id='64')
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.671563 30998 group.cpp:706]
Trying to get '/mesos/json.info_0000000064' in ZooKeeper
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.672611 30996 zookeeper.cpp:259]
A new leading master (UPID=master@172.31.19.240:5050) is detected
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.761471 30998 slave.cpp:4782]
Finished recovery
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.761726 30998 slave.cpp:4815]
Garbage collecting old agent 524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S2
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.761812 30998 slave.cpp:4815]
Garbage collecting old agent c9852a23-bc07-422d-8d69-23c167a1924d-S0
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.761808 31000 gc.cpp:55]
Scheduling '/mnt/mesos/slaves/524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S2' for gc 6.99999118329778days
in the future
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.761893 30998 slave.cpp:4815]
Garbage collecting old agent 524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S1
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.761904 31000 gc.cpp:55]
Scheduling '/mnt/mesos/meta/slaves/524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S2' for gc 6.99999118289481days
in the future
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.761978 31000 gc.cpp:55]
Scheduling '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S0' for gc 6.99999118225778days
in the future
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.761988 30998 slave.cpp:4815]
Garbage collecting old agent 524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S3
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.762029 31000 gc.cpp:55]
Scheduling '/mnt/mesos/meta/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S0' for gc 6.9999911819437days
in the future
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.762064 31000 gc.cpp:55]
Scheduling '/mnt/mesos/slaves/524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S1' for gc 6.99999118122667days
in the future
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.762105 31000 gc.cpp:55]
Scheduling '/mnt/mesos/meta/slaves/524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S1' for gc 6.9999911809037days
in the future
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.762168 31000 gc.cpp:55]
Scheduling '/mnt/mesos/slaves/524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S3' for gc 6.9999911798963days
in the future
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.762207 31000 gc.cpp:55]
Scheduling '/mnt/mesos/meta/slaves/524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S3' for gc 6.99999117958222days
in the future
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.762230 30998 slave.cpp:895]
New master detected at master@172.31.19.240:5050
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.762212 30993 status_update_manager.cpp:174]
Pausing sending status updates
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.762277 30998 slave.cpp:916]
No credentials provided. Attempting to register without authentication
> Aug  8 15:48:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:32.762344 30998 slave.cpp:927]
Detecting new master
> Aug  8 15:48:33 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:33.067215 30999 slave.cpp:1197]
Re-registered with master master@172.31.19.240:5050
> Aug  8 15:48:33 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:33.067312 30999 slave.cpp:1233]
Forwarding total oversubscribed resources
> Aug  8 15:48:33 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:33.067313 30993 status_update_manager.cpp:181]
Resuming sending status updates
> Aug  8 15:48:33 ip-172-31-23-17 mesos-slave[30961]: I0808 15:48:33.067610 31000 slave.cpp:2526]
Updated checkpointed resources from  to
> Aug  8 15:49:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:32.507480 31000 slave.cpp:1495]
Got assigned task 2 for framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:49:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:32.508069 31000 gc.cpp:83]
Unscheduling '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001'
from gc
> Aug  8 15:49:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:32.508167 30998 slave.cpp:1614]
Launching task 2 for framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:49:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:32.509050 30998 slave.cpp:5674]
Launching executor 2 of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 with resources
cpus(*):0.1; mem(*):32 in work directory '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001/executors/2/runs/a58a1b2c-14a4-4ada-a4b3-666d8d077597'
> Aug  8 15:49:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:32.509269 30998 slave.cpp:1840]
Queuing task '2' for executor '2' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:49:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:32.509430 30993 containerizer.cpp:781]
Starting container 'a58a1b2c-14a4-4ada-a4b3-666d8d077597' for executor '2' of framework 'c9852a23-bc07-422d-8d69-23c167a1924d-0001'
> Aug  8 15:49:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:32.511643 30995 provisioner.cpp:294]
Provisioning image rootfs '/mnt/mesos/provisioner/containers/a58a1b2c-14a4-4ada-a4b3-666d8d077597/backends/copy/rootfses/a688d80b-3eb9-4301-abfc-3e6a742cc8be'
for container a58a1b2c-14a4-4ada-a4b3-666d8d077597
> Aug  8 15:49:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:32.663204 30996 slave.cpp:4591]
Current disk usage 17.44%. Max allowed age: 5.078955723221944days
> Aug  8 15:49:38 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:38.511801 30995 slave.cpp:1495]
Got assigned task 3 for framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:49:38 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:38.512176 30995 slave.cpp:1614]
Launching task 3 for framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:49:38 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:38.512511 30995 slave.cpp:5674]
Launching executor 3 of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 with resources
cpus(*):0.1; mem(*):32 in work directory '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001/executors/3/runs/eb20f642-bb90-4293-8eec-6f1576ccaeb1'
> Aug  8 15:49:38 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:38.512737 30995 slave.cpp:1840]
Queuing task '3' for executor '3' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:49:38 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:38.512948 30998 containerizer.cpp:781]
Starting container 'eb20f642-bb90-4293-8eec-6f1576ccaeb1' for executor '3' of framework 'c9852a23-bc07-422d-8d69-23c167a1924d-0001'
> Aug  8 15:49:38 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:38.514993 30996 provisioner.cpp:294]
Provisioning image rootfs '/mnt/mesos/provisioner/containers/eb20f642-bb90-4293-8eec-6f1576ccaeb1/backends/copy/rootfses/8d2e66c8-f0d6-4891-86e3-7e5c222adfd2'
for container eb20f642-bb90-4293-8eec-6f1576ccaeb1
> Aug  8 15:49:44 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:44.524807 31000 slave.cpp:1495]
Got assigned task 4 for framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:49:44 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:44.525184 31000 slave.cpp:1614]
Launching task 4 for framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:49:44 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:44.525995 31000 slave.cpp:5674]
Launching executor 4 of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 with resources
cpus(*):0.1; mem(*):32 in work directory '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001/executors/4/runs/d248d565-e9d1-438e-8524-d71f601ba981'
> Aug  8 15:49:44 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:44.526221 31000 slave.cpp:1840]
Queuing task '4' for executor '4' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:49:44 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:44.526448 30995 containerizer.cpp:781]
Starting container 'd248d565-e9d1-438e-8524-d71f601ba981' for executor '4' of framework 'c9852a23-bc07-422d-8d69-23c167a1924d-0001'
> Aug  8 15:49:44 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:44.528301 30998 provisioner.cpp:294]
Provisioning image rootfs '/mnt/mesos/provisioner/containers/d248d565-e9d1-438e-8524-d71f601ba981/backends/copy/rootfses/d05cefcb-6f83-49b6-a1c1-8e53d463712c'
for container d248d565-e9d1-438e-8524-d71f601ba981
> Aug  8 15:49:50 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:50.531419 30996 slave.cpp:1495]
Got assigned task 5 for framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:49:50 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:50.531810 30996 slave.cpp:1614]
Launching task 5 for framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:49:50 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:50.532166 30996 slave.cpp:5674]
Launching executor 5 of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 with resources
cpus(*):0.1; mem(*):32 in work directory '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001/executors/5/runs/2f372338-3c40-4463-b61f-4f7e9e42766a'
> Aug  8 15:49:50 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:50.532412 30996 slave.cpp:1840]
Queuing task '5' for executor '5' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:49:50 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:50.532740 30998 containerizer.cpp:781]
Starting container '2f372338-3c40-4463-b61f-4f7e9e42766a' for executor '5' of framework 'c9852a23-bc07-422d-8d69-23c167a1924d-0001'
> Aug  8 15:49:50 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:50.534970 30993 provisioner.cpp:294]
Provisioning image rootfs '/mnt/mesos/provisioner/containers/2f372338-3c40-4463-b61f-4f7e9e42766a/backends/copy/rootfses/691d6235-1596-4a5d-aeef-21bf43b2cc03'
for container 2f372338-3c40-4463-b61f-4f7e9e42766a
> Aug  8 15:49:56 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:56.540551 30999 slave.cpp:1495]
Got assigned task 6 for framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:49:56 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:56.540967 30999 slave.cpp:1614]
Launching task 6 for framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:49:56 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:56.541805 30999 slave.cpp:5674]
Launching executor 6 of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 with resources
cpus(*):0.1; mem(*):32 in work directory '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001/executors/6/runs/b77f9aa9-b2e5-488d-9fdd-fcbd2e5c8455'
> Aug  8 15:49:56 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:56.542049 30999 slave.cpp:1840]
Queuing task '6' for executor '6' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:49:56 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:56.542263 30996 containerizer.cpp:781]
Starting container 'b77f9aa9-b2e5-488d-9fdd-fcbd2e5c8455' for executor '6' of framework 'c9852a23-bc07-422d-8d69-23c167a1924d-0001'
> Aug  8 15:49:56 ip-172-31-23-17 mesos-slave[30961]: I0808 15:49:56.543998 30996 provisioner.cpp:294]
Provisioning image rootfs '/mnt/mesos/provisioner/containers/b77f9aa9-b2e5-488d-9fdd-fcbd2e5c8455/backends/copy/rootfses/3601630b-5b57-4a45-9e2c-cfa79eacf2fd'
for container b77f9aa9-b2e5-488d-9fdd-fcbd2e5c8455
> Aug  8 15:50:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:50:32.664178 30995 slave.cpp:4591]
Current disk usage 31.17%. Max allowed age: 4.117972973602755days
> Aug  8 15:51:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:51:32.665262 30997 slave.cpp:4591]
Current disk usage 41.95%. Max allowed age: 3.363669281005290days
> Aug  8 15:52:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:52:32.510210 30999 slave.cpp:4539]
Terminating executor ''2' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001' because
it did not register within 3mins
> Aug  8 15:52:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:52:32.510499 31000 containerizer.cpp:1622]
Destroying container 'a58a1b2c-14a4-4ada-a4b3-666d8d077597'
> Aug  8 15:52:32 ip-172-31-23-17 mesos-slave[30961]: I0808 15:52:32.665740 30994 slave.cpp:4591]
Current disk usage 52.57%. Max allowed age: 2.620120712155822days
> Aug  8 15:52:38 ip-172-31-23-17 mesos-slave[30961]: I0808 15:52:38.512807 30995 slave.cpp:4539]
Terminating executor ''3' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001' because
it did not register within 3mins
> Aug  8 15:52:38 ip-172-31-23-17 mesos-slave[30961]: I0808 15:52:38.513015 30994 containerizer.cpp:1622]
Destroying container 'eb20f642-bb90-4293-8eec-6f1576ccaeb1'
> Aug  8 15:52:44 ip-172-31-23-17 mesos-slave[30961]: I0808 15:52:44.526844 30996 slave.cpp:4539]
Terminating executor ''4' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001' because
it did not register within 3mins
> Aug  8 15:52:44 ip-172-31-23-17 mesos-slave[30961]: I0808 15:52:44.527053 30996 containerizer.cpp:1622]
Destroying container 'd248d565-e9d1-438e-8524-d71f601ba981'
> Aug  8 15:52:50 ip-172-31-23-17 mesos-slave[30961]: I0808 15:52:50.532806 31000 slave.cpp:4539]
Terminating executor ''5' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001' because
it did not register within 3mins
> Aug  8 15:52:50 ip-172-31-23-17 mesos-slave[30961]: I0808 15:52:50.532981 31000 containerizer.cpp:1622]
Destroying container '2f372338-3c40-4463-b61f-4f7e9e42766a'
> Aug  8 15:52:56 ip-172-31-23-17 mesos-slave[30961]: I0808 15:52:56.542345 30995 slave.cpp:4539]
Terminating executor ''6' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001' because
it did not register within 3mins
> Aug  8 15:52:56 ip-172-31-23-17 mesos-slave[30961]: I0808 15:52:56.542580 30998 containerizer.cpp:1622]
Destroying container 'b77f9aa9-b2e5-488d-9fdd-fcbd2e5c8455'
> Aug  8 15:53:09 ip-172-31-23-17 mesos-slave[30961]: E0808 15:53:09.691315 30996 slave.cpp:3976]
Container 'eb20f642-bb90-4293-8eec-6f1576ccaeb1' for executor '3' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
failed to start: Container is being destroyed during provisioning
> Aug  8 15:53:09 ip-172-31-23-17 mesos-slave[30961]: W0808 15:53:09.695711 30996 composing.cpp:541]
Container 'eb20f642-bb90-4293-8eec-6f1576ccaeb1' is already destroyed
> Aug  8 15:53:11 ip-172-31-23-17 mesos-slave[30961]: E0808 15:53:11.270226 30993 slave.cpp:3976]
Container 'a58a1b2c-14a4-4ada-a4b3-666d8d077597' for executor '2' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
failed to start: Container is being destroyed during provisioning
> Aug  8 15:53:11 ip-172-31-23-17 mesos-slave[30961]: W0808 15:53:11.270447 30999 composing.cpp:541]
Container 'a58a1b2c-14a4-4ada-a4b3-666d8d077597' is already destroyed
> Aug  8 15:53:11 ip-172-31-23-17 mesos-slave[30961]: E0808 15:53:11.689219 30995 slave.cpp:3976]
Container 'd248d565-e9d1-438e-8524-d71f601ba981' for executor '4' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
failed to start: Container is being destroyed during provisioning
> Aug  8 15:53:11 ip-172-31-23-17 mesos-slave[30961]: W0808 15:53:11.689400 31000 composing.cpp:541]
Container 'd248d565-e9d1-438e-8524-d71f601ba981' is already destroyed
> Aug  8 15:53:12 ip-172-31-23-17 mesos-slave[30961]: E0808 15:53:12.400029 30999 slave.cpp:3976]
Container '2f372338-3c40-4463-b61f-4f7e9e42766a' for executor '5' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
failed to start: Container is being destroyed during provisioning
> Aug  8 15:53:12 ip-172-31-23-17 mesos-slave[30961]: W0808 15:53:12.400218 30994 composing.cpp:541]
Container '2f372338-3c40-4463-b61f-4f7e9e42766a' is already destroyed
> Aug  8 15:53:12 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:12.799772 30998 provisioner.cpp:434]
Destroying container rootfs at '/mnt/mesos/provisioner/containers/eb20f642-bb90-4293-8eec-6f1576ccaeb1/backends/copy/rootfses/8d2e66c8-f0d6-4891-86e3-7e5c222adfd2'
for container eb20f642-bb90-4293-8eec-6f1576ccaeb1
> Aug  8 15:53:12 ip-172-31-23-17 mesos-slave[30961]: E0808 15:53:12.799855 30995 slave.cpp:3976]
Container 'b77f9aa9-b2e5-488d-9fdd-fcbd2e5c8455' for executor '6' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
failed to start: Container is being destroyed during provisioning
> Aug  8 15:53:12 ip-172-31-23-17 mesos-slave[30961]: W0808 15:53:12.810742 30999 composing.cpp:541]
Container 'b77f9aa9-b2e5-488d-9fdd-fcbd2e5c8455' is already destroyed
> Aug  8 15:53:12 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:12.811067 30998 provisioner.cpp:434]
Destroying container rootfs at '/mnt/mesos/provisioner/containers/a58a1b2c-14a4-4ada-a4b3-666d8d077597/backends/copy/rootfses/a688d80b-3eb9-4301-abfc-3e6a742cc8be'
for container a58a1b2c-14a4-4ada-a4b3-666d8d077597
> Aug  8 15:53:12 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:12.812954 30998 provisioner.cpp:434]
Destroying container rootfs at '/mnt/mesos/provisioner/containers/d248d565-e9d1-438e-8524-d71f601ba981/backends/copy/rootfses/d05cefcb-6f83-49b6-a1c1-8e53d463712c'
for container d248d565-e9d1-438e-8524-d71f601ba981
> Aug  8 15:53:12 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:12.814383 30998 provisioner.cpp:434]
Destroying container rootfs at '/mnt/mesos/provisioner/containers/2f372338-3c40-4463-b61f-4f7e9e42766a/backends/copy/rootfses/691d6235-1596-4a5d-aeef-21bf43b2cc03'
for container 2f372338-3c40-4463-b61f-4f7e9e42766a
> Aug  8 15:53:12 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:12.815577 30998 provisioner.cpp:434]
Destroying container rootfs at '/mnt/mesos/provisioner/containers/b77f9aa9-b2e5-488d-9fdd-fcbd2e5c8455/backends/copy/rootfses/3601630b-5b57-4a45-9e2c-cfa79eacf2fd'
for container b77f9aa9-b2e5-488d-9fdd-fcbd2e5c8455
> Aug  8 15:53:15 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:15.995162 31000 slave.cpp:4082]
Executor '4' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 has terminated with unknown
status
> Aug  8 15:53:15 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:15.995355 31000 slave.cpp:3211]
Handling status update TASK_FAILED (UUID: 4f428b5a-1538-4f78-8383-6c9b68f67f3f) for task 4
of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 from @0.0.0.0:0
> Aug  8 15:53:15 ip-172-31-23-17 mesos-slave[30961]: W0808 15:53:15.995841 31000 containerizer.cpp:1451]
Ignoring update for unknown container: d248d565-e9d1-438e-8524-d71f601ba981
> Aug  8 15:53:15 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:15.996011 31000 status_update_manager.cpp:320]
Received status update TASK_FAILED (UUID: 4f428b5a-1538-4f78-8383-6c9b68f67f3f) for task 4
of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:53:15 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:15.996346 31000 slave.cpp:3604]
Forwarding the update TASK_FAILED (UUID: 4f428b5a-1538-4f78-8383-6c9b68f67f3f) for task 4
of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 to master@172.31.19.240:5050
> Aug  8 15:53:15 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:15.998983 30993 status_update_manager.cpp:392]
Received status update acknowledgement (UUID: 4f428b5a-1538-4f78-8383-6c9b68f67f3f) for task
4 of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:53:15 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:15.999192 30993 slave.cpp:4193]
Cleaning up executor '4' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.000690 31000 gc.cpp:55]
Scheduling '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001/executors/4/runs/d248d565-e9d1-438e-8524-d71f601ba981'
for gc 6.9999884190637days in the future
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.000754 31000 gc.cpp:55]
Scheduling '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001/executors/4'
for gc 6.99998841775407days in the future
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.344812 30995 slave.cpp:4082]
Executor '3' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 has terminated with unknown
status
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.344981 30995 slave.cpp:3211]
Handling status update TASK_FAILED (UUID: c33d9357-e749-4e05-82d2-cb305d1ff0d2) for task 3
of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 from @0.0.0.0:0
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: W0808 15:53:16.346043 30994 containerizer.cpp:1451]
Ignoring update for unknown container: eb20f642-bb90-4293-8eec-6f1576ccaeb1
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.346206 30998 status_update_manager.cpp:320]
Received status update TASK_FAILED (UUID: c33d9357-e749-4e05-82d2-cb305d1ff0d2) for task 3
of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.346482 30997 slave.cpp:3604]
Forwarding the update TASK_FAILED (UUID: c33d9357-e749-4e05-82d2-cb305d1ff0d2) for task 3
of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 to master@172.31.19.240:5050
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.348764 30998 status_update_manager.cpp:392]
Received status update acknowledgement (UUID: c33d9357-e749-4e05-82d2-cb305d1ff0d2) for task
3 of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.348863 30998 slave.cpp:4193]
Cleaning up executor '3' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.348984 30998 gc.cpp:55]
Scheduling '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001/executors/3/runs/eb20f642-bb90-4293-8eec-6f1576ccaeb1'
for gc 6.99999596136593days in the future
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.349020 30998 gc.cpp:55]
Scheduling '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001/executors/3'
for gc 6.99999596097185days in the future
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.498633 30996 slave.cpp:4082]
Executor '2' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 has terminated with unknown
status
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.498803 30996 slave.cpp:3211]
Handling status update TASK_FAILED (UUID: 3d818e6a-f600-412c-b07d-f21794a0af30) for task 2
of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 from @0.0.0.0:0
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.498921 30996 slave.cpp:4082]
Executor '5' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 has terminated with unknown
status
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.498991 30996 slave.cpp:3211]
Handling status update TASK_FAILED (UUID: 16048628-6112-42ed-b3db-5b418a29d40d) for task 5
of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 from @0.0.0.0:0
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: W0808 15:53:16.499202 30996 containerizer.cpp:1451]
Ignoring update for unknown container: a58a1b2c-14a4-4ada-a4b3-666d8d077597
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: W0808 15:53:16.499289 30996 containerizer.cpp:1451]
Ignoring update for unknown container: 2f372338-3c40-4463-b61f-4f7e9e42766a
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.499400 30997 status_update_manager.cpp:320]
Received status update TASK_FAILED (UUID: 3d818e6a-f600-412c-b07d-f21794a0af30) for task 2
of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.499665 30995 slave.cpp:3604]
Forwarding the update TASK_FAILED (UUID: 3d818e6a-f600-412c-b07d-f21794a0af30) for task 2
of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 to master@172.31.19.240:5050
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.499678 30997 status_update_manager.cpp:320]
Received status update TASK_FAILED (UUID: 16048628-6112-42ed-b3db-5b418a29d40d) for task 5
of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.499928 30997 slave.cpp:3604]
Forwarding the update TASK_FAILED (UUID: 16048628-6112-42ed-b3db-5b418a29d40d) for task 5
of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 to master@172.31.19.240:5050
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.504577 30994 status_update_manager.cpp:392]
Received status update acknowledgement (UUID: 3d818e6a-f600-412c-b07d-f21794a0af30) for task
2 of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.504712 30994 slave.cpp:4193]
Cleaning up executor '2' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.505421 30999 gc.cpp:55]
Scheduling '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001/executors/2/runs/a58a1b2c-14a4-4ada-a4b3-666d8d077597'
for gc 6.99999415104296days in the future
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.505486 30994 status_update_manager.cpp:392]
Received status update acknowledgement (UUID: 16048628-6112-42ed-b3db-5b418a29d40d) for task
5 of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.505493 30999 gc.cpp:55]
Scheduling '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001/executors/2'
for gc 6.99999415021926days in the future
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.505623 30994 slave.cpp:4193]
Cleaning up executor '5' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.505748 30993 gc.cpp:55]
Scheduling '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001/executors/5/runs/2f372338-3c40-4463-b61f-4f7e9e42766a'
for gc 6.99999414686222days in the future
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.505798 30993 gc.cpp:55]
Scheduling '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001/executors/5'
for gc 6.9999941464237days in the future
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.598873 30999 slave.cpp:4082]
Executor '6' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 has terminated with unknown
status
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.599030 30999 slave.cpp:3211]
Handling status update TASK_FAILED (UUID: 62805972-05e7-419f-8ff3-f80a88cf8199) for task 6
of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 from @0.0.0.0:0
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: W0808 15:53:16.599324 30996 containerizer.cpp:1451]
Ignoring update for unknown container: b77f9aa9-b2e5-488d-9fdd-fcbd2e5c8455
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.599454 30996 status_update_manager.cpp:320]
Received status update TASK_FAILED (UUID: 62805972-05e7-419f-8ff3-f80a88cf8199) for task 6
of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.599701 30993 slave.cpp:3604]
Forwarding the update TASK_FAILED (UUID: 62805972-05e7-419f-8ff3-f80a88cf8199) for task 6
of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001 to master@172.31.19.240:5050
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.602051 30996 status_update_manager.cpp:392]
Received status update acknowledgement (UUID: 62805972-05e7-419f-8ff3-f80a88cf8199) for task
6 of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.602171 30996 slave.cpp:4193]
Cleaning up executor '6' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.602313 31000 gc.cpp:55]
Scheduling '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001/executors/6/runs/b77f9aa9-b2e5-488d-9fdd-fcbd2e5c8455'
for gc 6.99999302943407days in the future
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.602334 30996 slave.cpp:4281]
Cleaning up framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.602388 31000 gc.cpp:55]
Scheduling '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001/executors/6'
for gc 6.99999302868148days in the future
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.602442 31000 gc.cpp:55]
Scheduling '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0001'
for gc 6.99999302741926days in the future
> Aug  8 15:53:16 ip-172-31-23-17 mesos-slave[30961]: I0808 15:53:16.602445 30999 status_update_manager.cpp:282]
Closing status update streams for framework c9852a23-bc07-422d-8d69-23c167a1924d-0001
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message