mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "niuminguo (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Deleted] (MESOS-7906) mesos send the wrong container data while health check is set
Date Tue, 22 Aug 2017 09:27:00 GMT

     [ https://issues.apache.org/jira/browse/MESOS-7906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

niuminguo updated MESOS-7906:
-----------------------------
    Comment: was deleted

(was: some logs come from our mesos sheduler:

INFO[2017-08-22 16:14:02] Preparing to launch App nginx15.default.xcm.cctv with 2 tasks
DEBU[2017-08-22 16:14:02] Create task ebd0fc723f0b.0.nginx15.default.xcm.cctv in db
DEBU[2017-08-22 16:14:02] Create task e935a28c58d6.1.nginx15.default.xcm.cctv in db
DEBU[2017-08-22 16:14:02] Finding suitable agent to run tasks
DEBU[2017-08-22 16:14:02] Find 1 agent(s) satisfied the constraints
DEBU[2017-08-22 16:14:02] Weighting resource to find the richest agent
DEBU[2017-08-22 16:14:02] Add pending task ebd0fc723f0b.0.nginx15.default.xcm.cctv
DEBU[2017-08-22 16:14:02] Add pending task e935a28c58d6.1.nginx15.default.xcm.cctv
DEBU[2017-08-22 16:14:02] Removing offer  07239dfa-2982-4046-8f1c-f8f3c14d4649-O40044
INFO[2017-08-22 16:14:02] Launching 2 task(s) on agent 172.16.1.18
DEBU[2017-08-22 16:14:02] Waiting for task ebd0fc723f0b.0.nginx15.default.xcm.cctv to be running
DEBU[2017-08-22 16:14:02] Waiting for task e935a28c58d6.1.nginx15.default.xcm.cctv to be running
DEBU[2017-08-22 16:14:04] Finding task ebd0fc723f0b.0.nginx15.default.xcm.cctv to send status
TASK_RUNNING
INFO[2017-08-22 16:14:04] ======ebd0fc723f0b.0.nginx15.default.xcm.cctv-[
    {
        "Id": "37ac5ec29e438d31d1cac933965d802cd16827ba4f04451ba1933ff66ac5ed53",
        "Created": "2017-08-22T07:57:11.865413694Z",
        "Path": "nginx",
        "Args": [
            "-g",
            "daemon off;"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 22927,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2017-08-22T07:57:12.746973518Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:e4e6d42c70b3f79c5d57c170526592168992eb3303a6594c439302fabd92d9a3",
        "ResolvConfPath": "/var/lib/docker/containers/37ac5ec29e438d31d1cac933965d802cd16827ba4f04451ba1933ff66ac5ed53/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/37ac5ec29e438d31d1cac933965d802cd16827ba4f04451ba1933ff66ac5ed53/hostname",
        "HostsPath": "/var/lib/docker/containers/37ac5ec29e438d31d1cac933965d802cd16827ba4f04451ba1933ff66ac5ed53/hosts",
        "LogPath": "/var/lib/docker/containers/37ac5ec29e438d31d1cac933965d802cd16827ba4f04451ba1933ff66ac5ed53/37ac5ec29e438d31d1cac933965d802cd16827ba4f04451ba1933ff66ac5ed53-json.log",
        "Name": "/mesos-07239dfa-2982-4046-8f1c-f8f3c14d4649-S4.04d86001-8814-4f06-967d-bbbd2fe9e605",
        "RestartCount": 0,
        "Driver": "overlay",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/home:/data:rw",
                "/data/mesos/slaves/07239dfa-2982-4046-8f1c-f8f3c14d4649-S4/frameworks/1b7e4dbd-0e15-4761-aa44-de163bc4cb28-0000/executors/ebd0fc723f0b.0.nginx15.default.xcm.cctv/runs/04d86001-8814-4f06-967d-bbbd2fe9e605:/mnt/mesos/sandbox"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "swan",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "host",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": true,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": [
                "label=disable"
            ],
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 10,
            "Memory": 33554432,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 67108864,
            "MemorySwappiness": -1,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay/42c1f074bd6f5c1c77e7aa2a4128f946895777ada1ca78609efdbb29e6e97973/root",
                "MergedDir": "/var/lib/docker/overlay/27b08a7b3f4731216e9abb933261fdbe588ed7f2572d98f373a45b255c81768a/merged",
                "UpperDir": "/var/lib/docker/overlay/27b08a7b3f4731216e9abb933261fdbe588ed7f2572d98f373a45b255c81768a/upper",
                "WorkDir": "/var/lib/docker/overlay/27b08a7b3f4731216e9abb933261fdbe588ed7f2572d98f373a45b255c81768a/work"
            },
            "Name": "overlay"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/home",
                "Destination": "/data",
                "Mode": "rw",
                "RW": true,
                "Propagation": ""
            },
            {
                "Type": "bind",
                "Source": "/data/mesos/slaves/07239dfa-2982-4046-8f1c-f8f3c14d4649-S4/frameworks/1b7e4dbd-0e15-4761-aa44-de163bc4cb28-0000/executors/ebd0fc723f0b.0.nginx15.default.xcm.cctv/runs/04d86001-8814-4f06-967d-bbbd2fe9e605",
                "Destination": "/mnt/mesos/sandbox",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "37ac5ec29e43",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": true,
            "AttachStderr": true,
            "ExposedPorts": {
                "80/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "MESOS_CONTAINER_NAME=mesos-07239dfa-2982-4046-8f1c-f8f3c14d4649-S4.04d86001-8814-4f06-967d-bbbd2fe9e605",
                "MESOS_SANDBOX=/mnt/mesos/sandbox",
                "SWAN_TASK_ID=ebd0fc723f0b.0.nginx15.default.xcm.cctv",
                "SWAN_TASK_NAME=0.nginx15.default.xcm.cctv",
                "SWAN_PORTS=31000",
                "SWAN_HOST=10.0.0.100",
                "SWAN_PORT0=31000",
                "SWAN_PORT_80=31000",
                "SWAN_APP_ID=nginx15.default.xcm.cctv",
                "SWAN_APP_VERSION=1503389642651995428",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "NGINX_VERSION=1.13.3-1~stretch",
                "NJS_VERSION=1.13.3.0.1.11-1~stretch"
            ],
            "Cmd": [
                "nginx",
                "-g",
                "daemon off;"
            ],
            "ArgsEscaped": true,
            "Image": "nginx",
            "Volumes": null,
            "WorkingDir": "/data",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {
                "SWAN_APP_ID": "nginx15.default.xcm.cctv",
                "SWAN_APP_VERSION": "1503389642651995428",
                "SWAN_HOST": "10.0.0.100",
                "SWAN_PORT0": "31000",
                "SWAN_PORTS": "31000",
                "SWAN_PORT_80": "31000",
                "SWAN_TASK_ID": "ebd0fc723f0b.0.nginx15.default.xcm.cctv",
                "SWAN_TASK_NAME": "0.nginx15.default.xcm.cctv"
            },
            "StopSignal": "SIGTERM"
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "13845fd33ab4d7393de5286bbc4944fdb5b28ec2601adc3872852de384a7e8c8",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/13845fd33ab4",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "swan": {
                    "IPAMConfig": {
                        "IPv4Address": "10.0.0.100"
                    },
                    "Links": null,
                    "Aliases": [
                        "37ac5ec29e43"
                    ],
                    "NetworkID": "a39622f1c18f71325e10d04237c853f3ecab068ed7848406c20b259b95e2b6f4",
                    "EndpointID": "7a905274a1d4254415f26319953dd7399f7197c65d3e8f4d15d209233a5f02b5",
                    "Gateway": "10.0.0.1",
                    "IPAddress": "10.0.0.100",
                    "IPPrefixLen": 24,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:a3:25:8c:4a"
                }
            }
        }
    }
]

DEBU[2017-08-22 16:14:04] Receiving status TASK_RUNNING for task ebd0fc723f0b.0.nginx15.default.xcm.cctv
INFO[2017-08-22 16:14:04] Launch task ebd0fc723f0b.0.nginx15.default.xcm.cctv succeed
DEBU[2017-08-22 16:14:04] Remove pending task ebd0fc723f0b.0.nginx15.default.xcm.cctv
DEBU[2017-08-22 16:14:04] Received status update TASK_RUNNING(false) for task:ebd0fc723f0b.0.nginx15.default.xcm.cctv,
reason:REASON_COMMAND_EXECUTOR_FAILED, message:
DEBU[2017-08-22 16:14:04] task ebd0fc723f0b.0.nginx15.default.xcm.cctv healthy & status:
unhealthy (pending) --> unhealthy (TASK_RUNNING) --> changed=false
DEBU[2017-08-22 16:14:04] Receiving 1 offer(s) from mesos
DEBU[2017-08-22 16:14:04] Received offer 07239dfa-2982-4046-8f1c-f8f3c14d4649-O40045 with
resource cpus:[3.98] mem:[6.62G] disk:[38.97G] ports:[[31000,32000]] from agent 172.16.1.18
DEBU[2017-08-22 16:14:05] Finding task e935a28c58d6.1.nginx15.default.xcm.cctv to send status
TASK_RUNNING
DEBU[2017-08-22 16:14:05] Receiving status TASK_RUNNING for task e935a28c58d6.1.nginx15.default.xcm.cctv
INFO[2017-08-22 16:14:05] Launch task e935a28c58d6.1.nginx15.default.xcm.cctv succeed
DEBU[2017-08-22 16:14:05] Remove pending task e935a28c58d6.1.nginx15.default.xcm.cctv
INFO[2017-08-22 16:14:05] launch app nginx15.default.xcm.cctv succeed
INFO[2017-08-22 16:14:05] ======e935a28c58d6.1.nginx15.default.xcm.cctv-[
    {
        "Id": "9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d",
        "Created": "2017-08-22T07:57:11.82945298Z",
        "Path": "nginx",
        "Args": [
            "-g",
            "daemon off;"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 22939,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2017-08-22T07:57:13.782847516Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:e4e6d42c70b3f79c5d57c170526592168992eb3303a6594c439302fabd92d9a3",
        "ResolvConfPath": "/var/lib/docker/containers/9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d/hostname",
        "HostsPath": "/var/lib/docker/containers/9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d/hosts",
        "LogPath": "/var/lib/docker/containers/9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d/9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d-json.log",
        "Name": "/mesos-07239dfa-2982-4046-8f1c-f8f3c14d4649-S4.f2f14c19-8d64-448f-abfe-6f3bbc6b2a69",
        "RestartCount": 0,
        "Driver": "overlay",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/home:/data:rw",
                "/data/mesos/slaves/07239dfa-2982-4046-8f1c-f8f3c14d4649-S4/frameworks/1b7e4dbd-0e15-4761-aa44-de163bc4cb28-0000/executors/e935a28c58d6.1.nginx15.default.xcm.cctv/runs/f2f14c19-8d64-448f-abfe-6f3bbc6b2a69:/mnt/mesos/sandbox"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "swan",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "host",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": true,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": [
                "label=disable"
            ],
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 10,
            "Memory": 33554432,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 67108864,
            "MemorySwappiness": -1,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay/42c1f074bd6f5c1c77e7aa2a4128f946895777ada1ca78609efdbb29e6e97973/root",
                "MergedDir": "/var/lib/docker/overlay/f01eeba9d537f8dc9a54413b5a6db5cfee4c553cf8f401825252e7d7c9081c9a/merged",
                "UpperDir": "/var/lib/docker/overlay/f01eeba9d537f8dc9a54413b5a6db5cfee4c553cf8f401825252e7d7c9081c9a/upper",
                "WorkDir": "/var/lib/docker/overlay/f01eeba9d537f8dc9a54413b5a6db5cfee4c553cf8f401825252e7d7c9081c9a/work"
            },
            "Name": "overlay"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/home",
                "Destination": "/data",
                "Mode": "rw",
                "RW": true,
                "Propagation": ""
            },
            {
                "Type": "bind",
                "Source": "/data/mesos/slaves/07239dfa-2982-4046-8f1c-f8f3c14d4649-S4/frameworks/1b7e4dbd-0e15-4761-aa44-de163bc4cb28-0000/executors/e935a28c58d6.1.nginx15.default.xcm.cctv/runs/f2f14c19-8d64-448f-abfe-6f3bbc6b2a69",
                "Destination": "/mnt/mesos/sandbox",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "9f5c311d52dc",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": true,
            "AttachStderr": true,
            "ExposedPorts": {
                "80/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "MESOS_CONTAINER_NAME=mesos-07239dfa-2982-4046-8f1c-f8f3c14d4649-S4.f2f14c19-8d64-448f-abfe-6f3bbc6b2a69",
                "MESOS_SANDBOX=/mnt/mesos/sandbox",
                "SWAN_PORT0=31001",
                "SWAN_PORT_80=31001",
                "SWAN_APP_ID=nginx15.default.xcm.cctv",
                "SWAN_APP_VERSION=1503389642651995428",
                "SWAN_TASK_ID=e935a28c58d6.1.nginx15.default.xcm.cctv",
                "SWAN_TASK_NAME=1.nginx15.default.xcm.cctv",
                "SWAN_PORTS=31001",
                "SWAN_HOST=",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "NGINX_VERSION=1.13.3-1~stretch",
                "NJS_VERSION=1.13.3.0.1.11-1~stretch"
            ],
            "Cmd": [
                "nginx",
                "-g",
                "daemon off;"
            ],
            "ArgsEscaped": true,
            "Image": "nginx",
            "Volumes": null,
            "WorkingDir": "/data",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {
                "SWAN_APP_ID": "nginx15.default.xcm.cctv",
                "SWAN_APP_VERSION": "1503389642651995428",
                "SWAN_HOST": "",
                "SWAN_PORT0": "31001",
                "SWAN_PORTS": "31001",
                "SWAN_PORT_80": "31001",
                "SWAN_TASK_ID": "e935a28c58d6.1.nginx15.default.xcm.cctv",
                "SWAN_TASK_NAME": "1.nginx15.default.xcm.cctv"
            },
            "StopSignal": "SIGTERM"
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "3128bbe43eb05032658a08cda5b7bc24db7ef10912b017300371ccfc4c4e3e12",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/3128bbe43eb0",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "swan": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "9f5c311d52dc"
                    ],
                    "NetworkID": "a39622f1c18f71325e10d04237c853f3ecab068ed7848406c20b259b95e2b6f4",
                    "EndpointID": "5bae82530eea0b098ce3b62e86d0d377d7d3956c31ee097d347059fd3938e36f",
                    "Gateway": "10.0.0.1",
                    "IPAddress": "10.0.0.84",
                    "IPPrefixLen": 24,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:9a:2a:41:fc"
                }
            }
        }
    }
]

DEBU[2017-08-22 16:14:05] Received status update TASK_RUNNING(false) for task:e935a28c58d6.1.nginx15.default.xcm.cctv,
reason:REASON_COMMAND_EXECUTOR_FAILED, message:
DEBU[2017-08-22 16:14:05] task e935a28c58d6.1.nginx15.default.xcm.cctv healthy & status:
unhealthy (pending) --> unhealthy (TASK_RUNNING) --> changed=false
DEBU[2017-08-22 16:14:05] Removing offer  07239dfa-2982-4046-8f1c-f8f3c14d4649-O40044
DEBU[2017-08-22 16:14:06] Finding task ebd0fc723f0b.0.nginx15.default.xcm.cctv to send status
TASK_RUNNING
INFO[2017-08-22 16:14:06] ======ebd0fc723f0b.0.nginx15.default.xcm.cctv-[
    {
        "Id": "9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d",
        "Created": "2017-08-22T07:57:11.82945298Z",
        "Path": "nginx",
        "Args": [
            "-g",
            "daemon off;"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 22939,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2017-08-22T07:57:13.782847516Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:e4e6d42c70b3f79c5d57c170526592168992eb3303a6594c439302fabd92d9a3",
        "ResolvConfPath": "/var/lib/docker/containers/9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d/hostname",
        "HostsPath": "/var/lib/docker/containers/9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d/hosts",
        "LogPath": "/var/lib/docker/containers/9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d/9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d-json.log",
        "Name": "/mesos-07239dfa-2982-4046-8f1c-f8f3c14d4649-S4.f2f14c19-8d64-448f-abfe-6f3bbc6b2a69",
        "RestartCount": 0,
        "Driver": "overlay",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/home:/data:rw",
                "/data/mesos/slaves/07239dfa-2982-4046-8f1c-f8f3c14d4649-S4/frameworks/1b7e4dbd-0e15-4761-aa44-de163bc4cb28-0000/executors/e935a28c58d6.1.nginx15.default.xcm.cctv/runs/f2f14c19-8d64-448f-abfe-6f3bbc6b2a69:/mnt/mesos/sandbox"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "swan",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "host",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": true,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": [
                "label=disable"
            ],
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 10,
            "Memory": 33554432,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 67108864,
            "MemorySwappiness": -1,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay/42c1f074bd6f5c1c77e7aa2a4128f946895777ada1ca78609efdbb29e6e97973/root",
                "MergedDir": "/var/lib/docker/overlay/f01eeba9d537f8dc9a54413b5a6db5cfee4c553cf8f401825252e7d7c9081c9a/merged",
                "UpperDir": "/var/lib/docker/overlay/f01eeba9d537f8dc9a54413b5a6db5cfee4c553cf8f401825252e7d7c9081c9a/upper",
                "WorkDir": "/var/lib/docker/overlay/f01eeba9d537f8dc9a54413b5a6db5cfee4c553cf8f401825252e7d7c9081c9a/work"
            },
            "Name": "overlay"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/home",
                "Destination": "/data",
                "Mode": "rw",
                "RW": true,
                "Propagation": ""
            },
            {
                "Type": "bind",
                "Source": "/data/mesos/slaves/07239dfa-2982-4046-8f1c-f8f3c14d4649-S4/frameworks/1b7e4dbd-0e15-4761-aa44-de163bc4cb28-0000/executors/e935a28c58d6.1.nginx15.default.xcm.cctv/runs/f2f14c19-8d64-448f-abfe-6f3bbc6b2a69",
                "Destination": "/mnt/mesos/sandbox",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "9f5c311d52dc",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": true,
            "AttachStderr": true,
            "ExposedPorts": {
                "80/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "MESOS_CONTAINER_NAME=mesos-07239dfa-2982-4046-8f1c-f8f3c14d4649-S4.f2f14c19-8d64-448f-abfe-6f3bbc6b2a69",
                "MESOS_SANDBOX=/mnt/mesos/sandbox",
                "SWAN_PORT0=31001",
                "SWAN_PORT_80=31001",
                "SWAN_APP_ID=nginx15.default.xcm.cctv",
                "SWAN_APP_VERSION=1503389642651995428",
                "SWAN_TASK_ID=e935a28c58d6.1.nginx15.default.xcm.cctv",
                "SWAN_TASK_NAME=1.nginx15.default.xcm.cctv",
                "SWAN_PORTS=31001",
                "SWAN_HOST=",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "NGINX_VERSION=1.13.3-1~stretch",
                "NJS_VERSION=1.13.3.0.1.11-1~stretch"
            ],
            "Cmd": [
                "nginx",
                "-g",
                "daemon off;"
            ],
            "ArgsEscaped": true,
            "Image": "nginx",
            "Volumes": null,
            "WorkingDir": "/data",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {
                "SWAN_APP_ID": "nginx15.default.xcm.cctv",
                "SWAN_APP_VERSION": "1503389642651995428",
                "SWAN_HOST": "",
                "SWAN_PORT0": "31001",
                "SWAN_PORTS": "31001",
                "SWAN_PORT_80": "31001",
                "SWAN_TASK_ID": "e935a28c58d6.1.nginx15.default.xcm.cctv",
                "SWAN_TASK_NAME": "1.nginx15.default.xcm.cctv"
            },
            "StopSignal": "SIGTERM"
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "3128bbe43eb05032658a08cda5b7bc24db7ef10912b017300371ccfc4c4e3e12",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/3128bbe43eb0",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "swan": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "9f5c311d52dc"
                    ],
                    "NetworkID": "a39622f1c18f71325e10d04237c853f3ecab068ed7848406c20b259b95e2b6f4",
                    "EndpointID": "5bae82530eea0b098ce3b62e86d0d377d7d3956c31ee097d347059fd3938e36f",
                    "Gateway": "10.0.0.1",
                    "IPAddress": "10.0.0.84",
                    "IPPrefixLen": 24,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:9a:2a:41:fc"
                }
            }
        }
    }
]

DEBU[2017-08-22 16:14:06] Received status update TASK_RUNNING(true) for task:ebd0fc723f0b.0.nginx15.default.xcm.cctv,
reason:REASON_COMMAND_EXECUTOR_FAILED, message:
DEBU[2017-08-22 16:14:06] task ebd0fc723f0b.0.nginx15.default.xcm.cctv healthy & status:
unhealthy (TASK_RUNNING) --> healty (TASK_RUNNING) --> changed=true
DEBU[2017-08-22 16:14:06] sending task changed event ebd0fc723f0b.0.nginx15.default.xcm.cctv
DEBU[2017-08-22 16:14:07] Finding task e935a28c58d6.1.nginx15.default.xcm.cctv to send status
TASK_RUNNING
INFO[2017-08-22 16:14:07] ======e935a28c58d6.1.nginx15.default.xcm.cctv-[
    {
        "Id": "9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d",
        "Created": "2017-08-22T07:57:11.82945298Z",
        "Path": "nginx",
        "Args": [
            "-g",
            "daemon off;"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 22939,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2017-08-22T07:57:13.782847516Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:e4e6d42c70b3f79c5d57c170526592168992eb3303a6594c439302fabd92d9a3",
        "ResolvConfPath": "/var/lib/docker/containers/9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d/hostname",
        "HostsPath": "/var/lib/docker/containers/9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d/hosts",
        "LogPath": "/var/lib/docker/containers/9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d/9f5c311d52dc3d549bb0e870d3d53d085497056534d6204505aa5fbcea1dc52d-json.log",
        "Name": "/mesos-07239dfa-2982-4046-8f1c-f8f3c14d4649-S4.f2f14c19-8d64-448f-abfe-6f3bbc6b2a69",
        "RestartCount": 0,
        "Driver": "overlay",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/home:/data:rw",
                "/data/mesos/slaves/07239dfa-2982-4046-8f1c-f8f3c14d4649-S4/frameworks/1b7e4dbd-0e15-4761-aa44-de163bc4cb28-0000/executors/e935a28c58d6.1.nginx15.default.xcm.cctv/runs/f2f14c19-8d64-448f-abfe-6f3bbc6b2a69:/mnt/mesos/sandbox"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "swan",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "host",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": true,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": [
                "label=disable"
            ],
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 10,
            "Memory": 33554432,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 67108864,
            "MemorySwappiness": -1,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay/42c1f074bd6f5c1c77e7aa2a4128f946895777ada1ca78609efdbb29e6e97973/root",
                "MergedDir": "/var/lib/docker/overlay/f01eeba9d537f8dc9a54413b5a6db5cfee4c553cf8f401825252e7d7c9081c9a/merged",
                "UpperDir": "/var/lib/docker/overlay/f01eeba9d537f8dc9a54413b5a6db5cfee4c553cf8f401825252e7d7c9081c9a/upper",
                "WorkDir": "/var/lib/docker/overlay/f01eeba9d537f8dc9a54413b5a6db5cfee4c553cf8f401825252e7d7c9081c9a/work"
            },
            "Name": "overlay"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/home",
                "Destination": "/data",
                "Mode": "rw",
                "RW": true,
                "Propagation": ""
            },
            {
                "Type": "bind",
                "Source": "/data/mesos/slaves/07239dfa-2982-4046-8f1c-f8f3c14d4649-S4/frameworks/1b7e4dbd-0e15-4761-aa44-de163bc4cb28-0000/executors/e935a28c58d6.1.nginx15.default.xcm.cctv/runs/f2f14c19-8d64-448f-abfe-6f3bbc6b2a69",
                "Destination": "/mnt/mesos/sandbox",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "9f5c311d52dc",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": true,
            "AttachStderr": true,
            "ExposedPorts": {
                "80/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "MESOS_CONTAINER_NAME=mesos-07239dfa-2982-4046-8f1c-f8f3c14d4649-S4.f2f14c19-8d64-448f-abfe-6f3bbc6b2a69",
                "MESOS_SANDBOX=/mnt/mesos/sandbox",
                "SWAN_PORT0=31001",
                "SWAN_PORT_80=31001",
                "SWAN_APP_ID=nginx15.default.xcm.cctv",
                "SWAN_APP_VERSION=1503389642651995428",
                "SWAN_TASK_ID=e935a28c58d6.1.nginx15.default.xcm.cctv",
                "SWAN_TASK_NAME=1.nginx15.default.xcm.cctv",
                "SWAN_PORTS=31001",
                "SWAN_HOST=",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "NGINX_VERSION=1.13.3-1~stretch",
                "NJS_VERSION=1.13.3.0.1.11-1~stretch"
            ],
            "Cmd": [
                "nginx",
                "-g",
                "daemon off;"
            ],
            "ArgsEscaped": true,
            "Image": "nginx",
            "Volumes": null,
            "WorkingDir": "/data",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {
                "SWAN_APP_ID": "nginx15.default.xcm.cctv",
                "SWAN_APP_VERSION": "1503389642651995428",
                "SWAN_HOST": "",
                "SWAN_PORT0": "31001",
                "SWAN_PORTS": "31001",
                "SWAN_PORT_80": "31001",
                "SWAN_TASK_ID": "e935a28c58d6.1.nginx15.default.xcm.cctv",
                "SWAN_TASK_NAME": "1.nginx15.default.xcm.cctv"
            },
            "StopSignal": "SIGTERM"
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "3128bbe43eb05032658a08cda5b7bc24db7ef10912b017300371ccfc4c4e3e12",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/3128bbe43eb0",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "swan": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "9f5c311d52dc"
                    ],
                    "NetworkID": "a39622f1c18f71325e10d04237c853f3ecab068ed7848406c20b259b95e2b6f4",
                    "EndpointID": "5bae82530eea0b098ce3b62e86d0d377d7d3956c31ee097d347059fd3938e36f",
                    "Gateway": "10.0.0.1",
                    "IPAddress": "10.0.0.84",
                    "IPPrefixLen": 24,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:9a:2a:41:fc"
                }
            }
        }
    }
]

DEBU[2017-08-22 16:14:07] Received status update TASK_RUNNING(true) for task:e935a28c58d6.1.nginx15.default.xcm.cctv,
reason:REASON_COMMAND_EXECUTOR_FAILED, message:
DEBU[2017-08-22 16:14:07] task e935a28c58d6.1.nginx15.default.xcm.cctv healthy & status:
unhealthy (TASK_RUNNING) --> healty (TASK_RUNNING) --> changed=true
DEBU[2017-08-22 16:14:07] sending task changed event e935a28c58d6.1.nginx15.default.xcm.cctv)

> mesos send the wrong container data while health check is set
> -------------------------------------------------------------
>
>                 Key: MESOS-7906
>                 URL: https://issues.apache.org/jira/browse/MESOS-7906
>             Project: Mesos
>          Issue Type: Bug
>          Components: executor
>    Affects Versions: 1.3.0
>            Reporter: niuminguo
>              Labels: health-check
>
> I found mesos send the wrong docker container data in statusUpdate when task health check
is set. 
> I have one app with two tasks: task1 and task2, set healthy check for each task. when
task become running(health is false), i got the
> container data from status update event and which is correct.  when health is true, i
received status update event again and found the container data is wrong: task1 has the container
data which is belong to task2. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Mime
View raw message