mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Rojas (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MESOS-7887) `GET_EXECUTORS` and `/state` is not consistent between master and agent
Date Mon, 14 Aug 2017 12:54:00 GMT

     [ https://issues.apache.org/jira/browse/MESOS-7887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alexander Rojas updated MESOS-7887:
-----------------------------------
    Description: 
The master seem not to keep information about the executors since they are not returned either
either by getting the master state (with either v0 and v1 API's) or with the call {{GET_EXECUTORS}}.
Creating a cluster as follows:

{noformat}
./bin/mesos-master.sh \
    --ip=${MASTER_IP} \
    --work_dir=/tmp/mesos/master \
    --log_dir=/tmp/mesos/master/log
{noformat}

{noformat}
sudo ./bin/mesos-agent.sh \
    --master=${MASTER_IP}:5050 \
    --work_dir=/tmp/mesos/agent \
    --log_dir=/tmp/mesos/agent/log \
    --containerizers=mesos,docker
{noformat}

And launch  a couple of frameworks as follows:

{noformat}
./src/mesos-execute \
    --master=${MASTER_IP}:5050 \
    --task='{"name":"test-custom-command","task_id":{"value":"test-custom-command-task-1"},"agent_id":{"value":"50f4e551-aa5c-42db-8967-4dc3ee11658f-S0"},"resources":[{"name":"cpus","type":"SCALAR","scalar":{"value":1}},{"name":"mem","type":"SCALAR","scalar":{"value":32}},{"name":"disk","type":"SCALAR","scalar":{"value":32}}],"executor":{"executor_id":{"value":"test-custom-command-executor"},"command":{"value":"while
true; do echo \"Hello World\"; sleep 5; done;"}}}'
{noformat}

{noformat}
./src/mesos-execute \
    --master=${MASTER_IP}:5050 \
    --name=test-command \
    --command='while true; do echo "Hello World"; sleep 5; done;' \
    --containerizer=docker \
    --docker_image=ubuntu:latest
{noformat}

Not using the operator endpoints on the agent:

{noformat}
$ http POST ${AGENT_IP}:5051/api/v1 type=GET_EXECUTORS

{
  "get_executors": {
    "completed_executors": [
    ],
    "executors": [
      {
        "executor_info": {
          "command": {
            "arguments": [
              "mesos-executor",
              "--launcher_dir=/workspace/mesos/build/src"
            ],
            "shell": false,
            "value": "/workspace/mesos/build/src/mesos-executor"
          },
          "container": {
            "docker": {
              "image": "ubuntu:latest",
              "network": "HOST",
              "privileged": false
            },
            "type": "DOCKER"
          },
          "executor_id": {
            "value": "test-command"
          },
          "framework_id": {
            "value": "87577bcd-093d-4240-a24b-107b4d1d21bd-0001"
          },
          "name": "Command Executor (Task: test-command) (Command: sh -c 'while true; ...')",
          "resources": [
            {
              "allocation_info": {
                "role": "*"
              },
              "name": "cpus",
              "scalar": {
                "value": 0.1
              },
              "type": "SCALAR"
            },
            {
              "allocation_info": {
                "role": "*"
              },
              "name": "mem",
              "scalar": {
                "value": 32
              },
              "type": "SCALAR"
            }
          ],
          "source": "test-command"
        }
      },
      {
        "executor_info": {
          "command": {
            "shell": true,
            "value": "while true; do echo \"Hello World\"; sleep 5; done;"
          },
          "executor_id": {
            "value": "test-custom-command-executor"
          },
          "framework_id": {
            "value": "87577bcd-093d-4240-a24b-107b4d1d21bd-0000"
          }
        }
      }
    ]
  },
  "type": "GET_EXECUTORS"
}
{noformat}

While the master does

{noformat}
 http POST ${MASTER_IP}:5050/api/v1 type=GET_EXECUTORS

{
    "get_executors": {},
    "type": "GET_EXECUTORS"
}
{noformat}

These results are consistent using the `/state` endpoint on both, agent and master as well
as using the {{GET_STATE}} v1 API call. The agent returns information about executors, while
the master response has none.

  was:
The master seem not to keep information about the executors since they are not returned either
either by getting the master state (with either v0 and v1 API's) or with the call `GET_EXECUTORS`.
Creating a cluster as follows:

{noformat}
./bin/mesos-master.sh \
    --ip=${MASTER_IP} \
    --work_dir=/tmp/mesos/master \
    --log_dir=/tmp/mesos/master/log
{noformat}

{noformat}
sudo ./bin/mesos-agent.sh \
    --master=${MASTER_IP}:5050 \
    --work_dir=/tmp/mesos/agent \
    --log_dir=/tmp/mesos/agent/log \
    --containerizers=mesos,docker
{noformat}

And launch  a couple of frameworks as follows:

{noformat}
./src/mesos-execute \
    --master=${MASTER_IP}:5050 \
    --task='{"name":"test-custom-command","task_id":{"value":"test-custom-command-task-1"},"agent_id":{"value":"50f4e551-aa5c-42db-8967-4dc3ee11658f-S0"},"resources":[{"name":"cpus","type":"SCALAR","scalar":{"value":1}},{"name":"mem","type":"SCALAR","scalar":{"value":32}},{"name":"disk","type":"SCALAR","scalar":{"value":32}}],"executor":{"executor_id":{"value":"test-custom-command-executor"},"command":{"value":"while
true; do echo \"Hello World\"; sleep 5; done;"}}}'
{noformat}

{noformat}
./src/mesos-execute \
    --master=${MASTER_IP}:5050 \
    --name=test-command \
    --command='while true; do echo "Hello World"; sleep 5; done;' \
    --containerizer=docker \
    --docker_image=ubuntu:latest
{noformat}

Not using the operator endpoints on the agent:

{noformat}
$ http POST ${AGENT_IP}:5051/api/v1 type=GET_EXECUTORS

{
  "get_executors": {
    "completed_executors": [
    ],
    "executors": [
      {
        "executor_info": {
          "command": {
            "arguments": [
              "mesos-executor",
              "--launcher_dir=/workspace/mesos/build/src"
            ],
            "shell": false,
            "value": "/workspace/mesos/build/src/mesos-executor"
          },
          "container": {
            "docker": {
              "image": "ubuntu:latest",
              "network": "HOST",
              "privileged": false
            },
            "type": "DOCKER"
          },
          "executor_id": {
            "value": "test-command"
          },
          "framework_id": {
            "value": "87577bcd-093d-4240-a24b-107b4d1d21bd-0001"
          },
          "name": "Command Executor (Task: test-command) (Command: sh -c 'while true; ...')",
          "resources": [
            {
              "allocation_info": {
                "role": "*"
              },
              "name": "cpus",
              "scalar": {
                "value": 0.1
              },
              "type": "SCALAR"
            },
            {
              "allocation_info": {
                "role": "*"
              },
              "name": "mem",
              "scalar": {
                "value": 32
              },
              "type": "SCALAR"
            }
          ],
          "source": "test-command"
        }
      },
      {
        "executor_info": {
          "command": {
            "shell": true,
            "value": "while true; do echo \"Hello World\"; sleep 5; done;"
          },
          "executor_id": {
            "value": "test-custom-command-executor"
          },
          "framework_id": {
            "value": "87577bcd-093d-4240-a24b-107b4d1d21bd-0000"
          }
        }
      }
    ]
  },
  "type": "GET_EXECUTORS"
}
{noformat}

While the master does

{noformat}
 http POST ${MASTER_IP}:5050/api/v1 type=GET_EXECUTORS

{
    "get_executors": {},
    "type": "GET_EXECUTORS"
}
{noformat}

These results are consistent using the `/state` endpoint on both, agent and master as well
as using the {{GET_STATE}} v1 API call. The agent returns information about executors, while
the master response has none.


> `GET_EXECUTORS` and `/state` is not consistent between master and agent
> -----------------------------------------------------------------------
>
>                 Key: MESOS-7887
>                 URL: https://issues.apache.org/jira/browse/MESOS-7887
>             Project: Mesos
>          Issue Type: Bug
>          Components: HTTP API, master
>    Affects Versions: 1.3.0
>            Reporter: Alexander Rojas
>              Labels: master, mesosphere, v1_api
>
> The master seem not to keep information about the executors since they are not returned
either either by getting the master state (with either v0 and v1 API's) or with the call {{GET_EXECUTORS}}.
Creating a cluster as follows:
> {noformat}
> ./bin/mesos-master.sh \
>     --ip=${MASTER_IP} \
>     --work_dir=/tmp/mesos/master \
>     --log_dir=/tmp/mesos/master/log
> {noformat}
> {noformat}
> sudo ./bin/mesos-agent.sh \
>     --master=${MASTER_IP}:5050 \
>     --work_dir=/tmp/mesos/agent \
>     --log_dir=/tmp/mesos/agent/log \
>     --containerizers=mesos,docker
> {noformat}
> And launch  a couple of frameworks as follows:
> {noformat}
> ./src/mesos-execute \
>     --master=${MASTER_IP}:5050 \
>     --task='{"name":"test-custom-command","task_id":{"value":"test-custom-command-task-1"},"agent_id":{"value":"50f4e551-aa5c-42db-8967-4dc3ee11658f-S0"},"resources":[{"name":"cpus","type":"SCALAR","scalar":{"value":1}},{"name":"mem","type":"SCALAR","scalar":{"value":32}},{"name":"disk","type":"SCALAR","scalar":{"value":32}}],"executor":{"executor_id":{"value":"test-custom-command-executor"},"command":{"value":"while
true; do echo \"Hello World\"; sleep 5; done;"}}}'
> {noformat}
> {noformat}
> ./src/mesos-execute \
>     --master=${MASTER_IP}:5050 \
>     --name=test-command \
>     --command='while true; do echo "Hello World"; sleep 5; done;' \
>     --containerizer=docker \
>     --docker_image=ubuntu:latest
> {noformat}
> Not using the operator endpoints on the agent:
> {noformat}
> $ http POST ${AGENT_IP}:5051/api/v1 type=GET_EXECUTORS
> {
>   "get_executors": {
>     "completed_executors": [
>     ],
>     "executors": [
>       {
>         "executor_info": {
>           "command": {
>             "arguments": [
>               "mesos-executor",
>               "--launcher_dir=/workspace/mesos/build/src"
>             ],
>             "shell": false,
>             "value": "/workspace/mesos/build/src/mesos-executor"
>           },
>           "container": {
>             "docker": {
>               "image": "ubuntu:latest",
>               "network": "HOST",
>               "privileged": false
>             },
>             "type": "DOCKER"
>           },
>           "executor_id": {
>             "value": "test-command"
>           },
>           "framework_id": {
>             "value": "87577bcd-093d-4240-a24b-107b4d1d21bd-0001"
>           },
>           "name": "Command Executor (Task: test-command) (Command: sh -c 'while true;
...')",
>           "resources": [
>             {
>               "allocation_info": {
>                 "role": "*"
>               },
>               "name": "cpus",
>               "scalar": {
>                 "value": 0.1
>               },
>               "type": "SCALAR"
>             },
>             {
>               "allocation_info": {
>                 "role": "*"
>               },
>               "name": "mem",
>               "scalar": {
>                 "value": 32
>               },
>               "type": "SCALAR"
>             }
>           ],
>           "source": "test-command"
>         }
>       },
>       {
>         "executor_info": {
>           "command": {
>             "shell": true,
>             "value": "while true; do echo \"Hello World\"; sleep 5; done;"
>           },
>           "executor_id": {
>             "value": "test-custom-command-executor"
>           },
>           "framework_id": {
>             "value": "87577bcd-093d-4240-a24b-107b4d1d21bd-0000"
>           }
>         }
>       }
>     ]
>   },
>   "type": "GET_EXECUTORS"
> }
> {noformat}
> While the master does
> {noformat}
>  http POST ${MASTER_IP}:5050/api/v1 type=GET_EXECUTORS
> {
>     "get_executors": {},
>     "type": "GET_EXECUTORS"
> }
> {noformat}
> These results are consistent using the `/state` endpoint on both, agent and master as
well as using the {{GET_STATE}} v1 API call. The agent returns information about executors,
while the master response has none.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message