mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Rojas (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MESOS-8283) Operator API `SUBSCRIBE` call doesn't fully respect acls
Date Thu, 30 Nov 2017 15:45:00 GMT

     [ https://issues.apache.org/jira/browse/MESOS-8283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alexander Rojas updated MESOS-8283:
-----------------------------------
    Description: 
When an operator subscribes to the event stream using the {{SUBSCRIBE}} call, the initial
response is properly authorized (with filter in the state). The same holds true for the events
{{TASK_ADDED}} and {{TASK_UPDATED}} which, if a user has no right to see them he won't receive
the updates, {{AGENT_ADDED}} seems to be respected too. However any user will get {{FRAMEWORK_ADDED}}
and {{FRAMEWORK_UPDATED}} events.

I didn't {{AGENT_REMOVED}}

In order to test I used the following acls:

{code:javascript}
{
  "permissive": true,
  "view_roles" : [
   {
     "principals" : { "type" : "ANY" },
     "roles" : { "values" : ["*"] }
   },
   {
     "principals" : { "values" : ["hal-9000"] },
     "roles" : { "type" : "ANY" }
   },
   {
     "principals" : { "values" : ["glados"] },
     "roles" : { "type" : "NONE" }
   }
  ],
  "view_framework": [
   {
     "principals" : { "values" : ["hal-9000"] },
     "users" : { "type" : "ANY" }
   },
   {
     "principals" : { "values" : ["glados"] },
     "users" : { "type" : "NONE" }
   }
  ],
  "view_tasks": [
   {
     "principals" : { "values" : ["hal-9000"] },
     "users" : { "type" : "ANY" }
   },
   {
     "principals" : { "values" : ["glados"] },
     "users" : { "type" : "NONE" }
   }
  ],
  "register_agent": [
   {
     "principals" : { "values" : ["hal-9000"] },
     "users" : { "type" : "ANY" }
   },
   {
     "principals" : { "values" : ["glados"] },
     "users" : { "type" : "NONE" }
   }
  ]
}
{code}

the following credential files:

{noformat}
super super
hal-9000 hal-9000
glados glados
{noformat}

And launch a master and an agent as follows:

{noformat}
./mesos-master.sh \
  --work_dir=/tmp/$USER/mesos/master \
  --log_dir=/tmp/$USER/mesos/master/log \
  --ip=${MASTER_IP} \
  --agent_ping_timeout=5secs \
  --max_agent_ping_timeouts=2 \
  --agent_removal_rate_limit=1/1secs \
  --http_framework_authenticators=basic \
  --authenticate_http \
  --authenticate_http_frameworks \
  --acls=$HOME/testing/acls.json \
  --credentials=$HOME/testing/credentials.txt

sudo ./mesos-agent.sh \
     --work_dir=/tmp/$USER/mesos/agent \
     --log_dir=/tmp/$USER/mesos/agent/log \
     --containerizers=mesos,docker \
     --master=${MASTER_IP}:5050 \
     --authenticate_http_readwrite \
     --http_authenticators=basic \
     --acls=$HOME/Documents/workspace/testing/acls.json \
     --http_credentials=$HOME/testing/credentials.txt
{noformat}

Once the master and agent are running, use the following script:

{code}
#! /usr/bin/env python3

import json
import recordio
import requests
import sys

from contextlib import closing
from typing import Dict

MACHINE_NAME='ip.to.machine'
MACHINE_IP='ip.to.machine'

def test_mesos_v1_master_subscribe(host: str, ip: str) -> None:
    base_url = 'http://{}:5050'.format(host)

    headers = {
        'Content-Type': 'application/json',
        'Accept': 'application/json',
        'Connection': 'close',
    }

    decoder = recordio.Decoder(lambda s: json.loads(s.decode("UTF-8")))

    with closing(requests.post(
            base_url + '/api/v1',
            json={'type': 'SUBSCRIBE'},
            auth=('hal-9000', 'hal-9000'),
            headers=headers,
            stream=True,
    )) as response:
        assert response.status_code == 200, response.text
        chunks = response.iter_content(chunk_size=None)
        events = []  # type: ignore
        for chunk in chunks:
            for record in decoder.decode(chunk):
                if record['type'] != 'HEARTBEAT':
                    print('got event "' + record['type'])
                    events.append(record['type'])
            # It takes at least 10 mins for an agent to be marked as gone.
            if len(events) >= 7:
                print('got the following events "' + '", "'.join(events) + '"')
                break

    with closing(requests.post(
            base_url + '/api/v1',
            json={'type': 'SUBSCRIBE'},
            auth=('glados', 'glados'),
            headers=headers,
            stream=True,
    )) as response:
        assert response.status_code == 200, response.text
        chunks = response.iter_content(chunk_size=None)
        for chunk in chunks:
            for record in decoder.decode(chunk):
                if record['type'] not in ('HEARTBEAT', 'SUBSCRIBED'):
                    print('It wasn\'t supposed to get "' + record['type'] + '"')

def main(args: Dict[str, str]) -> None:
    test_mesos_v1_master_subscribe(MACHINE_NAME, MACHINE_IP)

if __name__ == '__main__':
    main(sys.argv)
{code}

Finally while the machine is running you do the following:

1. Launch an extra agent:
{noformat}
sudo ./mesos-agent.sh \
     --work_dir=/tmp/$USER/mesos/agent-2 \
     --log_dir=/tmp/$USER/mesos/agent-2/log \
     --containerizers=mesos,docker \
     --port=5152 \
     --master=${MASTER_IP}:5050 \
     --authenticate_http_readwrite \
     --http_authenticators=basic \
     --acls=$HOME/testing/acls.json \
     --http_credentials=$HOME/testing/credentials.txt
{noformat}
2. Teardown the agent.
3. Launch a framework and task:
{noformat}
./src/mesos-execute --master=${MASTER_IP}:5050 --command='while true; do echo "Hello World";
sleep 5; done;' --resources="cpus:1;mem:128;disk:32;ports:[31002-31003]" --name=hello-discovery
--principal=hal-9000 --secret=hal-9000
{noformat}
4. Teardown the framework by killing the program {{mesos-execute}}.
5. Repeat (1) and (2).

The output of the script is the following:

{noformat}
got the following events "SUBSCRIBED", "FRAMEWORK_ADDED", "TASK_ADDED", "TASK_UPDATED", "TASK_UPDATED",
"FRAMEWORK_REMOVED", "AGENT_ADDED"
It wasn't supposed to get "FRAMEWORK_ADDED"
It wasn't supposed to get "FRAMEWORK_REMOVED"
{noformat}

The last two lines should not appear.

  was:
When an operator subscribes to the event stream using the {{SUBSCRIBE}} call, the initial
response is properly authorized (with filter in the state). The same holds true for the events
{{TASK_ADDED}} and {{TASK_UPDATED}} which, if a user has no right to see them he won't receive
the updates, {{AGENT_ADDED}} seems to be respected too. However any user will get {{FRAMEWORK_ADDED}}
and {{FRAMEWORK_UPDATED}} events.

I didn't {{AGENT_REMOVED}}

In order to test I used the following acls:

{code:javascript}
{
  "permissive": true,
  "view_roles" : [
   {
     "principals" : { "type" : "ANY" },
     "roles" : { "values" : ["*"] }
   },
   {
     "principals" : { "values" : ["hal-9000"] },
     "roles" : { "type" : "ANY" }
   },
   {
     "principals" : { "values" : ["glados"] },
     "roles" : { "type" : "NONE" }
   }
  ],
  "view_framework": [
   {
     "principals" : { "values" : ["hal-9000"] },
     "users" : { "type" : "ANY" }
   },
   {
     "principals" : { "values" : ["glados"] },
     "users" : { "type" : "NONE" }
   }
  ],
  "view_tasks": [
   {
     "principals" : { "values" : ["hal-9000"] },
     "users" : { "type" : "ANY" }
   },
   {
     "principals" : { "values" : ["glados"] },
     "users" : { "type" : "NONE" }
   }
  ],
  "register_agent": [
   {
     "principals" : { "values" : ["hal-9000"] },
     "users" : { "type" : "ANY" }
   },
   {
     "principals" : { "values" : ["glados"] },
     "users" : { "type" : "NONE" }
   }
  ]
}
{code}

the following credential files:

{noformat}
super super
hal-9000 hal-9000
glados glados
{noformat}

And launch a master and an agent as follows:

{noformat}
./mesos-master.sh \
  --work_dir=/tmp/$USER/mesos/master \
  --log_dir=/tmp/$USER/mesos/master/log \
  --ip=${MASTER_IP} \
  --agent_ping_timeout=5secs \
  --max_agent_ping_timeouts=2 \
  --agent_removal_rate_limit=1/1secs \
  --http_framework_authenticators=basic \
  --authenticate_http \
  --authenticate_http_frameworks \
  --acls=$HOME/testing/acls.json \
  --credentials=$HOME/testing/credentials.txt

sudo ./mesos-agent.sh \
     --work_dir=/tmp/$USER/mesos/agent \
     --log_dir=/tmp/$USER/mesos/agent/log \
     --containerizers=mesos,docker \
     --master=${MASTER_IP}:5050 \
     --authenticate_http_readwrite \
     --http_authenticators=basic \
     --acls=$HOME/Documents/workspace/testing/acls.json \
     --http_credentials=$HOME/testing/credentials.txt
{noformat}

Once the master and agent are running, use the following script:

{code}
#! /usr/bin/env python3

import json
import recordio
import requests
import sys

from contextlib import closing
from typing import Dict

MACHINE_NAME='ip.to.machine'
MACHINE_IP='ip.to.machine'

def test_mesos_v1_master_subscribe(host: str, ip: str) -> None:
    base_url = 'http://{}:5050'.format(host)

    headers = {
        'Content-Type': 'application/json',
        'Accept': 'application/json',
        'Connection': 'close',
    }

    decoder = recordio.Decoder(lambda s: json.loads(s.decode("UTF-8")))

    with closing(requests.post(
            base_url + '/api/v1',
            json={'type': 'SUBSCRIBE'},
            auth=('hal-9000', 'hal-9000'),
            headers=headers,
            stream=True,
    )) as response:
        assert response.status_code == 200, response.text
        chunks = response.iter_content(chunk_size=None)
        events = []  # type: ignore
        for chunk in chunks:
            for record in decoder.decode(chunk):
                if record['type'] != 'HEARTBEAT':
                    print('got event "' + record['type'])
                    events.append(record['type'])
            # It takes at least 10 mins for an agent to be marked as gone.
            if len(events) >= 7:
                print('got the following events "' + '", "'.join(events) + '"')
                break

    with closing(requests.post(
            base_url + '/api/v1',
            json={'type': 'SUBSCRIBE'},
            auth=('glados', 'glados'),
            headers=headers,
            stream=True,
    )) as response:
        assert response.status_code == 200, response.text
        chunks = response.iter_content(chunk_size=None)
        for chunk in chunks:
            for record in decoder.decode(chunk):
                if record['type'] not in ('HEARTBEAT', 'SUBSCRIBED'):
                    print('It wasn\'t supposed to get "' + record['type'] + '"')

def main(args: Dict[str, str]) -> None:
    test_mesos_v1_master_subscribe(MACHINE_NAME, MACHINE_IP)

if __name__ == '__main__':
    main(sys.argv)
{code}

Finally while the machine is running you do the following:

1. Launch an extra agent:
{noformat}
sudo ./mesos-agent.sh \
     --work_dir=/tmp/$USER/mesos/agent-2 \
     --log_dir=/tmp/$USER/mesos/agent-2/log \
     --containerizers=mesos,docker \
     --port=5152 \
     --master=${MASTER_IP}:5050 \
     --authenticate_http_readwrite \
     --http_authenticators=basic \
     --acls=$HOME/testing/acls.json \
     --http_credentials=$HOME/testing/credentials.txt
{noformat}
2. Teardown the agent.
3. Launch a framework and task:
{noformat}
./src/mesos-execute --master=${MASTER_IP}:5050 --command='while true; do echo "Hello World";
sleep 5; done;' --resources="cpus:1;mem:128;disk:32;ports:[31002-31003]" --name=hello-discovery
--principal=hal-9000 --secret=hal-9000
{noformat}
4. Teardown the framework by killing the program {{mesos-execute}}.
5. Repeat (1) and (2).

The output of the script is the following:

{noformat}
got the following events "SUBSCRIBED", "FRAMEWORK_ADDED", "TASK_ADDED", "TASK_UPDATED", "TASK_UPDATED",
"FRAMEWORK_REMOVED", "AGENT_ADDED", "AGENT_ADDED"
It wasn't supposed to get "FRAMEWORK_ADDED"
It wasn't supposed to get "FRAMEWORK_REMOVED"
{noformat}


> Operator API `SUBSCRIBE` call doesn't fully respect acls
> --------------------------------------------------------
>
>                 Key: MESOS-8283
>                 URL: https://issues.apache.org/jira/browse/MESOS-8283
>             Project: Mesos
>          Issue Type: Bug
>          Components: HTTP API, master
>    Affects Versions: 1.4.0
>            Reporter: Alexander Rojas
>              Labels: mesosphere
>
> When an operator subscribes to the event stream using the {{SUBSCRIBE}} call, the initial
response is properly authorized (with filter in the state). The same holds true for the events
{{TASK_ADDED}} and {{TASK_UPDATED}} which, if a user has no right to see them he won't receive
the updates, {{AGENT_ADDED}} seems to be respected too. However any user will get {{FRAMEWORK_ADDED}}
and {{FRAMEWORK_UPDATED}} events.
> I didn't {{AGENT_REMOVED}}
> In order to test I used the following acls:
> {code:javascript}
> {
>   "permissive": true,
>   "view_roles" : [
>    {
>      "principals" : { "type" : "ANY" },
>      "roles" : { "values" : ["*"] }
>    },
>    {
>      "principals" : { "values" : ["hal-9000"] },
>      "roles" : { "type" : "ANY" }
>    },
>    {
>      "principals" : { "values" : ["glados"] },
>      "roles" : { "type" : "NONE" }
>    }
>   ],
>   "view_framework": [
>    {
>      "principals" : { "values" : ["hal-9000"] },
>      "users" : { "type" : "ANY" }
>    },
>    {
>      "principals" : { "values" : ["glados"] },
>      "users" : { "type" : "NONE" }
>    }
>   ],
>   "view_tasks": [
>    {
>      "principals" : { "values" : ["hal-9000"] },
>      "users" : { "type" : "ANY" }
>    },
>    {
>      "principals" : { "values" : ["glados"] },
>      "users" : { "type" : "NONE" }
>    }
>   ],
>   "register_agent": [
>    {
>      "principals" : { "values" : ["hal-9000"] },
>      "users" : { "type" : "ANY" }
>    },
>    {
>      "principals" : { "values" : ["glados"] },
>      "users" : { "type" : "NONE" }
>    }
>   ]
> }
> {code}
> the following credential files:
> {noformat}
> super super
> hal-9000 hal-9000
> glados glados
> {noformat}
> And launch a master and an agent as follows:
> {noformat}
> ./mesos-master.sh \
>   --work_dir=/tmp/$USER/mesos/master \
>   --log_dir=/tmp/$USER/mesos/master/log \
>   --ip=${MASTER_IP} \
>   --agent_ping_timeout=5secs \
>   --max_agent_ping_timeouts=2 \
>   --agent_removal_rate_limit=1/1secs \
>   --http_framework_authenticators=basic \
>   --authenticate_http \
>   --authenticate_http_frameworks \
>   --acls=$HOME/testing/acls.json \
>   --credentials=$HOME/testing/credentials.txt
> sudo ./mesos-agent.sh \
>      --work_dir=/tmp/$USER/mesos/agent \
>      --log_dir=/tmp/$USER/mesos/agent/log \
>      --containerizers=mesos,docker \
>      --master=${MASTER_IP}:5050 \
>      --authenticate_http_readwrite \
>      --http_authenticators=basic \
>      --acls=$HOME/Documents/workspace/testing/acls.json \
>      --http_credentials=$HOME/testing/credentials.txt
> {noformat}
> Once the master and agent are running, use the following script:
> {code}
> #! /usr/bin/env python3
> import json
> import recordio
> import requests
> import sys
> from contextlib import closing
> from typing import Dict
> MACHINE_NAME='ip.to.machine'
> MACHINE_IP='ip.to.machine'
> def test_mesos_v1_master_subscribe(host: str, ip: str) -> None:
>     base_url = 'http://{}:5050'.format(host)
>     headers = {
>         'Content-Type': 'application/json',
>         'Accept': 'application/json',
>         'Connection': 'close',
>     }
>     decoder = recordio.Decoder(lambda s: json.loads(s.decode("UTF-8")))
>     with closing(requests.post(
>             base_url + '/api/v1',
>             json={'type': 'SUBSCRIBE'},
>             auth=('hal-9000', 'hal-9000'),
>             headers=headers,
>             stream=True,
>     )) as response:
>         assert response.status_code == 200, response.text
>         chunks = response.iter_content(chunk_size=None)
>         events = []  # type: ignore
>         for chunk in chunks:
>             for record in decoder.decode(chunk):
>                 if record['type'] != 'HEARTBEAT':
>                     print('got event "' + record['type'])
>                     events.append(record['type'])
>             # It takes at least 10 mins for an agent to be marked as gone.
>             if len(events) >= 7:
>                 print('got the following events "' + '", "'.join(events) + '"')
>                 break
>     with closing(requests.post(
>             base_url + '/api/v1',
>             json={'type': 'SUBSCRIBE'},
>             auth=('glados', 'glados'),
>             headers=headers,
>             stream=True,
>     )) as response:
>         assert response.status_code == 200, response.text
>         chunks = response.iter_content(chunk_size=None)
>         for chunk in chunks:
>             for record in decoder.decode(chunk):
>                 if record['type'] not in ('HEARTBEAT', 'SUBSCRIBED'):
>                     print('It wasn\'t supposed to get "' + record['type'] + '"')
> def main(args: Dict[str, str]) -> None:
>     test_mesos_v1_master_subscribe(MACHINE_NAME, MACHINE_IP)
> if __name__ == '__main__':
>     main(sys.argv)
> {code}
> Finally while the machine is running you do the following:
> 1. Launch an extra agent:
> {noformat}
> sudo ./mesos-agent.sh \
>      --work_dir=/tmp/$USER/mesos/agent-2 \
>      --log_dir=/tmp/$USER/mesos/agent-2/log \
>      --containerizers=mesos,docker \
>      --port=5152 \
>      --master=${MASTER_IP}:5050 \
>      --authenticate_http_readwrite \
>      --http_authenticators=basic \
>      --acls=$HOME/testing/acls.json \
>      --http_credentials=$HOME/testing/credentials.txt
> {noformat}
> 2. Teardown the agent.
> 3. Launch a framework and task:
> {noformat}
> ./src/mesos-execute --master=${MASTER_IP}:5050 --command='while true; do echo "Hello
World"; sleep 5; done;' --resources="cpus:1;mem:128;disk:32;ports:[31002-31003]" --name=hello-discovery
--principal=hal-9000 --secret=hal-9000
> {noformat}
> 4. Teardown the framework by killing the program {{mesos-execute}}.
> 5. Repeat (1) and (2).
> The output of the script is the following:
> {noformat}
> got the following events "SUBSCRIBED", "FRAMEWORK_ADDED", "TASK_ADDED", "TASK_UPDATED",
"TASK_UPDATED", "FRAMEWORK_REMOVED", "AGENT_ADDED"
> It wasn't supposed to get "FRAMEWORK_ADDED"
> It wasn't supposed to get "FRAMEWORK_REMOVED"
> {noformat}
> The last two lines should not appear.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message