mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jie Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MESOS-7675) Isolate network ports.
Date Tue, 22 Aug 2017 20:57:00 GMT

    [ https://issues.apache.org/jira/browse/MESOS-7675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16137399#comment-16137399
] 

Jie Yu commented on MESOS-7675:
-------------------------------

commit 40906e31a44848b826a94fbcde668661fe2028d4
Author: James Peach <jpeach@apache.org>
Date:   Tue Aug 22 13:37:55 2017 -0700

    Moved the libnl3 configure checks into a macro.

    Since the `network/ports` isolator will depend on libnl3, move those
    checks into a separate macro so that we can call it again when we
    add a configure option to enable it.

    Review: https://reviews.apache.org/r/60902/

commit f7a38d7b1b1de6d52d5134364f257679de69505b
Author: James Peach <jpeach@apache.org>
Date:   Tue Aug 22 13:37:51 2017 -0700

    Used common port range interval code in the port_mapping isolator.

    Switched the port_mapping isolator over to start using the
    common values code to parse port ranges into an IntervalSet.

    Review: https://reviews.apache.org/r/61538/

commit daa77c66cd211b2f33c4fe4bd3dd0aa7f78430a8
Author: James Peach <jpeach@apache.org>
Date:   Tue Aug 22 13:37:49 2017 -0700

    Added IntervalSet to Ranges conversion helpers.

    Added a new `common/values.hpp` header file to expose IntervalSet
    to Ranges conversion helper declarations.

    The most common use of Range resources is for representing network
    ports. Since ports are bounded to uint16_t it is awkward to store
    them in a IntervalSet<uint64_t>. To address this, convert the
    IntervalSet helpers to templates so that we can convert between
    IntervalSets of the appropriate type.

    Review: https://reviews.apache.org/r/60836/

Last login: Sun Jul 30 16:55:41 on console
Jies-MacBook-Pro:~ jie$ tmux











































-- Installing: /Users/jie/workspace/csi-spec/build/3rdparty/include/google/protobuf/source_context.proto
-- Installing: /Users/jie/workspace/csi-spec/build/3rdparty/include/google/protobuf/struct.proto
-- Installing: /Users/jie/workspace/csi-spec/build/3rdparty/include/google/protobuf/timestamp.proto
-- Installing: /Users/jie/workspace/csi-spec/build/3rdparty/include/google/protobuf/type.proto
-- Installing: /Users/jie/workspace/csi-spec/build/3rdparty/include/google/protobuf/wrappers.proto
-- Installing: /Users/jie/workspace/csi-spec/build/3rdparty/include/google/protobuf/compiler/plugin.proto
-- Installing: /Users/jie/workspace/csi-spec/build/3rdparty/include/google/protobuf/compiler/profile.proto
-- Installing: /Users/jie/workspace/csi-spec/build/3rdparty/lib/cmake/protobuf/protobuf-targets.cmake
-- Installing: /Users/jie/workspace/csi-spec/build/3rdparty/lib/cmake/protobuf/protobuf-targets-noconfig.cmake
-- Up-to-date: /Users/jie/workspace/csi-spec/build/3rdparty/lib/cmake/protobuf
-- Installing: /Users/jie/workspace/csi-spec/build/3rdparty/lib/cmake/protobuf/protobuf-config-version.cmake
-- Installing: /Users/jie/workspace/csi-spec/build/3rdparty/lib/cmake/protobuf/protobuf-config.cmake
-- Installing: /Users/jie/workspace/csi-spec/build/3rdparty/lib/cmake/protobuf/protobuf-module.cmake
-- Installing: /Users/jie/workspace/csi-spec/build/3rdparty/lib/cmake/protobuf/protobuf-options.cmake
[ 66%] Completed 'protobuf-3.3.2'
[ 66%] Built target protobuf-3.3.2
Scanning dependencies of target protoc
[ 66%] Built target protoc
[ 75%] Generating csi.proto
[ 83%] Running C++ protocol buffer compiler
Scanning dependencies of target csi
[ 91%] Building CXX object CMakeFiles/csi.dir/csi.pb.cc.o
[100%] Linking CXX static library libcsi.a
[100%] Built target csi
bash-3.2$ make
[ 66%] Built target protobuf-3.3.2
[ 66%] Built target protoc
[100%] Built target csi
bash-3.2$ make
[ 66%] Built target protobuf-3.3.2
[ 66%] Built target protoc
[100%] Built target csi
bash-3.2$ exit

bash-3.2$ cmake ..
-- Could NOT find Protobuf (missing: Protobuf_LIBRARIES Protobuf_INCLUDE_DIR)
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/jie/workspace/csi-spec/build
bash-3.2$ make
[ 66%] Built target protobuf-3.3.2
[ 66%] Built target protoc
[100%] Built target csi
bash-3.2$
0:vim  1:bash- 2:bash* 3:bash  4:bash                                                    
                             "Jies-MacBook-Pro.loca" 13:32 03-Aug-17                     
                                                                "Jies-MacBook-Pro.loca" 18:09
02-Aug-17
  [Restored Aug 3, 2017, 2:17:52 PM]
Last login: Thu Aug  3 14:17:40 on console
Jies-MacBook-Pro:~ jie$ ls
Applications   Desktop        Documents      Downloads      Dropbox        Google Drive  
Library        Movies         Music          Pictures       Public         VirtualBox VMs
workspace
Jies-MacBook-Pro:~ jie$ tmux

    Review: https://reviews.apache.org/r/60836/

commit 16cbd203bf5626ec1377a3b4ce772ce6dbaeb78a
Author: James Peach <jpeach@apache.org>
Date:   Tue Aug 22 13:37:45 2017 -0700

    Use a consistent preprocessor check for ENABLE_PORT_MAPPING_ISOLATOR.

    There's no need to also check for Windows when testing the
    ENABLE_PORT_MAPPING_ISOLATOR feature macro, because
    ENABLE_PORT_MAPPING_ISOLATOR requires libnl3, which is a
    Linux-specific features.

    Review: https://reviews.apache.org/r/60901/

commit 2505b77ff397f81c615d96007665e1396248f355
Author: James Peach <jpeach@apache.org>
Date:   Tue Aug 22 13:37:42 2017 -0700

    Refactored isolator dependency checking.

    Refactored the isolator dependency checks to immediately tokenize
    the isolator string, which makes it easier to check various consistency
    conditions.

    Review: https://reviews.apache.org/r/60764/

commit 092e4c5f1ab3753a7ba1dccaeb88b2fb58c0a3e6
Author: James Peach <jpeach@apache.org>
Date:   Tue Aug 22 13:37:40 2017 -0700

    Exposed LinuxLauncher cgroups helper.

    Expose the LinuxLauncher cgroups helper to generate the cgroups
    path from a container ID. This is needed by the `network/ports`
    isolator.

    Review: https://reviews.apache.org/r/60494/

commit 5fb4281aae4b350ca20e9fe563c89d6a60763e2e
Author: James Peach <jpeach@apache.org>
Date:   Tue Aug 22 13:37:38 2017 -0700

    Removed diagnostic socket IPv4 assumptions.

    Don't assume the diagnostic socket only returns IPv4 addresses.

    Review: https://reviews.apache.org/r/60493/

commit 9128060cf4e6fd00d9cd3a45070e2a3cae3e7b66
Author: James Peach <jpeach@apache.org>
Date:   Tue Aug 22 13:37:35 2017 -0700

    Captured the inode when scanning for sockets.

    Capture the socket inode in the diagnosis Info when we use netlink
    to enumerate the open sockets. This can be used to identify which
    process(es) have the socket open.

    Review: https://reviews.apache.org/r/60491/

> Isolate network ports.
> ----------------------
>
>                 Key: MESOS-7675
>                 URL: https://issues.apache.org/jira/browse/MESOS-7675
>             Project: Mesos
>          Issue Type: Improvement
>          Components: agent
>            Reporter: James Peach
>            Assignee: James Peach
>            Priority: Minor
>
> If a task uses network ports, there is no isolator that can enforce that it only listens
on the ports that it has resources for. Implement a ports isolator that can limit tasks to
listen only on allocated TCP ports.
> Roughly, the algorithm for this follows what standard tools like {{lsof}} and {{ss}}
do.
> * Find all the listening TCP sockets (using netlink)
> * Index the sockets by their node (from the netlink information)
> * Find all the open sockets on the system (by scanning {{/proc/\*/fd/\*}} links)
> * For each open socket, check whether its node (given in the link target) in the set
of listen sockets that we scanned
> * If the socket is a listening socket and the corresponding PID is in the task, send
a resource limitation for the task
> Matching pids to tasks depends on using cgroup isolation, otherwise we would have to
build a full process tree, which would be nice to avoid.
> Scanning all the open sockets can be avoided by using the {{net_cls}} isolator with kernel
+ libnl3 patches to publish the socket classid when we find the listening socket.
> Design Doc: https://docs.google.com/document/d/1BGmANq8IW-H4-YVUlpdf6qZFTZnDe-OKAY_e7uNp7LA
> Kernel Patch: http://marc.info/?l=linux-kernel&m=150293015025396&w=2



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message