hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Vasudev (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-5673) [Umbrella] Re-write container-executor to improve security, extensibility, and portability
Date Mon, 26 Sep 2016 14:08:20 GMT
Varun Vasudev created YARN-5673:

             Summary: [Umbrella] Re-write container-executor to improve security, extensibility,
and portability
                 Key: YARN-5673
                 URL: https://issues.apache.org/jira/browse/YARN-5673
             Project: Hadoop YARN
          Issue Type: New Feature
          Components: nodemanager
            Reporter: Varun Vasudev
            Assignee: Varun Vasudev

As YARN adds support for new features that require administrator privileges(such as support
for network throttling and docker), we’ve had to add new capabilities to the container-executor.
This has led to a recognition that the current container-executor security features as well
as the code could be improved. The current code is fragile and it’s hard to add new features
without causing regressions. Some of the improvements that need to be made are -


Currently the container-executor has limited security features. It relies primarily on the
permissions set on the binary but does little additional security beyond that. There are few
outstanding issues today -

- No audit log
- No way to disable features - network throttling and docker support are built in and there’s
no way to turn them off at a container-executor level
- Code can be improved - a lot of the code switches users back and forth in an arbitrary manner
- No input validation - the paths, and files provided at invocation are not validated or required
to be in some specific location
- No signing functionality - there is no way to enforce that the binary was invoked by the
NM and not by any other process

*Code Issues*

The code layout and implementation themselves can be improved. Some issues there are -

- No support for log levels - everything is logged and this can’t be turned on or off
- Extremely long set of invocation parameters(specifically during container launch) which
makes turning features on or off complicated
- Poor test coverage - it’s easy to introduce regressions today due to the lack of a proper
test setup
- Duplicate functionality - there is some amount of code duplication
- Hard to make improvements or add new features due to the issues raised above


 - The container-executor mixes platform dependent APIs with platform independent APIs making
it hard to run it on multiple platforms. Allowing it to run on multiple platforms also improves
the overall code structure .

One option is to improve the existing container-executor, however it might be easier to start
from scratch. That allows existing functionality to be supported until we are ready to switch
to the new code.

This umbrella JIRA is to capture all the work required for the new code. I'm going to work
on a design doc for the changes - any suggestions or improvements are welcome.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

View raw message