hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <ey...@apache.org>
Subject Re: [DISCUSS] Feature Branch Merge and Security Audits
Date Sun, 22 Oct 2017 15:38:54 GMT
Looks good and +1 for markdown documentations to provide per release
specific information.

On Sat, Oct 21, 2017 at 8:47 AM, larry mccay <lmccay@apache.org> wrote:

> New Revision...
>
> This revision acknowledges the reality that we often have multiple phases
> of feature lifecycle and that we need to account for each phase.
> It has also been made more generic.
> I have created a Tech Preview Security Audit list and a GA Readiness
> Security Audit list.
> I've also included suggested items into the GA Readiness list.
>
> It has also been suggested that we publish the information as part of docs
> so that the state of such features can be easily determined from these
> pages. We can discuss this aspect as well.
>
> Thoughts?
>
> *Tech Preview Security Audit*
> For features that are being merged without full security model coverage,
> there need to be a base line of assurances that they do not introduce new
> attack vectors in deployments that are from actual releases or even just
> built from trunk.
>
> *1. UIs*
>
> 1.1. Are there new UIs added with this merge?
> 1.2. Are they enabled/accessible by default?
> 1.3. Are they hosted in existing processes or as part of a new
> process/server?
> 1.4. If new process/server, is it launched by default?
>
> *2. APIs*
>
> 2.1. Are there new REST APIs added with this merge?
> 2.2. Are they enabled by default?
> 2.3. Are there RPC based APIs added with this merge?
> 2.4. Are they enabled by default?
>
> *3. Secure Clusters*
>
> 3.1. Is this feature disabled completely in secure deployments?
> 3.2. If not, is there some justification as to why it should be available?
>
> *4. CVEs*
>
> 4.1. Have all dependencies introduced by this merge been checked for known
> issues?
>
>
> ------------------------------------------------------------
> ------------------------------------------------------------
> --------------------------
>
>
> *GA Readiness Security Audit*
> At this point, we are merging full or partial security model
> implementations.
> Let's inventory what is covered by the model at this point and whether
> there are future merges required to be full.
>
> *1. UIs*
>
> 1.1. What sort of validation is being done on any accepted user input?
> (pointers to code would be appreciated)
> 1.2. What explicit protections have been built in for (pointers to code
> would be appreciated):
>   1.2.1. cross site scripting
>   1.2.2. cross site request forgery
>   1.2.3. click jacking (X-Frame-Options)
> 1.3. What sort of authentication is required for access to the UIs?
>   1.3.1. Kerberos
>     1.3.1.1. has TGT renewal been accounted for
>     1.3.1.2. SPNEGO support?
>     1.3.1.3. Delegation token?
>   1.3.2. Proxy User ACL?
> 1.4. What authorization is available for determining who can access what
> capabilities of the UIs for either viewing, modifying data and/or related
> processes?
> 1.5. Is there any input that will ultimately be persisted in configuration
> for executing shell commands or processes?
> 1.6. Do the UIs support the trusted proxy pattern with doas impersonation?
> 1.7. Is there TLS/SSL support?
>
> *2. REST APIs*
>
> 2.1. Do the REST APIs support the trusted proxy pattern with doas
> impersonation capabilities?
> 2.2. What explicit protections have been built in for:
>   2.2.1. cross site scripting (XSS)
>   2.2.2. cross site request forgery (CSRF)
>   2.2.3. XML External Entity (XXE)
> 2.3. What is being used for authentication - Hadoop Auth Module?
> 2.4. Are there separate processes for the HTTP resources (UIs and REST
> endpoints) or are they part of existing processes?
> 2.5. Is there TLS/SSL support?
> 2.6. Are there new CLI commands and/or clients for accessing the REST APIs?
> 2.7. What authorization enforcement points are there within the REST APIs?
>
> *3. Encryption*
>
> 3.1. Is there any support for encryption of persisted data?
> 3.2. If so, is KMS and the hadoop key command used for key management?
> 3.3. KMS interaction with Proxy Users?
>
> *4. Configuration*
>
> 4.1. Are there any passwords or secrets being added to configuration?
> 4.2. If so, are they accessed via Configuration.getPassword() to allow for
> provisioning to credential providers?
> 4.3. Are there any settings that are used to launch docker containers or
> shell out command execution, etc?
>
> *5. HA*
>
> 5.1. Are there provisions for HA?
> 5.2. Are there any single point of failures?
>
> *6. CVEs*
>
> Dependencies need to have been checked for known issues before we merge.
> We don't however want to list any CVEs that have been fixed but not
> released yet.
>
> 6.1. All dependencies checked for CVEs?
>
>
>
>
> On Sat, Oct 21, 2017 at 10:26 AM, larry mccay <lmccay@apache.org> wrote:
>
>> Hi Marton -
>>
>> I don't think there is any denying that it would be great to have such
>> documentation for all of those reasons.
>> If it is a natural extension of getting the checklist information as an
>> assertion of security state when merging then we can certainly include it.
>>
>> I think that backfilling all such information across the project is a
>> different topic altogether and wouldn't want to expand the scope of this
>> discussion in that direction.
>>
>> Thanks for the great thoughts on this!
>>
>> thanks,
>>
>> --larry
>>
>>
>>
>>
>>
>> On Sat, Oct 21, 2017 at 3:00 AM, Elek, Marton <hdp@anzix.net> wrote:
>>
>>>
>>>
>>> On 10/21/2017 02:41 AM, larry mccay wrote:
>>>
>>>>
>>>> "We might want to start a security section for Hadoop wiki for each of
>>>>> the
>>>>> services and components.
>>>>> This helps to track what has been completed."
>>>>>
>>>>
>>>> Do you mean to keep the audit checklist for each service and component
>>>> there?
>>>> Interesting idea, I wonder what sort of maintenance that implies and
>>>> whether we want to take on that burden even though it would be great
>>>> information to have for future reviewers.
>>>>
>>>
>>> I think we should care about the maintenance of the documentation
>>> anyway. We also need to maintain all the other documentations. I think it
>>> could be even part of the generated docs and not the wiki.
>>>
>>> I also suggest to fill this list about the current trunk/3.0 as a first
>>> step.
>>>
>>> 1. It would be a very usefull documentation for the end-users (some
>>> answers could link the existing documentation, it exists, but I am not sure
>>> if all the answers are in the current documentation.)
>>>
>>> 2. It would be a good example who the questions could be answered.
>>>
>>> 3. It would help to check, if something is missing from the list.
>>>
>>> 4. There are future branches where some of the components are not
>>> touched. For example, no web ui or no REST service. A prefilled list could
>>> help to check if the branch doesn't break any old security functionality on
>>> trunk.
>>>
>>> 5. It helps to document the security features in one place. If we have a
>>> list for the existing functionality in the same format, it would be easy to
>>> merge the new documentation of the new features as they will be reported in
>>> the same form. (So it won't be so hard to maintain the list...).
>>>
>>> Marton
>>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message