hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3398) Log Aggregation broken in Secure Mode
Date Tue, 15 Nov 2011 05:25:52 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150245#comment-13150245

Siddharth Seth commented on MAPREDUCE-3398:

Some options to fix this.
1. Change the NM and RM interaction - to have the RM wait for all NMs to notify the RM about
contianer / app completion (or timeout). This would pretty much be a reversal of the current
mechanism where the RM is responsible for app completion. Would be risky - the RM should not
have to wait for NMs.
2. Have the NMs run as a proxy user - has a lot of operational issues, and possibly security
3. Aggregate logs as the NM user - this breaks the CLI.
4. The NMs asks for application token renewal while log aggregation is still to complete.
This seems like the best option. Credit to Vinod for suggesting it.
> Log Aggregation broken in Secure Mode
> -------------------------------------
>                 Key: MAPREDUCE-3398
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3398
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2, nodemanager
>    Affects Versions: 0.23.0
>            Reporter: Siddharth Seth
>            Assignee: Siddharth Seth
>            Priority: Critical
> Log aggregation in secure mode does not work with MAPREDUCE-2977. The nodemanager relies
on the users credentials to write out logs to HDFS. These credentials are currently cancelled
once a job completes, before the NM can write out the logs.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message