hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Omkar Vinit Joshi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-578) NodeManager should use SecureIOUtils for serving and aggregating logs
Date Wed, 29 May 2013 00:40:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13668862#comment-13668862
] 

Omkar Vinit Joshi commented on YARN-578:
----------------------------------------

Thanks Vinod ..

bq. LogValue: If we run into an exception reading one file, like the security exception, the
remaining logs will be completely skipped. Not directly caused by your patch but it gets worse
with the patch. I think we should fix it here.
sounds good... Now making sure that Log aggregator will try to upload all container log files.
In case of an exception it will log an error. (error will also be added into aggregated log
file corresponding to log file entry for future purpose). Added test case to verify this.
(test case for log aggregation for different log files for same container with different ownerships
is bit difficult with current implementation unless we make two append calls). Adding test
for two containers trying to append log files one after the other and then verifying aggregated
log file output.

bq. LogValue: When such security exception happens, we should log it specifically so as to
help debugging later. When we have log-aggregation status exposed to users, we can add a diagnostic
message there.

Fixed.

bq. Also please add a test verifying the above.
added..

bq. ContainerLogsPage: the info log message isn't useful, can be removed.

removed..

bq. Instead of matching messages in the exception block, why not separate the try {} catch
{} block for the SecureIOUtils check?
humm.. thought about it ..was earlier thinking to keep it in same block.... separating try
catch for open call and other read calls.

                
> NodeManager should use SecureIOUtils for serving and aggregating logs
> ---------------------------------------------------------------------
>
>                 Key: YARN-578
>                 URL: https://issues.apache.org/jira/browse/YARN-578
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Omkar Vinit Joshi
>         Attachments: yarn-578-20130426.patch, YARN-578-20130506.patch, YARN-578-20130520.1.patch,
YARN-578-20130520.branch-2.patch, YARN-578-20130520.patch, YARN-578-20130528.patch
>
>
> Log servlets for serving logs and the ShuffleService for serving intermediate outputs
both should use SecureIOUtils for avoiding symlink attacks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message