hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yang Hao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6220) Provide option to forbid stdout of MapReduce task
Date Fri, 10 Apr 2015 06:25:12 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14489023#comment-14489023

Yang Hao commented on MAPREDUCE-6220:

Hi, thank you for reply so quickly.

I have had an internship in a big company where many groups use MapReduce and Spark. Sometimes
newers may use a lot of stdout or misuse another lib that may print a lot of standout. There
are two or three times some MapReduce job produce more than 1TB log. Even if we tells them
not use stdout, but it stills happen. Then we decide to forbid the stdout and stderr. Following
your suggestion, we just forbid the stdout. And our group also forbid the stdout for Spark
on YARN.

> Provide option  to forbid stdout of MapReduce task
> --------------------------------------------------
>                 Key: MAPREDUCE-6220
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6220
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: mrv2
>            Reporter: Yang Hao
>            Assignee: Yang Hao
>         Attachments: MAPREDUCE-6220.patch, MAPREDUCE-6220.v2.patch
> System.out is a ugly way to print log, and many times it would do harm to Hadoop cluster.
So we can provide an option to forbid it

This message was sent by Atlassian JIRA

View raw message