hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-1770) have option to limit stderr output from user scripts in hadoop streaming
Date Fri, 18 Jul 2014 22:16:17 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Allen Wittenauer resolved MAPREDUCE-1770.
-----------------------------------------

    Resolution: Fixed

Log limits were implemented generically eons ago.

> have option to limit stderr output from user scripts in hadoop streaming
> ------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1770
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1770
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Joydeep Sen Sarma
>
> People often echo to stderr to debug their programs.  We have had cases where someone
inadvertently dumped large amounts of data to stderr. this created large userlog directories
and caused quite a few mapred slaves to become dysfunctional (logs partition was full).
> Hence looking for a defensive measure against such errant scripts. We are currently putting
in a limit on the number of bytes from the script's stderr that are forwarded to System.err.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message