hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kuhu Shukla (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-6960) Shuffle Handler prints disk error stack traces for every read failure.
Date Mon, 18 Sep 2017 20:45:00 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kuhu Shukla updated MAPREDUCE-6960:
-----------------------------------
    Attachment: MAPREDUCE-6960.001.patch

v1 patch that logs the exception without the stack trace only in the case of DiskErrorException.
We can do the same for all IOExceptions but that would make any other kinds of IOException
debugging difficult.

> Shuffle Handler prints disk error stack traces for every read failure.
> ----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6960
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6960
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Kuhu Shukla
>            Assignee: Kuhu Shukla
>         Attachments: MAPREDUCE-6960.001.patch
>
>
> {code}
>  } catch (IOException e) {
>           LOG.error("Shuffle error :", e);
> {code}
> In cases where the read from a disk fails and throws a DiskErrorException, the shuffle
handler prints the entire stack trace for each and every one of the failures causing the nodemanager
logs to quickly fill up the disk. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message