Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-dev@hadoop.apache.org
Date: Thu, 15 Nov 2012 20:05:13 +0000 (UTC)
From: "Jason Lowe (JIRA)" <jira@apache.org>
To: mapreduce-dev@hadoop.apache.org
Message-ID: <554150771.120811.1353009913447.JavaMail.jiratomcat@arcas>
Subject: [jira] [Created] (MAPREDUCE-4801) ShuffleHandler can generate large
 logs due to prematurely closed channels
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Jason Lowe created MAPREDUCE-4801:
-------------------------------------

             Summary: ShuffleHandler can generate large logs due to prematurely closed channels
                 Key: MAPREDUCE-4801
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4801
             Project: Hadoop Map/Reduce
          Issue Type: Bug
    Affects Versions: 2.0.1-alpha, 0.23.3
            Reporter: Jason Lowe
            Priority: Critical


We ran into an instance where many nodes on a cluster ran out of disk space because the nodemanager logs were huge.  Examining the logs showed many, many shuffle errors due to either ClosedChannelException or IOException from "Connection reset by peer" or "Broken pipe".


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira