Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <1615608320.1204243071126.JavaMail.jira@brutus>
Date: Thu, 28 Feb 2008 15:57:51 -0800 (PST)
From: "Christian Kunz (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-2907) dead datanodes because of
 OutOfMemoryError
In-Reply-To: <1667842456.1204075251034.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12573523#action_12573523 ] 

Christian Kunz commented on HADOOP-2907:
----------------------------------------

Every map application streams through about 40-60 DFS files and writes directly to 2 DFS files (besides sending data to map-reduce framework). None of the maps accesses the same data.

Every reduce application writes to about 70 DFS files.

Keep in mind, that the datanodes ran out of memory already during the map phase.

This access pattern did not change for a while now. But we started to see datanodes running out of memory with about nightly build #810.

I am surprised about the high number of connections. Are some of them stale?

> dead datanodes because of OutOfMemoryError
> ------------------------------------------
>
>                 Key: HADOOP-2907
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2907
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Christian Kunz
>
> We see more dead datanodes than in previous releases. The common exception is found in the out file:
> Exception in thread "org.apache.hadoop.dfs.DataBlockScanner@18166e5" java.lang.OutOfMemoryError: Java heap space
> Exception in thread "DataNode: [dfs.data.dir-value]" java.lang.OutOfMemoryError: Java heap space

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.