Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Fri, 13 May 2011 22:58:47 +0000 (UTC)
From: "Todd Lipcon (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: 
 <1851618039.11890.1305327527610.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HDFS-1332) When unable to place replicas,
 BlockPlacementPolicy should log reasons nodes were excluded
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/HDFS-1332?page=3Dcom.atlassian.=
jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D13033=
372#comment-13033372 ]=20

Todd Lipcon commented on HDFS-1332:
-----------------------------------

bq. So what=E2=80=99s the catch? You will be keeping one string builder aro=
und for each new thread that ever enters processRecord(). This could potent=
ially end up as lots of string builders ...

In this case, there is a bounded set of IPC handler threads in the NameNode=
. So, each of those will keep around a stringbuilder, which might have a 1K=
B buffer. So, total memory usage might be a few hundred KB.

> When unable to place replicas, BlockPlacementPolicy should log reasons no=
des were excluded
> -------------------------------------------------------------------------=
-----------------
>
>                 Key: HDFS-1332
>                 URL: https://issues.apache.org/jira/browse/HDFS-1332
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>            Reporter: Todd Lipcon
>            Assignee: Ted Yu
>            Priority: Minor
>              Labels: newbie
>             Fix For: 0.23.0
>
>         Attachments: HDFS-1332.patch
>
>
> Whenever the block placement policy determines that a node is not a "good=
 target" it could add the reason for exclusion to a list, and then when we =
log "Not able to place enough replicas" we could say why each node was refu=
sed. This would help new users who are having issues on pseudo-distributed =
(eg because their data dir is on /tmp and /tmp is full). Right now it's ver=
y difficult to figure out the issue.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira