hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1332) When unable to place replicas, BlockPlacementPolicy should log reasons nodes were excluded
Date Mon, 16 May 2011 22:53:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13034403#comment-13034403
] 

Tsz Wo (Nicholas), SZE commented on HDFS-1332:
----------------------------------------------

Thanks Ted, the new patch is much better.  Some comments.
- The following should be removed.
{code}
+    @Override
+    public StringBuilder get() {
+      StringBuilder b = super.get();
+      return b;
+    }
{code}

- {{FSNamesystem.LOG.isDebugEnabled()}} is a dynamic property.  So the following does not
work.
{code}
+  static boolean isDebugEnabled = FSNamesystem.LOG.isDebugEnabled();
{code}

- How about change {{"\n" + e.getMessage()}} to {{"; " + e);}}?
{code}
+               + numOfReplicas + " to reach " + totalReplicasExpected + "\n"
+               + e.getMessage());
{code}

- The first two lines below should be moved inside the if-statement.
{code}
+      String failingReason = "Node "+NodeBase.getPath(node)+
+            " is not chosen because the rack has too many chosen nodes";
       if(FSNamesystem.LOG.isDebugEnabled()) {
-        FSNamesystem.LOG.debug("Node "+NodeBase.getPath(node)+
-            " is not chosen because the rack has too many chosen nodes");
+        FSNamesystem.LOG.debug(failingReason);
+        threadLocalBuilder.get().append(node.toString()).append(": ")
+          .append(failingReason).append(" ");
       }
{code}

- I think it is hard to create new tests for this.  Could you test it manually and post some
sample log messages?

> When unable to place replicas, BlockPlacementPolicy should log reasons nodes were excluded
> ------------------------------------------------------------------------------------------
>
>                 Key: HDFS-1332
>                 URL: https://issues.apache.org/jira/browse/HDFS-1332
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>            Reporter: Todd Lipcon
>            Assignee: Ted Yu
>            Priority: Minor
>              Labels: newbie
>             Fix For: 0.23.0
>
>         Attachments: HDFS-1332-concise.patch
>
>
> Whenever the block placement policy determines that a node is not a "good target" it
could add the reason for exclusion to a list, and then when we log "Not able to place enough
replicas" we could say why each node was refused. This would help new users who are having
issues on pseudo-distributed (eg because their data dir is on /tmp and /tmp is full). Right
now it's very difficult to figure out the issue.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message