From hadoop-dev-return-1273-apmail-lucene-hadoop-dev-archive=lucene.apache.org@lucene.apache.org Fri May 05 18:48:06 2006 Return-Path: Delivered-To: apmail-lucene-hadoop-dev-archive@locus.apache.org Received: (qmail 6179 invoked from network); 5 May 2006 18:48:06 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 5 May 2006 18:48:06 -0000 Received: (qmail 96043 invoked by uid 500); 5 May 2006 18:48:05 -0000 Delivered-To: apmail-lucene-hadoop-dev-archive@lucene.apache.org Received: (qmail 96023 invoked by uid 500); 5 May 2006 18:48:05 -0000 Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-dev@lucene.apache.org Delivered-To: mailing list hadoop-dev@lucene.apache.org Received: (qmail 96014 invoked by uid 99); 5 May 2006 18:48:05 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 May 2006 11:48:05 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [209.237.227.198] (HELO brutus.apache.org) (209.237.227.198) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 May 2006 11:48:04 -0700 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 9E1A67142BB for ; Fri, 5 May 2006 18:47:28 +0000 (GMT) Message-ID: <19352265.1146854848644.JavaMail.jira@brutus> Date: Fri, 5 May 2006 18:47:28 +0000 (GMT+00:00) From: "Hairong Kuang (JIRA)" To: hadoop-dev@lucene.apache.org Subject: [jira] Commented: (HADOOP-96) name server should log decisions that affect data: block creation, removal, replication In-Reply-To: <949326300.1142961002892.JavaMail.jira@ajax> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N [ http://issues.apache.org/jira/browse/HADOOP-96?page=comments#action_12378099 ] Hairong Kuang commented on HADOOP-96: ------------------------------------- Hi Doug, Thanks for commiting this patch. The reason that I removed the line 'cd $HADOOP_HOME' is that I had difficulty starting hadoop from any directory except for "HADOOP_HOME". In my configuration, the log dir & pid dir are relative to "HADOOP_HOME". But "HADOOP_HOME" is relative to the current directory ".". If the script changes the current directory, it is not able to get the log dir and pid dir right. An alternative fix is to set HADOOP_HOME to be its absolute path. > name server should log decisions that affect data: block creation, removal, replication > --------------------------------------------------------------------------------------- > > Key: HADOOP-96 > URL: http://issues.apache.org/jira/browse/HADOOP-96 > Project: Hadoop > Type: Improvement > Components: dfs > Versions: 0.1.0 > Reporter: Yoram Arnon > Assignee: Hairong Kuang > Priority: Critical > Fix For: 0.2 > Attachments: namenodeLogging.patch, namenodeLogging.patch, namenodeLogging.patch > > currently, there's no way to analyze and debug DFS errors where blocks disapear. > name server should log its decisions that affect data, including block creation, removal, replication: > - block created, assigned to datanodes A, B, ... > - datanode A dead, block underreplicated(1), replicating to datanode C > - datanode B dead, block underreplicated(2), replicating to datanode D > - datanode A alive, block overreplicated, removing from datanode D > - block from datanodes C, D, ... > that will enable me to track down, two weeks later, a block that's missing from a file, and to debug the name server. > extra credit: > - rotate log file, as it might grow large > - make this behaviour optional/configurable -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira