Return-Path: Delivered-To: apmail-hadoop-hdfs-dev-archive@minotaur.apache.org Received: (qmail 73392 invoked from network); 12 May 2010 05:23:03 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 12 May 2010 05:23:03 -0000 Received: (qmail 18693 invoked by uid 500); 12 May 2010 05:23:03 -0000 Delivered-To: apmail-hadoop-hdfs-dev-archive@hadoop.apache.org Received: (qmail 18449 invoked by uid 500); 12 May 2010 05:23:03 -0000 Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-dev@hadoop.apache.org Delivered-To: mailing list hdfs-dev@hadoop.apache.org Received: (qmail 18433 invoked by uid 99); 12 May 2010 05:23:02 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 May 2010 05:23:02 +0000 X-ASF-Spam-Status: No, hits=-1416.2 required=10.0 tests=ALL_TRUSTED,AWL X-Spam-Check-By: apache.org Received: from [140.211.11.22] (HELO thor.apache.org) (140.211.11.22) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 May 2010 05:23:01 +0000 Received: from thor (localhost [127.0.0.1]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o4C5Mf2W016013 for ; Wed, 12 May 2010 05:22:41 GMT Message-ID: <19974435.19461273641761337.JavaMail.jira@thor> Date: Wed, 12 May 2010 01:22:41 -0400 (EDT) From: "dhruba borthakur (JIRA)" To: hdfs-dev@hadoop.apache.org Subject: [jira] Created: (HDFS-1147) Reduce NN startup time by reducing the processing time of block reports MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Reduce NN startup time by reducing the processing time of block reports ----------------------------------------------------------------------- Key: HDFS-1147 URL: https://issues.apache.org/jira/browse/HDFS-1147 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Reporter: dhruba borthakur The NameNode restart times are impacted to a large extent by the processing time of block reports. For a cluster with 150 millions blocks, the block report processing in the NN can take upto 20 minutes or so. The NN is open for business only after it has processed the block reports from most of the datanodes. If we can reduce the processing time of a block report, that will directly reduce the restart time of the NN. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.