Return-Path: Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: (qmail 79371 invoked from network); 14 Aug 2009 00:11:30 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 14 Aug 2009 00:11:30 -0000 Received: (qmail 91084 invoked by uid 500); 14 Aug 2009 00:11:37 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 91031 invoked by uid 500); 14 Aug 2009 00:11:37 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 91021 invoked by uid 99); 14 Aug 2009 00:11:37 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Aug 2009 00:11:37 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Aug 2009 00:11:35 +0000 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id C9704234C004 for ; Thu, 13 Aug 2009 17:11:14 -0700 (PDT) Message-ID: <1963156621.1250208674811.JavaMail.jira@brutus> Date: Thu, 13 Aug 2009 17:11:14 -0700 (PDT) From: "Boris Shkolnik (JIRA)" To: hdfs-issues@hadoop.apache.org Subject: [jira] Updated: (HDFS-457) better handling of volume failure in Data Node storage In-Reply-To: <1817020889.1246388327191.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HDFS-457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boris Shkolnik updated HDFS-457: -------------------------------- Attachment: HDFS-457-2.patch fixed two issues: 1. make sure that blockreport is sent before starting verification 2. delete files in the directory instead of the whole directory (because deleting directory with locked files in it may fail on Windows or on NFS). > better handling of volume failure in Data Node storage > ------------------------------------------------------ > > Key: HDFS-457 > URL: https://issues.apache.org/jira/browse/HDFS-457 > Project: Hadoop HDFS > Issue Type: Bug > Components: data-node > Reporter: Boris Shkolnik > Assignee: Boris Shkolnik > Attachments: HDFS-457-1.patch, HDFS-457-2.patch, HDFS-457-2.patch, HDFS-457.patch > > > Current implementation shuts DataNode down completely when one of the configured volumes of the storage fails. > This is rather wasteful behavior because it decreases utilization (good storage becomes unavailable) and imposes extra load on the system (replication of the blocks from the good volumes). These problems will become even more prominent when we move to mixed (heterogeneous) clusters with many more volumes per Data Node. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.