Return-Path: Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: (qmail 51363 invoked from network); 17 Mar 2011 23:37:51 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 17 Mar 2011 23:37:51 -0000 Received: (qmail 39213 invoked by uid 500); 17 Mar 2011 23:37:51 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 39174 invoked by uid 500); 17 Mar 2011 23:37:51 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 39165 invoked by uid 99); 17 Mar 2011 23:37:51 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 17 Mar 2011 23:37:51 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED,T_RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 17 Mar 2011 23:37:50 +0000 Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116]) by hel.zones.apache.org (Postfix) with ESMTP id A00A93AE237 for ; Thu, 17 Mar 2011 23:37:29 +0000 (UTC) Date: Thu, 17 Mar 2011 23:37:29 +0000 (UTC) From: "Todd Lipcon (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: <1836701301.10474.1300405049652.JavaMail.tomcat@hel.zones.apache.org> In-Reply-To: <992817949.10119.1300396229628.JavaMail.tomcat@hel.zones.apache.org> Subject: [jira] Commented: (HDFS-1766) Datanode is marked dead, but datanode process is alive and verifying blocks MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-1766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008220#comment-13008220 ] Todd Lipcon commented on HDFS-1766: ----------------------------------- Yep, the catch (Exception e) > Datanode is marked dead, but datanode process is alive and verifying blocks > --------------------------------------------------------------------------- > > Key: HDFS-1766 > URL: https://issues.apache.org/jira/browse/HDFS-1766 > Project: Hadoop HDFS > Issue Type: Bug > Components: data-node > Affects Versions: 0.23.0 > Reporter: Hairong Kuang > Assignee: Hairong Kuang > Fix For: 0.23.0 > > Attachments: killDN.patch > > > We have a datanode marked dead in the namenode, and it is not taking any traffic. But it is verifying blocks continuously, so the DataNode process is definitely not dead. Jstack shows that the main thread and the offerService thread are gone but the JVM stuck at waiting for other threads to die. It seems to me that the offerService thread has died abnormally, for example, by a runtime exception and it did not shut down other threads before exiting. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira