Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 17ECA200CE6 for ; Wed, 2 Aug 2017 01:03:06 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 1666C1682D9; Tue, 1 Aug 2017 23:03:06 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 5BA381682D8 for ; Wed, 2 Aug 2017 01:03:05 +0200 (CEST) Received: (qmail 94500 invoked by uid 500); 1 Aug 2017 23:03:04 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 94489 invoked by uid 99); 1 Aug 2017 23:03:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Aug 2017 23:03:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id E7C69C122C for ; Tue, 1 Aug 2017 23:03:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.202 X-Spam-Level: X-Spam-Status: No, score=-99.202 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id g55h4lgWxzIy for ; Tue, 1 Aug 2017 23:03:03 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 7E3225FC43 for ; Tue, 1 Aug 2017 23:03:02 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id AA31CE0E1B for ; Tue, 1 Aug 2017 23:03:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 0D4A72464D for ; Tue, 1 Aug 2017 23:03:01 +0000 (UTC) Date: Tue, 1 Aug 2017 23:03:01 +0000 (UTC) From: "Lukas Majercak (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HDFS-11576) Block recovery will fail indefinitely if recovery time > heartbeat interval MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 01 Aug 2017 23:03:06 -0000 [ https://issues.apache.org/jira/browse/HDFS-11576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16109960#comment-16109960 ] Lukas Majercak commented on HDFS-11576: --------------------------------------- Hi guys, Sorry for the delay again. [~shv], thanks for the feedback, I've : 1. Renamed UnderRecoveryBlocks to PendingRecoveryBlocks 2. Created a config for the timeout multiplier instead (need it to be configurable for the tests), and made it 60x by default 3. Removed one method from the PendingRecoveryBlocks class and changed the names to add/remove/contains > Block recovery will fail indefinitely if recovery time > heartbeat interval > --------------------------------------------------------------------------- > > Key: HDFS-11576 > URL: https://issues.apache.org/jira/browse/HDFS-11576 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, hdfs, namenode > Affects Versions: 2.7.1, 2.7.2, 2.7.3, 3.0.0-alpha1, 3.0.0-alpha2 > Reporter: Lukas Majercak > Assignee: Lukas Majercak > Priority: Critical > Attachments: HDFS-11576.001.patch, HDFS-11576.002.patch, HDFS-11576.003.patch, HDFS-11576.004.patch, HDFS-11576.005.patch, HDFS-11576.006.patch, HDFS-11576.007.patch, HDFS-11576.008, HDFS-11576.repro.patch > > > Block recovery will fail indefinitely if the time to recover a block is always longer than the heartbeat interval. Scenario: > 1. DN sends heartbeat > 2. NN sends a recovery command to DN, recoveryID=X > 3. DN starts recovery > 4. DN sends another heartbeat > 5. NN sends a recovery command to DN, recoveryID=X+1 > 6. DN calls commitBlockSyncronization after succeeding with first recovery to NN, which fails because X < X+1 > ... -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org