Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id C0640200C21 for ; Mon, 20 Feb 2017 18:48:52 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id BEECD160B73; Mon, 20 Feb 2017 17:48:52 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 14F00160B58 for ; Mon, 20 Feb 2017 18:48:51 +0100 (CET) Received: (qmail 28707 invoked by uid 500); 20 Feb 2017 17:48:51 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 28696 invoked by uid 99); 20 Feb 2017 17:48:51 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 20 Feb 2017 17:48:51 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id CBEDF1A05A1 for ; Mon, 20 Feb 2017 17:48:50 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -1.199 X-Spam-Level: X-Spam-Status: No, score=-1.199 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id EiEMG3Hv-f-O for ; Mon, 20 Feb 2017 17:48:50 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id DC7525F5FD for ; Mon, 20 Feb 2017 17:48:49 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 595D5E03A1 for ; Mon, 20 Feb 2017 17:48:44 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 162B02411F for ; Mon, 20 Feb 2017 17:48:44 +0000 (UTC) Date: Mon, 20 Feb 2017 17:48:44 +0000 (UTC) From: "Elek, Marton (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HDFS-11295) Check storage remaining instead of node remaining in BlockPlacementPolicyDefault.chooseReplicaToDelete() MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Mon, 20 Feb 2017 17:48:52 -0000 [ https://issues.apache.org/jira/browse/HDFS-11295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elek, Marton updated HDFS-11295: -------------------------------- Attachment: HDFS-11295.jpg > Check storage remaining instead of node remaining in BlockPlacementPolicyDefault.chooseReplicaToDelete() > -------------------------------------------------------------------------------------------------------- > > Key: HDFS-11295 > URL: https://issues.apache.org/jira/browse/HDFS-11295 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode > Affects Versions: 2.7.1 > Reporter: Xiao Liang > Assignee: Elek, Marton > Attachments: HDFS-11295.001.patch, HDFS-11295.002.patch, HDFS-11295.003.patch, HDFS-11295.jpg > > > Currently in BlockPlacementPolicyDefault.chooseReplicaToDelete() the logic for choosing replica to delete is to pick the node with the least free space(node.getRemaining()), if all hearbeats are within the tolerable heartbeat interval. > However, a node may have multiple storages and node.getRemaining() is a sum of the remainings of them, if free space of the storage with the block to be delete is low, free space of the node could still be high due to other storages of the node, finally the storage chosen may not be the storage with least free space. > So using storage.getRemaining() to choose a storage with least free space for choosing replica to delete may be a better way to balance storage usage. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org