Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 22 Feb 2017 14:58:44 +0000 (UTC)
From: "Ewan Higgs (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12938986.1455327519000.31343.1487775524257@Atlassian.JIRA>
In-Reply-To: <JIRA.12938986.1455327519000@Atlassian.JIRA>
References: <JIRA.12938986.1455327519000@Atlassian.JIRA> <JIRA.12938986.1455327519155@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HDFS-9807) Add an optional StorageID to writes
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 22 Feb 2017 14:58:48 -0000


    [ https://issues.apache.org/jira/browse/HDFS-9807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878382#comment-15878382 ] 

Ewan Higgs commented on HDFS-9807:
----------------------------------

The most obvious way to implement this is for the NN to choose a volume and have the DN do what it's told. But the NN doesn't choose a precise volume. It only uses the first Storage of a given StorageType (see {{DataNodeDescriptor.chooseStorage4Block}}). This means that all blocks would only be written to the first volume instead of spread around.

We can either change the algorithm in {{DataNodeDescriptor}} to choose a better selection. Or, [~chris.douglas], did you have a different policy in mind when you wrote the ticket? Perhaps passing the Storage ID into {{FsVolumeList.getNextVolume}} and ignoring it if the {{StorageType}} is not {{PROVIDED}}?

> Add an optional StorageID to writes
> -----------------------------------
>
>                 Key: HDFS-9807
>                 URL: https://issues.apache.org/jira/browse/HDFS-9807
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Chris Douglas
>            Assignee: Ewan Higgs
>
> The {{BlockPlacementPolicy}} considers specific storages, but when the replica is written the DN {{VolumeChoosingPolicy}} is unaware of any preference or constraints from other policies affecting placement. This limits heterogeneity to the declared storage types, which are treated as fungible within the target DN. It should be possible to influence or constrain the DN policy to select a particular storage.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org