hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Virajith Jalaparti (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9810) Allow support for more than one block replica per datanode
Date Sat, 13 Feb 2016 21:16:18 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15146194#comment-15146194

Virajith Jalaparti commented on HDFS-9810:

Our use case is driven by HDFS-9806 -- the idea is when a "provided" block is fetched by a
datanode, the datanode can cache it on local storage media (disk, SSD etc.) so that future
reads of the data are served from it. When the block is cached, the datanode has to maintain
information for two replicas (in FSDatasetImpl), one for the local replica and the other for
the "provided" replica. With this, if the local replica is lost, it can still be accessed
from the provided replica.

This is a modification in the FSDatasetImpl to enable the possibility of a datanode storing
multiple replicas for a block. We are not proposing to change the BlockPlacementPolicyDefault,
which determines how the replicas are placed. The BlockPlacementPolicyDefault will still work
as it works today and hence, the failure model will remain the same. 

> Allow support for more than one block replica per datanode
> ----------------------------------------------------------
>                 Key: HDFS-9810
>                 URL: https://issues.apache.org/jira/browse/HDFS-9810
>             Project: Hadoop HDFS
>          Issue Type: Task
>          Components: datanode
>            Reporter: Virajith Jalaparti
> Datanodes report and store only one replica of each block. It should be possible to store
multiple replicas among its different configured storage types, particularly to support non-durable
media and remote storage.

This message was sent by Atlassian JIRA

View raw message