hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8415) Erasure coding: rotated parity placement (RAID5/6)
Date Thu, 26 Nov 2015 06:51:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028276#comment-15028276

Zhe Zhang commented on HDFS-8415:

[~szetszwo] I think the upsides are similar to the upsides of RAID5 (rotating parity) compared
to RAID3. Quoting from Wikipedia:
In comparison to RAID 4, RAID 5's distributed parity evens out the stress of a dedicated parity
disk among all RAID members. Additionally, read performance is increased since all RAID members
participate in serving of the read requests.
If we rotate the role of data / parity blocks for each stripe of cells, then we'll be reading
from {{DN0, DN1, DN2, DN3, DN4, DN5}} for the 1st stripe, and {{DN1, DN2, DN3, DN4, DN5, DN6}}
for the 2nd stripe. In theory this could leverage 9 disk spindles in parallel, instead of
6 without rotation.

The downside is the added complexity.

> Erasure coding: rotated parity placement (RAID5/6)
> --------------------------------------------------
>                 Key: HDFS-8415
>                 URL: https://issues.apache.org/jira/browse/HDFS-8415
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
> Our current implementation uses fixed internal blocks for parity data, similar to RAID3.
Rotated parity placement like RAID5/6 has interesting tradeoffs that we should investigate.

This message was sent by Atlassian JIRA

View raw message