hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-12534) Provide logical BlockLocations for EC files for better split calculation
Date Sat, 23 Sep 2017 01:20:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-12534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Andrew Wang updated HDFS-12534:
    Target Version/s: 3.0.0  (was: 3.0.0-beta1)

> Provide logical BlockLocations for EC files for better split calculation
> ------------------------------------------------------------------------
>                 Key: HDFS-12534
>                 URL: https://issues.apache.org/jira/browse/HDFS-12534
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: erasure-coding
>    Affects Versions: 3.0.0-beta1
>            Reporter: Andrew Wang
>              Labels: hdfs-ec-3.0-must-do
> I talked to [~vanzin] and [~alex.behm] some more about split calculation with EC. It
turns out HDFS-12222 was resolved prematurely. Applications depend on HDFS BlockLocation to
understand where the split points are. The current scheme of returning one BlockLocation per
block group loses this information.
> We should change this to provide logical blocks. Divide the file length by the block
size and provide suitable BlockLocations to match, with virtual offsets and lengths too.
> I'm not marking this as incompatible, since changing it this way would in fact make it
more compatible from the perspective of applications that are scheduling against replicated
files. Thus, it'd be good for beta1 if possible, but okay for later too.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message