hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrey Stepachev (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3689) Add support for variable length block
Date Wed, 20 Aug 2014 22:29:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14104713#comment-14104713

Andrey Stepachev commented on HDFS-3689:

[~tlipcon], here can be full sparse support, or sparse semantics for variable length blocks.
idea was to make sparse hdfs files. hdfs block will carry additional field, which will denote
amount of real data written to it.
on reading side client code will be able to:
a) query exactly how many data available, and effectively skip zeros.
b) if this data will read client, which don't bother on zeros client side code will generate
zeros locally.

But idea to use sparse files for block storage is good by its own. Especially in case of immutable

> Add support for variable length block
> -------------------------------------
>                 Key: HDFS-3689
>                 URL: https://issues.apache.org/jira/browse/HDFS-3689
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: datanode, hdfs-client, namenode
>    Affects Versions: 3.0.0
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: HDFS-3689.000.patch, HDFS-3689.001.patch
> Currently HDFS supports fixed length blocks. Supporting variable length block will allow
new use cases and features to be built on top of HDFS. 

This message was sent by Atlassian JIRA

View raw message