hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8907) Configurable striping read buffer threhold
Date Thu, 19 Jan 2017 00:01:26 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Andrew Wang updated HDFS-8907:
    Component/s: erasure-coding

> Configurable striping read buffer threhold
> ------------------------------------------
>                 Key: HDFS-8907
>                 URL: https://issues.apache.org/jira/browse/HDFS-8907
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: erasure-coding
>            Reporter: Kai Zheng
>            Assignee: Kai Zheng
> In striping input stream, positional read merges all the possible strips together, while
stateful read reads a strip a time. The former is efficient but may incur too large chunk
buffers for a client to afford, the latter is simple good but can be improved for better throughput.
This would consolidate the both and use a configurable (new or existing) buffer threshold
to control how it goes. Fixed chunk buffers for the read will be allocated accordingly and
reused time and time, as existing stateful read does. The aligned strips to read a time may
be computed against the threshold. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message