hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-8907) Configurable striping read buffer threhold
Date Mon, 17 Aug 2015 13:57:46 GMT
Kai Zheng created HDFS-8907:

             Summary: Configurable striping read buffer threhold
                 Key: HDFS-8907
                 URL: https://issues.apache.org/jira/browse/HDFS-8907
             Project: Hadoop HDFS
          Issue Type: Sub-task
            Reporter: Kai Zheng
            Assignee: Kai Zheng

In striping input stream, positional read merges all the possible strips together, while stateful
read reads a strip a time. The former is efficient but may incur too large chunk buffers for
a client to afford, the latter is simple good but can be improved for better throughput. This
would consolidate the both and use a configurable (new or existing) buffer threshold to control
how it goes. Fixed chunk buffers for the read will be allocated accordingly and reused time
and time. The aligned strips to read a time may be computed against the threshold. 

This message was sent by Atlassian JIRA

View raw message