hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard Chen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-5727) introduce a self-maintaining io queue handling mechanism
Date Wed, 08 Jan 2014 13:51:57 GMT

     [ https://issues.apache.org/jira/browse/HDFS-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Richard Chen updated HDFS-5727:

    Summary: introduce a self-maintaining io queue handling mechanism  (was: introduce a self-maintain
io queue handling mechanism)

> introduce a self-maintaining io queue handling mechanism
> --------------------------------------------------------
>                 Key: HDFS-5727
>                 URL: https://issues.apache.org/jira/browse/HDFS-5727
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: datanode
>    Affects Versions: 3.0.0
>            Reporter: Liang Xie
>            Assignee: Liang Xie
> Currently the datanode read/write SLA is difficult to be guaranteed for HBase online
requirement. One of major reasons is we don't support io priority or io request reorder inside
> I propose introducing a self-maintain io queue mechanism to handle io request priority.
Imaging there're lots of concurrent read/write requests from HBase side, and a background
datanode block scanner is running(default is every 21 days, IIRC) just in time, then the HBase
read/write 99% or 99.9% percentile latency would be vulnerable despite we have a bg thread
> the reorder stuff i have not thought clearly enough, but definitely the reorder in the
queue in the app side would beat the currently relying OS's io queue merge.

This message was sent by Atlassian JIRA

View raw message