hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-5092) Add support for incremental cache reports
Date Wed, 22 Jan 2014 21:56:53 GMT

     [ https://issues.apache.org/jira/browse/HDFS-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Andrew Wang updated HDFS-5092:

    Labels: caching  (was: )

> Add support for incremental cache reports
> -----------------------------------------
>                 Key: HDFS-5092
>                 URL: https://issues.apache.org/jira/browse/HDFS-5092
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode, namenode
>            Reporter: Colin Patrick McCabe
>            Assignee: Andrew Wang
>            Priority: Minor
>              Labels: caching
> The initial {{cacheReport}} patch at HDFS-5051 does frequent full reports of DN cache
state. Better would be a scheme similar to how block reports are currently done: send incremental
cache reports on every heartbeat (seconds), and full reports on a longer time scale (minutes
to hours). This should reduce network traffic and allow us to make incremental reports even
> As per discussion on HDFS-5051, we should also roll-up the following review comments:
> - Remove gen stamp and length from {{cacheReport}}, unnecessary until we do auto-caching
of appended data
> - Only jitter full cache reports, similar to how full block reports are jittered
> - On DN startup, skip all cache reports until the cache is populated. The NN can just
assume the DN cache is empty in the meantime.

This message was sent by Atlassian JIRA

View raw message