hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tomasz Nykiel (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-2362) More Improvements on NameNode Scalability
Date Thu, 20 Oct 2011 04:12:11 GMT

     [ https://issues.apache.org/jira/browse/HDFS-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tomasz Nykiel updated HDFS-2362:
--------------------------------

    Description: 
This jira acts as an umbrella jira to track all the improvements we've done recently to improve
Namenode's performance, responsiveness, and hence scalability. Those improvements include:
1. Incremental block reports (HDFS-395)
2. BlockManager.reportDiff optimization for processing block reports (HDFS-2477)
3. Upgradable lock to allow simutaleous read operation while reportDiff is in progress in
processing block reports
4. More CPU efficient data structure for under-replicated/over-replicated/invalidate blocks
(HDFS-2476)
5. Increase granularity of write operations in ReplicationMonitor thus reducing contention
for write lock
6. Support variable block sizes
7. Release RPC handlers while waiting for edit log is synced to disk
8. Reduce network traffic pressure to the master rack where NN is located by lowering read
priority of the replicas on the rack
9. A standalone KeepAlive heartbeat thread
10. Reduce Multiple traversals of path directory to one for most namespace manipulations
11. Move logging out of write lock section.



  was:
This jira acts as an umbrella jira to track all the improvements we've done recently to improve
Namenode's performance, responsiveness, and hence scalability. Those improvements include:
1. Incremental block reports (HDFS-395)
2. Upgradable lock to allow simutaleous read operation while reportDiff is in progress in
processing block reports
3. More CPU efficient data structure for under-replicated/over-replicated/invalidate blocks
(HDFS-2476)
4. Increase granularity of write operations in ReplicationMonitor thus reducing contention
for write lock
5. Support variable block sizes
6. Release RPC handlers while waiting for edit log is synced to disk
7. Reduce network traffic pressure to the master rack where NN is located by lowering read
priority of the replicas on the rack
8. A standalone KeepAlive heartbeat thread
9. Reduce Multiple traversals of path directory to one for most namespace manipulations
10. Move logging out of write lock section.


    
> More Improvements on NameNode Scalability
> -----------------------------------------
>
>                 Key: HDFS-2362
>                 URL: https://issues.apache.org/jira/browse/HDFS-2362
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>            Reporter: Hairong Kuang
>
> This jira acts as an umbrella jira to track all the improvements we've done recently
to improve Namenode's performance, responsiveness, and hence scalability. Those improvements
include:
> 1. Incremental block reports (HDFS-395)
> 2. BlockManager.reportDiff optimization for processing block reports (HDFS-2477)
> 3. Upgradable lock to allow simutaleous read operation while reportDiff is in progress
in processing block reports
> 4. More CPU efficient data structure for under-replicated/over-replicated/invalidate
blocks (HDFS-2476)
> 5. Increase granularity of write operations in ReplicationMonitor thus reducing contention
for write lock
> 6. Support variable block sizes
> 7. Release RPC handlers while waiting for edit log is synced to disk
> 8. Reduce network traffic pressure to the master rack where NN is located by lowering
read priority of the replicas on the rack
> 9. A standalone KeepAlive heartbeat thread
> 10. Reduce Multiple traversals of path directory to one for most namespace manipulations
> 11. Move logging out of write lock section.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message