hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5535) Umbrella jira for improved HDFS rolling upgrades
Date Wed, 15 Jan 2014 17:37:24 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13872303#comment-13872303
] 

Kihwal Lee commented on HDFS-5535:
----------------------------------

{quote}
bq. This will allow an 8600 node cluster to complete in 24 hours.
I dint understand this calculation. 
{quote}
I believe this number came out from #nodes * 10 seconds.  Currently a big data node can take
minutes to be fully up after restart and it gets worse with -upgrade.  Perhaps we should refine
this requirement since the timing is affected by many factors in both software and hardware.
- # of blocks
- # of volumes (disks)
- performance of disks and controller
- amount of memory available for caching file system objects (mainly directory entries and
inodes).
- type of underlying file system and operating system

> Umbrella jira for improved HDFS rolling upgrades
> ------------------------------------------------
>
>                 Key: HDFS-5535
>                 URL: https://issues.apache.org/jira/browse/HDFS-5535
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: datanode, ha, hdfs-client, namenode
>    Affects Versions: 3.0.0, 2.2.0
>            Reporter: Nathan Roberts
>         Attachments: HDFSRollingUpgradesHighLevelDesign.pdf
>
>
> In order to roll a new HDFS release through a large cluster quickly and safely, a few
enhancements are needed in HDFS. An initial High level design document will be attached to
this jira, and sub-jiras will itemize the individual tasks.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message