hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Chansler (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-2816) Cluster summary at name node web has confusing report for space utilization
Date Tue, 21 Oct 2008 22:48:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Robert Chansler updated HADOOP-2816:
------------------------------------

    Release Note: Improved space reporting for NameNode Web UI. Applications that parse the
Web UI output should be reviewed.  (was: This fix changes the following:
1) Datanode heartbeat reported Capacity information is changed. Earlier the Capacity was sum
of all the diskspace of data directories. With this change, it is sum of all the diskspace
of data directories minus the reserved space configured using dfs.datanode.du.reserved config
param. This change is reflected by changing the protocol version from 17 to 18.

2) The Namenode Web UI is changed accordingly as detailed below...

Cluster Summary
Capacity : Currently, this is sum of the file system capacity of all the data directories.
This is changed to Sum of the file system capacity of all the data directories minus Reserved
space. The name is changed to "Configured Capacity".

Present Capacity: This is newly added and represents the present capacity available for DFS
use. This is sum of DFS Remaining and DFS Used given below

DFS Remaining : This will remain as it is
DFS Used : This will remain as it is
DFS Used% : This is changed. It is calculated based on Present Capacity and not Configured
Capacity.
Live Nodes : This will remain as it is
Dead Nodes : This will remain as it is

Node data prints currently:
Node Last Contact Admin State Size (TB) Used (%) Used (%) Remaining (TB) Blocks

It will be change to:
Node Last Contact Admin State Capacity (TB) Present Capacity (TB) Used (%) Used (%) Remaining
(TB) Blocks

Size column is renamed as Total Capacity. Previously this was calculated as sum of file system
capacity of all the data directories. It is changed to exclude reserved space and will be
calculated as (sum of file system capacity of all the data directories - reserved space)
)
    Hadoop Flags: [Incompatible change, Reviewed]  (was: [Reviewed, Incompatible change])

This fix changes the following:
1) Datanode heartbeat reported Capacity information is changed. Earlier the Capacity was sum
of all the diskspace of data directories. With this change, it is sum of all the diskspace
of data directories minus the reserved space configured using dfs.datanode.du.reserved config
param. This change is reflected by changing the protocol version from 17 to 18.

2) The Namenode Web UI is changed accordingly as detailed below...

Cluster Summary
Capacity : Currently, this is sum of the file system capacity of all the data directories.
This is changed to Sum of the file system capacity of all the data directories minus Reserved
space. The name is changed to "Configured Capacity".

Present Capacity: This is newly added and represents the present capacity available for DFS
use. This is sum of DFS Remaining and DFS Used given below

DFS Remaining : This will remain as it is
DFS Used : This will remain as it is
DFS Used% : This is changed. It is calculated based on Present Capacity and not Configured
Capacity.
Live Nodes : This will remain as it is
Dead Nodes : This will remain as it is

Node data prints currently:
Node Last Contact Admin State Size (TB) Used (%) Used (%) Remaining (TB) Blocks

It will be change to:
Node Last Contact Admin State Capacity (TB) Present Capacity (TB) Used (%) Used (%) Remaining
(TB) Blocks

Size column is renamed as Total Capacity. Previously this was calculated as sum of file system
capacity of all the data directories. It is changed to exclude reserved space and will be
calculated as (sum of file system capacity of all the data directories - reserved space)

> Cluster summary at name node web has confusing report for space utilization
> ---------------------------------------------------------------------------
>
>                 Key: HADOOP-2816
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2816
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Robert Chansler
>            Assignee: Suresh Srinivas
>             Fix For: 0.19.0
>
>         Attachments: HADOOP-2816.patch, HADOOP-2816.patch, HADOOP-2816.patch
>
>
> In one example:
> Cluster Summary
> Capacity	:	1.15 PB
> DFS Remaining	:	192 TB
> DFS Used	:	717 TB
> DFS Used%	:	62 %
> Why is Capacity not equal Used plus Remaining?
> (The answer is that there is an estimated reserve for local files.)
> The presentation should be easily understood by the user.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message