Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <478127535.1224629324534.JavaMail.jira@brutus>
Date: Tue, 21 Oct 2008 15:48:44 -0700 (PDT)
From: "Robert Chansler (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Updated: (HADOOP-2816) Cluster summary at name node web has
 confusing report for space utilization
In-Reply-To: <27122215.1202956388376.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HADOOP-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Chansler updated HADOOP-2816:
------------------------------------

    Release Note: Improved space reporting for NameNode Web UI. Applications that parse the Web UI output should be reviewed.  (was: This fix changes the following:
1) Datanode heartbeat reported Capacity information is changed. Earlier the Capacity was sum of all the diskspace of data directories. With this change, it is sum of all the diskspace of data directories minus the reserved space configured using dfs.datanode.du.reserved config param. This change is reflected by changing the protocol version from 17 to 18.

2) The Namenode Web UI is changed accordingly as detailed below...

Cluster Summary
Capacity : Currently, this is sum of the file system capacity of all the data directories. This is changed to Sum of the file system capacity of all the data directories minus Reserved space. The name is changed to "Configured Capacity".

Present Capacity: This is newly added and represents the present capacity available for DFS use. This is sum of DFS Remaining and DFS Used given below

DFS Remaining : This will remain as it is
DFS Used : This will remain as it is
DFS Used% : This is changed. It is calculated based on Present Capacity and not Configured Capacity.
Live Nodes : This will remain as it is
Dead Nodes : This will remain as it is

Node data prints currently:
Node Last Contact Admin State Size (TB) Used (%) Used (%) Remaining (TB) Blocks

It will be change to:
Node Last Contact Admin State Capacity (TB) Present Capacity (TB) Used (%) Used (%) Remaining (TB) Blocks

Size column is renamed as Total Capacity. Previously this was calculated as sum of file system capacity of all the data directories. It is changed to exclude reserved space and will be calculated as (sum of file system capacity of all the data directories - reserved space)
)
    Hadoop Flags: [Incompatible change, Reviewed]  (was: [Reviewed, Incompatible change])

This fix changes the following:
1) Datanode heartbeat reported Capacity information is changed. Earlier the Capacity was sum of all the diskspace of data directories. With this change, it is sum of all the diskspace of data directories minus the reserved space configured using dfs.datanode.du.reserved config param. This change is reflected by changing the protocol version from 17 to 18.

2) The Namenode Web UI is changed accordingly as detailed below...

Cluster Summary
Capacity : Currently, this is sum of the file system capacity of all the data directories. This is changed to Sum of the file system capacity of all the data directories minus Reserved space. The name is changed to "Configured Capacity".

Present Capacity: This is newly added and represents the present capacity available for DFS use. This is sum of DFS Remaining and DFS Used given below

DFS Remaining : This will remain as it is
DFS Used : This will remain as it is
DFS Used% : This is changed. It is calculated based on Present Capacity and not Configured Capacity.
Live Nodes : This will remain as it is
Dead Nodes : This will remain as it is

Node data prints currently:
Node Last Contact Admin State Size (TB) Used (%) Used (%) Remaining (TB) Blocks

It will be change to:
Node Last Contact Admin State Capacity (TB) Present Capacity (TB) Used (%) Used (%) Remaining (TB) Blocks

Size column is renamed as Total Capacity. Previously this was calculated as sum of file system capacity of all the data directories. It is changed to exclude reserved space and will be calculated as (sum of file system capacity of all the data directories - reserved space)

> Cluster summary at name node web has confusing report for space utilization
> ---------------------------------------------------------------------------
>
>                 Key: HADOOP-2816
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2816
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Robert Chansler
>            Assignee: Suresh Srinivas
>             Fix For: 0.19.0
>
>         Attachments: HADOOP-2816.patch, HADOOP-2816.patch, HADOOP-2816.patch
>
>
> In one example:
> Cluster Summary
> Capacity	:	1.15 PB
> DFS Remaining	:	192 TB
> DFS Used	:	717 TB
> DFS Used%	:	62 %
> Why is Capacity not equal Used plus Remaining?
> (The answer is that there is an estimated reserve for local files.)
> The presentation should be easily understood by the user.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.