hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anu Engineer (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-10824) MiniDFSCluster#storageCapacities has no effects on real capacity
Date Tue, 13 Sep 2016 17:23:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15487799#comment-15487799
] 

Anu Engineer edited comment on HDFS-10824 at 9/13/16 5:23 PM:
--------------------------------------------------------------

Thanks for the patch. Couple of Questions.

# I was wondering if storageCapacities is a good member name at all ? if you had called it
StorageCap instead of storageCapacities, you save lots of unnecessary edits where you are
replacing the storageCapacities with StorageCap. 
# Since we made storageCapacities a member, does it make sense it pass thru as an argument
to all functions. Most of the time, storageCap is nothing but {{this.StorageCapacities}}.
So was wondering if we should just use the member variable directly ? 
# Could you please tell me why we have added tiggerHeartbeat and triggerBlock in line 2288
and 2289. I am not able to understand why we need to trigger them manually with datanode restart.
# In the test, can we please pass different capacities to MiniDFSCluster and make sure that
we are able to support that ? 


was (Author: anu):
Thanks for the patch. Couple of Questions.

# I was wondering if storageCapacities is a good member name at all ? if you had called it
StorageCap instead of storageCapacities, you save lots of unnecessary edits where you are
replacing the storageCapacities with StorageCap. 
# Since we made storageCapacities a member, does it make sense it pass thru as an argument
to all functions. Most of the time, storageCap is nothing but {{this.StorageCapacities}}.
So was wondering if we should just use the member variable directly ? 
# Could you please tell me why we have added tiggerHeartbeat and triggerBlock in line 2288
and 2289. I am not able to understand why we need to trigger them manually with datanode restart.

> MiniDFSCluster#storageCapacities has no effects on real capacity
> ----------------------------------------------------------------
>
>                 Key: HDFS-10824
>                 URL: https://issues.apache.org/jira/browse/HDFS-10824
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Xiaobing Zhou
>            Assignee: Xiaobing Zhou
>         Attachments: HDFS-10824.000.patch, HDFS-10824.001.patch
>
>
> It has been noticed MiniDFSCluster#storageCapacities has no effects on real capacity.
It can be reproduced by explicitly setting storageCapacities and then call ClientProtocol#getDatanodeStorageReport(DatanodeReportType.LIVE)
to compare results. The following are  storage report for one node with two volumes after
I set capacity as 300 * 1024. Apparently, the capacity is not changed.
> adminState|DatanodeInfo$AdminStates  (id=6861)
> |blockPoolUsed|215192|
> |cacheCapacity|0|
> |cacheUsed|0|
> |capacity|998164971520|
> |datanodeUuid|"839912e9-5bcb-45d1-81cf-9a9c9c02a00b" (id=6862)|
> |dependentHostNames|LinkedList<E>  (id=6863)|
> |dfsUsed|215192|
> |hostName|"127.0.0.1" (id=6864)|
> |infoPort|64222|
> |infoSecurePort|0|
> |ipAddr|"127.0.0.1" (id=6865)|
> |ipcPort|64223|
> |lastUpdate|1472682790948|
> |lastUpdateMonotonic|209605640|
> |level|0|
> |location|"/default-rack" (id=6866)|
> |maintenanceExpireTimeInMS|0|
> |parent|null|
> |peerHostName|null|
> |remaining|20486512640|
> |softwareVersion|null|
> |upgradeDomain|null|
> |xceiverCount|1|
> |xferAddr|"127.0.0.1:64220" (id=6855)|
> |xferPort|64220|
> [0]StorageReport  (id=6856)
> |blockPoolUsed|4096|
> |capacity|499082485760|
> |dfsUsed|4096|
> |failed|false|
> |remaining|10243256320|
> |storage|DatanodeStorage  (id=6869)|
> [1]StorageReport  (id=6859)
> |blockPoolUsed|211096|
> |capacity|499082485760|
> |dfsUsed|211096|
> |failed|false|
> |remaining|10243256320|
> |storage|DatanodeStorage  (id=6872)|



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message