hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6899) Allow changing MiniDFSCluster volumes per DN and capacity per volume
Date Fri, 22 Aug 2014 00:32:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106239#comment-14106239
] 

Jing Zhao commented on HDFS-6899:
---------------------------------

Thanks for the update, [~arpitagarwal]! The new patch looks good to me. Some minors:
# In the following code, if {{Builder#numDataNodes}} and/or {{Builder#storagesPerDatanode}}
is called after {{storageCapacities(long[])}} then the size of the capacities may be wrong?
I think we can remove {{storageCapacities(long[])}} here.
# Maybe we can add some code in the following segment to check the correctness of the size
of {{storageCapacities}}.
{code}
+    if (storageCapacities != null) {
+      for (int i = curDatanodesNum; i < curDatanodesNum+numDataNodes; ++i) {
+        List<? extends FsVolumeSpi> volumes = dns[i].getFSDataset().getVolumes();
+        assert volumes.size() == storagesPerDatanode;
+        
+        for (int j = 0; j < volumes.size(); ++j) {
+          FsVolumeImpl volume = (FsVolumeImpl) volumes.get(j);
+          volume.setCapacityForTesting(storageCapacities[i][j]);
+        }
+      }
+    }
{code}

> Allow changing MiniDFSCluster volumes per DN and capacity per volume
> --------------------------------------------------------------------
>
>                 Key: HDFS-6899
>                 URL: https://issues.apache.org/jira/browse/HDFS-6899
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode, test
>    Affects Versions: 2.5.0
>            Reporter: Arpit Agarwal
>            Assignee: Arpit Agarwal
>         Attachments: HDFS-6899.01.patch, HDFS-6899.02.patch
>
>
> MiniDFSCluster hardcodes the number of directories per volume to two. Propose removing
the hard-coded restriction.
> It would be useful to limit the capacity of individual storage directories for testing
purposes. There is already a way to do so for SimulatedFSDataset, we can add one when using
real volumes.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message