hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10794) [SPS]: Provide storage policy satisfy worker at DN for co-ordinating the block storage movement work
Date Mon, 05 Sep 2016 13:47:21 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15465042#comment-15465042

Kai Zheng commented on HDFS-10794:

Thanks [~Rakesh] for the work. It's a good start.

1. Ref. the codes below
+  private final List<Future<Void>> moverTaskFuturesMap;
How about: moverTaskFuturesMap => moverTaskFutures

2. Maybe: {{BlockToMoveStorageTask}} => {{BlockMovingTask}}; {{BlockStorageMovementCommand}}
=> {{BlockMovingCommand}}; {{BlockInfoToMoveStorage}} => {{BlockMovingInfo}}?

3. In {{testMoveSingleBlockToAnotherDatanode}}, not sure about why it needs to stop some datanode
since what's to test is if storage policy is changed, whether any block movement will happen.

4. Ref. the codes below,
+      StoragePolicySatisfyWorker worker = new StoragePolicySatisfyWorker(conf,
+          src);
+      List<BlockInfoToMoveStorage> blockToMoveStorageTasks = new ArrayList<>();
+      BlockInfoToMoveStorage storagePair = prepareBlockToMoveStoragePair(
+          lb.getBlock(), lb.getLocations()[0], targetDnInfo,
+          lb.getStorageTypes()[0], StorageType.ARCHIVE);
+      blockToMoveStorageTasks.add(storagePair);
+      INode inode = cluster.getNamesystem().getFSDirectory().getINode(file);
+      worker.processBlockToMoveStorageTasks(inode.getId(),
+          blockToMoveStorageTasks);
+      cluster.triggerHeartbeats();
Such codes are better to be equipped with some comments like: "TODO: it doesn't need here
and will be cleaned, when NN is implemented to be able to send block moving commands."

> [SPS]: Provide storage policy satisfy worker at DN for co-ordinating the block storage
movement work
> ----------------------------------------------------------------------------------------------------
>                 Key: HDFS-10794
>                 URL: https://issues.apache.org/jira/browse/HDFS-10794
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, namenode
>    Affects Versions: HDFS-10285
>            Reporter: Rakesh R
>            Assignee: Rakesh R
>         Attachments: HDFS-10794-00.patch, HDFS-10794-HDFS-10285.00.patch, HDFS-10794-HDFS-10285.01.patch,
> The idea of this jira is to implement a mechanism to move the blocks to the given target
in order to satisfy the block storage policy. Datanode receives {{blocktomove}} details via
heart beat response from NN. More specifically, its a datanode side extension to handle the
block storage movement commands.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message