hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11182) Update DataNode to use DatasetVolumeChecker
Date Mon, 19 Dec 2016 22:06:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15762449#comment-15762449
] 

ASF GitHub Bot commented on HDFS-11182:
---------------------------------------

Github user arp7 commented on a diff in the pull request:

    https://github.com/apache/hadoop/pull/168#discussion_r93130106
  
    --- Diff: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeList.java
---
    @@ -235,23 +233,14 @@ public void run() {
        * Use {@link checkDirsLock} to allow only one instance of checkDirs() call.
        *
        * @return list of all the failed volumes.
    +   * @param failedVolumes
        */
    -  Set<StorageLocation> checkDirs() {
    +  void handleVolumeFailures(Set<FsVolumeSpi> failedVolumes) {
         try (AutoCloseableLock lock = checkDirsLock.acquire()) {
    -      Set<StorageLocation> failedLocations = null;
    -      // Make a copy of volumes for performing modification 
    -      final List<FsVolumeImpl> volumeList = getVolumes();
     
    -      for(Iterator<FsVolumeImpl> i = volumeList.iterator(); i.hasNext(); ) {
    -        final FsVolumeImpl fsv = i.next();
    +      for(FsVolumeSpi vol : failedVolumes) {
    +        FsVolumeImpl fsv = (FsVolumeImpl) vol;
    --- End diff --
    
    Yes. FsVolumeList is part of the fsdataset.impl package and its methods are only invoked
from FsDatasetImpl so it is safe to assume that the volume is an FsVolumeImpl.
    
    At least one existing method also makes the same assumption (see copyReplicaWithNewBlockIdAndGS).
    ```
      private File[] copyReplicaWithNewBlockIdAndGS(
          ReplicaInfo replicaInfo, String bpid, long newBlkId, long newGS)
          throws IOException {
        String blockFileName = Block.BLOCK_FILE_PREFIX + newBlkId;
        FsVolumeImpl v = (FsVolumeImpl) replicaInfo.getVolume();
    ```


> Update DataNode to use DatasetVolumeChecker
> -------------------------------------------
>
>                 Key: HDFS-11182
>                 URL: https://issues.apache.org/jira/browse/HDFS-11182
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode
>            Reporter: Arpit Agarwal
>            Assignee: Arpit Agarwal
>
> Update DataNode to use the DatasetVolumeChecker class introduced in HDFS-11149 to parallelize
disk checks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message