hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jinglun (JIRA)" <j...@apache.org>
Subject [jira] [Reopened] (HDFS-14631) The DirectoryScanner doesn't fix the wrongly placed replica.
Date Wed, 07 Aug 2019 13:32:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-14631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jinglun reopened HDFS-14631:
----------------------------

upload patch for branch-2.x

> The DirectoryScanner doesn't fix the wrongly placed replica.
> ------------------------------------------------------------
>
>                 Key: HDFS-14631
>                 URL: https://issues.apache.org/jira/browse/HDFS-14631
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Jinglun
>            Assignee: Jinglun
>            Priority: Major
>             Fix For: 3.3.0, 3.2.1, 3.1.3
>
>         Attachments: HDFS-14631-branch-2.9.001.patch, HDFS-14631.001.patch, HDFS-14631.002.patch,
HDFS-14631.003.patch, HDFS-14631.004.patch
>
>
> When DirectoryScanner scans block files, if the block refers to the block file does not
exist the DirectoryScanner will update the block based on the replica file found on the disk.
See FsDatasetImpl#checkAndUpdate.
>  
> {code:java}
> /*
> * Block exists in volumeMap and the block file exists on the disk
> */
> // Compare block files
> if (memBlockInfo.blockDataExists()) {
>   ...
> } else {
>   // Block refers to a block file that does not exist.
>   // Update the block with the file found on the disk. Since the block
>   // file and metadata file are found as a pair on the disk, update
>   // the block based on the metadata file found on the disk
>   LOG.warn("Block file in replica "
>       + memBlockInfo.getBlockURI()
>       + " does not exist. Updating it to the file found during scan "
>       + diskFile.getAbsolutePath());
>   memBlockInfo.updateWithReplica(
>   StorageLocation.parse(diskFile.toString()));
>   LOG.warn("Updating generation stamp for block " + blockId
>       + " from " + memBlockInfo.getGenerationStamp() + " to " + diskGS);
>   memBlockInfo.setGenerationStamp(diskGS);
> }
> {code}
> But the DirectoryScanner doesn't really fix it because in LocalReplica#parseBaseDir()
the 'subdir' are ignored.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org


Mime
View raw message