hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hari Sekhon (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-8297) Ability to online trigger data dir rescan for blocks
Date Thu, 30 Apr 2015 11:22:06 GMT
Hari Sekhon created HDFS-8297:

             Summary: Ability to online trigger data dir rescan for blocks
                 Key: HDFS-8297
                 URL: https://issues.apache.org/jira/browse/HDFS-8297
             Project: Hadoop HDFS
          Issue Type: New Feature
          Components: datanode
    Affects Versions: 2.6.0
         Environment: HDP 2.2
            Reporter: Hari Sekhon

Feature request to add functionality to online trigger data dir rescan for available blocks
without having to restart datanode.

Motivation is if using HDFS storage tiering with an archive tier to a separate hyperscale
storage device over the network (Hedvig in this case) which may go away and then return due
to say a network interruption or other temporary error, this leaves HDFS fsck declaring missing
blocks, that are clearly visible on the mount point for the node's archive directory. An online
trigger for data dir rescsan for available blocks would avoid having to do a rolling restart
of all datanodes across a cluster. I did try sending a kill -HUP to the datanode process (both
SecureDataNodeStarter parent and child) while tailing the log hoping this might do it, but
nothing happened in the log.

Hari Sekhon

This message was sent by Atlassian JIRA

View raw message