hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Toshihiro Suzuki (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13949) Correct the description of dfs.datanode.disk.check.timeout in hdfs-default.xml
Date Tue, 09 Oct 2018 23:43:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644253#comment-16644253
] 

Toshihiro Suzuki commented on HDFS-13949:
-----------------------------------------

Thank you very much for reviewing [~nandakumar131].

The property has been also used in ThrottledAsyncChecker that's initialized in the constructor
of DatasetVolumeChecker:
{code}
    diskCheckTimeout = conf.getTimeDuration(
        DFSConfigKeys.DFS_DATANODE_DISK_CHECK_TIMEOUT_KEY,
        DFSConfigKeys.DFS_DATANODE_DISK_CHECK_TIMEOUT_DEFAULT,
        TimeUnit.MILLISECONDS);
....
    delegateChecker = new ThrottledAsyncChecker<>(
        timer, minDiskCheckGapMs, diskCheckTimeout,
        Executors.newCachedThreadPool(
            new ThreadFactoryBuilder()
                .setNameFormat("DataNode DiskChecker thread %d")
                .setDaemon(true)
                .build()));
{code}
This timeout is used in ThrottledAsyncChecker#schedule. And this method is called by DatasetVolumeChecker#checkVolume.
DatasetVolumeChecker#checkVolume is called by DataNode#checkDiskErrorAsync that's called when
there might possibly be a disk failure. So it looks like to me the property is not only used
during DataNode startup.

> Correct the description of dfs.datanode.disk.check.timeout in hdfs-default.xml
> ------------------------------------------------------------------------------
>
>                 Key: HDFS-13949
>                 URL: https://issues.apache.org/jira/browse/HDFS-13949
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: documentation
>            Reporter: Toshihiro Suzuki
>            Assignee: Toshihiro Suzuki
>            Priority: Minor
>         Attachments: HDFS-13949.1.patch
>
>
> The description of dfs.datanode.disk.check.timeout in hdfs-default.xml is as follows:
> {code}
> <property>
>   <name>dfs.datanode.disk.check.timeout</name>
>   <value>10m</value>
>   <description>
>     Maximum allowed time for a disk check to complete during DataNode
>     startup. If the check does not complete within this time interval
>     then the disk is declared as failed. This setting supports
>     multiple time unit suffixes as described in dfs.heartbeat.interval.
>     If no suffix is specified then milliseconds is assumed.
>   </description>
> </property>
> {code}
> I don't think the value of this config is used only during DataNode startup. I think
it's used whenever checking volumes.
> The description is misleading so we need to correct it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message