hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (HDDS-2267) Container metadata scanner interval mismatch
Date Wed, 09 Oct 2019 06:33:00 GMT

     [ https://issues.apache.org/jira/browse/HDDS-2267?focusedWorklogId=325501&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-325501
]

ASF GitHub Bot logged work on HDDS-2267:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 09/Oct/19 06:32
            Start Date: 09/Oct/19 06:32
    Worklog Time Spent: 10m 
      Work Description: adoroszlai commented on pull request #1624: HDDS-2267. Container metadata
scanner interval mismatch
URL: https://github.com/apache/hadoop/pull/1624
 
 
   ## What changes were proposed in this pull request?
   
   Fix time unit mismatch in container metadata scanner.  Elapsed time is measured in nanoseconds,
needs to be converted to milliseconds.
   
   https://issues.apache.org/jira/browse/HDDS-2267
   
   ## How was this patch tested?
   
   Tested on `ozone` docker-compose cluster.
   
   ```
   datanode_1  | 2019-10-09 06:18:00 INFO  ContainerMetadataScanner:60 - Background ContainerMetadataScanner
starting up
   datanode_1  | 2019-10-09 06:18:00 INFO  ContainerMetadataScanner:88 - Completed an iteration
of container metadata scrubber in 0 minutes. Number of  iterations (since the data-node restart)
: 1, Number of containers scanned in this iteration : 0, Number of unhealthy containers found
in this iteration : 0
   datanode_1  | 2019-10-09 06:19:00 INFO  ContainerMetadataScanner:88 - Completed an iteration
of container metadata scrubber in 0 minutes. Number of  iterations (since the data-node restart)
: 2, Number of containers scanned in this iteration : 2, Number of unhealthy containers found
in this iteration : 0
   ...
   datanode_1  | STARTUP_MSG: Starting HddsDatanodeService
   datanode_1  | 2019-10-09 06:23:52 INFO  ContainerMetadataScanner:60 - Background ContainerMetadataScanner
starting up
   datanode_1  | 2019-10-09 06:23:52 INFO  ContainerMetadataScanner:88 - Completed an iteration
of container metadata scrubber in 0 minutes. Number of  iterations (since the data-node restart)
: 1, Number of containers scanned in this iteration : 2, Number of unhealthy containers found
in this iteration : 0
   datanode_1  | 2019-10-09 06:24:52 INFO  ContainerMetadataScanner:88 - Completed an iteration
of container metadata scrubber in 0 minutes. Number of  iterations (since the data-node restart)
: 2, Number of containers scanned in this iteration : 2, Number of unhealthy containers found
in this iteration : 0
   ```
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 325501)
    Remaining Estimate: 0h
            Time Spent: 10m

> Container metadata scanner interval mismatch
> --------------------------------------------
>
>                 Key: HDDS-2267
>                 URL: https://issues.apache.org/jira/browse/HDDS-2267
>             Project: Hadoop Distributed Data Store
>          Issue Type: Bug
>          Components: Ozone Datanode
>            Reporter: Attila Doroszlai
>            Assignee: Attila Doroszlai
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Container metadata scanner can be configured to run at specific time intervals, eg. hourly
({{hdds.containerscrub.metadata.scan.interval}}).  However, the actual run interval does not
match the configuration.  After a datanode restart, it runs in quick succession, later it
runs at apparently random intervals.
> {noformat:title=sample log}
> datanode_1  | 2019-10-08 14:05:30 INFO  ContainerMetadataScanner:88 - Completed an iteration
of container metadata scrubber in 0 minutes. Number of  iterations (since the data-node restart)
: 1, Number of containers scanned in this iteration : 0, Number of unhealthy containers found
in this iteration : 0
> datanode_1  | 2019-10-08 14:09:33 INFO  ContainerMetadataScanner:88 - Completed an iteration
of container metadata scrubber in 0 minutes. Number of  iterations (since the data-node restart)
: 1, Number of containers scanned in this iteration : 6, Number of unhealthy containers found
in this iteration : 0
> ...
> datanode_1  | 2019-10-08 14:09:33 INFO  ContainerMetadataScanner:88 - Completed an iteration
of container metadata scrubber in 0 minutes. Number of  iterations (since the data-node restart)
: 28, Number of containers scanned in this iteration : 6, Number of unhealthy containers found
in this iteration : 0
> datanode_1  | 2019-10-08 14:21:01 INFO  ContainerMetadataScanner:88 - Completed an iteration
of container metadata scrubber in 0 minutes. Number of  iterations (since the data-node restart)
: 29, Number of containers scanned in this iteration : 6, Number of unhealthy containers found
in this iteration : 0
> datanode_1  | 2019-10-08 14:21:01 INFO  ContainerMetadataScanner:88 - Completed an iteration
of container metadata scrubber in 0 minutes. Number of  iterations (since the data-node restart)
: 30, Number of containers scanned in this iteration : 6, Number of unhealthy containers found
in this iteration : 0
> datanode_1  | 2019-10-08 15:30:38 INFO  ContainerMetadataScanner:88 - Completed an iteration
of container metadata scrubber in 0 minutes. Number of  iterations (since the data-node restart)
: 31, Number of containers scanned in this iteration : 6, Number of unhealthy containers found
in this iteration : 0
> datanode_1  | 2019-10-08 16:45:01 INFO  ContainerMetadataScanner:88 - Completed an iteration
of container metadata scrubber in 0 minutes. Number of  iterations (since the data-node restart)
: 32, Number of containers scanned in this iteration : 6, Number of unhealthy containers found
in this iteration : 0
> {noformat}
> The problem is that time elapsed is measured in nanoseconds, while the configuration
is in milliseconds.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message