hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3194) DataNode block scanner is running too frequently
Date Thu, 16 Aug 2012 18:01:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436166#comment-13436166
] 

Uma Maheswara Rao G commented on HDFS-3194:
-------------------------------------------

Eli, Have seen my previous comment?
 Actually there was another potential issue here and we have raised mainly to address that
functional issue.

DataBlockScanner frequently run the scan by sleeping 5sec. Here we are persisting block scan
times in verification log to track the blocks which are really completed 3weeks period or
not. The problem what I got is, scan method is cleaning that verification log completely after
every run. We have the behavior to take one level backup of verification log. 

so for every 15secs, we will lose the verification log history completely. From then onwards,
blocks will be scanned again even though that blocks scanned just before 15sec back. I hope
this will clear my scenario.

I am not sure I am missing some thing here.

Eli or Andy, have tested this behavior with your patch? your patch solving that issue also?

Thanks
Uma
                
> DataNode block scanner is running too frequently
> ------------------------------------------------
>
>                 Key: HDFS-3194
>                 URL: https://issues.apache.org/jira/browse/HDFS-3194
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node
>    Affects Versions: 2.0.0-alpha
>            Reporter: suja s
>            Assignee: Andy Isaacson
>             Fix For: 2.2.0-alpha
>
>         Attachments: HDFS-3194_1.patch, hdfs-3194-1.txt, HDFS-3194_2.patch, HDFS-3194_4.patch,
HDFS-3194_6.patch, HDFS-3194_7.patch, HDFS-3194.patch
>
>
> Block scanning interval by default should be taken as 21 days(3 weeks) and each block
scanning should happen once in 21 days.
> Here the block is being scanned continuosly.
> 2012-04-03 10:44:47,056 INFO org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner:
Verification succeeded for BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473_1003
> 2012-04-03 10:45:02,064 INFO org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner:
Verification succeeded for BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473_1003
> 2012-04-03 10:45:17,071 INFO org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner:
Verification succeeded for BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473_1003
> 2012-04-03 10:45:32,079 INFO org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner:
Verification succeeded for BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message