hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Сергей Казаков <ss.f...@gmail.com>
Subject Datanode blockreport interval
Date Wed, 27 May 2015 12:20:16 GMT
Hi!

According to defaults in hdfs.site (Hadoop 2.5.1), blockreport interval is
set to 6 hours. We observe a little more frequent reports in hadoop.log:

2015-05-27 03:49:42,580 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: Sent 1 blockreports
1940728 blocks total. Took 25948 msec to generate and 2788 msecs for RPC
and NN processing.  Got back commands none
2015-05-27 04:22:28,762 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: Sent 1 blockreports
1941057 blocks total. Took 2250 msec to generate and 9607 msecs for RPC and
NN processing.  Got back commands
org.apache.hadoop.hdfs.server.protocol.FinalizeCommand@4ad5c971
2015-05-27 09:49:45,548 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: Sent 1 blockreports
1944166 blocks total. Took 22699 msec to generate and 9342 msecs for RPC
and NN processing.  Got back commands none
2015-05-27 10:22:29,260 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: Sent 1 blockreports
1944607 blocks total. Took 3230 msec to generate and 8027 msecs for RPC and
NN processing.  Got back commands
org.apache.hadoop.hdfs.server.protocol.FinalizeCommand@7776078e
2015-05-27 15:49:52,063 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: Sent 1 blockreports
1950516 blocks total. Took 23297 msec to generate and 16673 msecs for RPC
and NN processing.  Got back commands none
2015-05-27 16:22:38,911 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: Sent 1 blockreports
1950871 blocks total. Took 5328 msec to generate and 14004 msecs for RPC
and NN processing.  Got back commands
org.apache.hadoop.hdfs.server.protocol.FinalizeCommand@6bb6eec9

Does anyone know why second blockreport occurs 33 minutes after first one?

-- 
Kind regards,
Sergey Kazakov

Mime
View raw message