hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Da Zheng <zhengda1...@gmail.com>
Subject message transmission in Hadoop
Date Sat, 29 Jan 2011 03:08:49 GMT
Hello,

I monitored system calls of HDFS with systemtap and found HDFS actually sends
many 1-byte data to the network. I could also see many 8-byte and 64-byte data
written to the OS though I don't know whether they are written to the disk or
sent to the network. I did see many 8-byte data sent to the network. The number
of these data is several times more than 64KB data packet sent by HDFS.

Could anyone tell me why HDFS sends so many small packets? heartbeat messages?
RPCs? It doesn't seem to me these messages can be just 1 byte.

Thanks,
Da

Mime
View raw message