Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of zhengda1936@gmail.com
 designates 209.85.216.48 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:user-agent:mime-version:to:subject:references
         :in-reply-to:content-type:content-transfer-encoding;
        b=W2pu1i+KWT5TI/Fn6OWZKti+EzN8x0Hc7aG+0mkBi9yCqwVbvy6UHvZBhINE7lbqH1
         qJBWT6NJ8W7cpOmDkwOry7uVSADcL6y3/yZHVCstTnEseFfqbpwCMJ/1Pcf0X31DWWLi
         ssK4so/um0WO4jXHJS3DskBORY/i0SsVDczco=
Message-ID: <4D46EA8E.30500@gmail.com>
Date: Mon, 31 Jan 2011 11:59:58 -0500
From: Da Zheng <zhengda1936@gmail.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
 rv:1.9.2.13) Gecko/20101208 Thunderbird/3.1.7
MIME-Version: 1.0
To: common-user@hadoop.apache.org
Subject: Re: message transmission in Hadoop
References: <4D4384C1.5060709@gmail.com>
 <AANLkTimNAdXBoQnrAs22tt+95MOGcVsREXTJrfxtmX3B@mail.gmail.com>
In-Reply-To: <AANLkTimNAdXBoQnrAs22tt+95MOGcVsREXTJrfxtmX3B@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Yes, this is exactly what I observed. reading is another problem. Thanks.

Best,
Da

On 01/30/2011 05:25 PM, Jeff Hammerbacher wrote:
> Hey Da,
>
> You may have observed https://issues.apache.org/jira/browse/HDFS-1601.
>
> Regards,
> Jeff
>
> On Fri, Jan 28, 2011 at 7:08 PM, Da Zheng<zhengda1936@gmail.com>  wrote:
>
>> Hello,
>>
>> I monitored system calls of HDFS with systemtap and found HDFS actually
>> sends
>> many 1-byte data to the network. I could also see many 8-byte and 64-byte
>> data
>> written to the OS though I don't know whether they are written to the disk
>> or
>> sent to the network. I did see many 8-byte data sent to the network. The
>> number
>> of these data is several times more than 64KB data packet sent by HDFS.
>>
>> Could anyone tell me why HDFS sends so many small packets? heartbeat
>> messages?
>> RPCs? It doesn't seem to me these messages can be just 1 byte.
>>
>> Thanks,
>> Da
>>