hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gokulakannan M <gok...@huawei.com>
Subject RE: is dfsclient caches the data block to local disk before writing?
Date Mon, 27 Sep 2010 02:42:33 GMT
Thank you dhruba.... 

 

Regards,

Gokul

 

  _____  

From: Dhruba Borthakur [mailto:dhruba@gmail.com] 
Sent: Monday, September 27, 2010 1:30 AM
To: hdfs-user@hadoop.apache.org; gokulm@huawei.com
Cc: hdfs-dev@hadoop.apache.org
Subject: Re: is dfsclient caches the data block to local disk before
writing?

 

DFS client does not write the data to local disk first. Instead, it streams
data directly to the datanodes in the write pipeline. I will update the
document.

On Sun, Sep 26, 2010 at 5:21 AM, Gokulakannan M <gokulm@huawei.com> wrote:

Hi,

 

            Is staging still used in hdfs when writing the data? This doubt
arose when I was going through the hdfs documents. 

            ref :
http://hadoop.apache.org/hdfs/docs/current/hdfs_design.html#Staging

            

            I believe dfsclient does not cache the datablock to local fs (as
the document says) but it does streaming of 64KB packets to the datanode and
caches the

            packets of current block only in memory via dataqueue and
ackqueue.

            

            Is the document needs to be corrected or my understanding is
wrong?

 

 Thanks,

  Gokul

 

  

 

****************************************************************************
***********
This e-mail and attachments contain confidential information from HUAWEI,
which is intended only for the person or entity whose address is listed
above. Any use of the information contained herein in any way (including,
but not limited to, total or partial disclosure, reproduction, or
dissemination) by persons other than the intended recipient's) is
prohibited. If you receive this e-mail in error, please notify the sender by
phone or email immediately and delete it!

 




-- 
Connect to me at http://www.facebook.com/dhruba


Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message