hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dhruba Borthakur <dhr...@gmail.com>
Subject Re: is dfsclient caches the data block to local disk before writing?
Date Sun, 26 Sep 2010 19:59:43 GMT
DFS client does not write the data to local disk first. Instead, it streams
data directly to the datanodes in the write pipeline. I will update the
document.

On Sun, Sep 26, 2010 at 5:21 AM, Gokulakannan M <gokulm@huawei.com> wrote:

>  Hi,
>
>
>
>             Is staging still used in hdfs when writing the data? This doubt
> arose when I was going through the hdfs documents.
>
>             ref :
> http://hadoop.apache.org/hdfs/docs/current/hdfs_design.html#Staging
>
>
>
>             I believe dfsclient *does not cache the datablock to local fs*(as the document
says) but it does streaming of 64KB packets to the datanode
> and caches the
>
>             packets of current block only in memory via dataqueue and
> ackqueue.
>
>
>
>             Is the document needs to be corrected or my understanding is
> wrong?
>
>
>
>  Thanks,
>
>   Gokul
>
>
>
>
>
>
>
>
> ***************************************************************************************
> This e-mail and attachments contain confidential information from HUAWEI,
> which is intended only for the person or entity whose address is listed
> above. Any use of the information contained herein in any way (including,
> but not limited to, total or partial disclosure, reproduction, or
> dissemination) by persons other than the intended recipient's) is
> prohibited. If you receive this e-mail in error, please notify the sender by
> phone or email immediately and delete it!
>
>
>



-- 
Connect to me at http://www.facebook.com/dhruba

Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message