hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vidur Goyal" <vi...@students.iiit.ac.in>
Subject Re: Overwriting the same block instead of creating a new one
Date Tue, 22 Jun 2010 05:03:37 GMT
Dear Todd,

By truncating i meant removing unused *blocks* from the namespace and let
them be garbage collected. There will be no truncation of the last
block(even if it is not full). This way , rather then garbage collecting
all the blocks of a file , we will only be garbage collecting the
remaining blocks.

-vidur


> HDFS assumes in hundreds of places that blocks never shrink. So, there is
> no
> option to truncate a block.
>
> -Todd
>
> On Mon, Jun 21, 2010 at 9:41 PM, Vidur Goyal
> <vidur@students.iiit.ac.in>wrote:
>
>> Hi All,
>>
>> In FSNamesystem#startFileInternal , whenever there is a overwrite flag
>> set
>> , why is the INode removed from the namespace and a new
>> INodeFileUnderConstruction is created. Why can't we use the convert the
>> same INode to INodeFileUnderConstruction. And we start writing to the
>> same
>> blocks at the same datanodes (after incrementing the GS) followed by
>> either truncating the remaining blocks(if the file size decreases) or
>> allocating new blocks (if the file size increases). This will decrease
>> data redundancy and the job of garbage collector and will increase
>> security.
>>
>> vidur
>>
>>
>>
>>
>> --
>> This message has been scanned for viruses and
>> dangerous content by MailScanner, and is
>> believed to be clean.
>>
>>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
>
>


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


Mime
View raw message