hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shumin Guo <gsmst...@gmail.com>
Subject Re: Replace a block with a new one
Date Fri, 18 Jul 2014 21:10:27 GMT
That will break the consistency of the file system, but it doesn't hurt to
try.
On Jul 17, 2014 8:48 PM, "Zesheng Wu" <wuzesheng86@gmail.com> wrote:

> How about write a new block with new checksum file, and replace the old
> block file and checksum file both?
>
>
> 2014-07-17 19:34 GMT+08:00 Wellington Chevreuil <
> wellington.chevreuil@gmail.com>:
>
>> Hi,
>>
>> there's no way to do that, as HDFS does not provide file updates
>> features. You'll need to write a new file with the changes.
>>
>> Notice that even if you manage to find the physical block replica files
>> on the disk, corresponding to the part of the file you want to change, you
>> can't simply update it manually, as this would give a different checksum,
>> making HDFS mark such blocks as corrupt.
>>
>> Regards,
>> Wellington.
>>
>>
>>
>> On 17 Jul 2014, at 10:50, Zesheng Wu <wuzesheng86@gmail.com> wrote:
>>
>> > Hi guys,
>> >
>> > I recently encounter a scenario which needs to replace an exist block
>> with a newly written block
>> > The most straightforward way to finish may be like this:
>> > Suppose the original file is A, and we write a new file B which is
>> composed by the new data blocks, then we merge A and B to C which is the
>> file we wanted
>> > The obvious shortcoming of this method is wasting of network bandwidth
>> >
>> > I'm wondering whether there is a way to replace the old block by the
>> new block directly.
>> > Any thoughts?
>> >
>> > --
>> > Best Wishes!
>> >
>> > Yours, Zesheng
>>
>>
>
>
> --
> Best Wishes!
>
> Yours, Zesheng
>

Mime
View raw message