hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jason_jice@yahoo.com" <jason_j...@yahoo.com>
Subject umsubscribe
Date Fri, 18 Jul 2014 04:24:29 GMT

-------- Original Message --------
From: Wellington Chevreuil <wellington.chevreuil@gmail.com>
Sent: Thursday, July 17, 2014 04:34 AM
To: user@hadoop.apache.org
Subject: Re: Replace a block with a new one

>there's no way to do that, as HDFS does not provide file updates features. You'll need
to write a new file with the changes. 
>Notice that even if you manage to find the physical block replica files on the disk, corresponding
to the part of the file you want to change, you can't simply update it manually, as this would
give a different checksum, making HDFS mark such blocks as corrupt.
>On 17 Jul 2014, at 10:50, Zesheng Wu <wuzesheng86@gmail.com> wrote:
>> Hi guys,
>> I recently encounter a scenario which needs to replace an exist block with a newly
written block
>> The most straightforward way to finish may be like this:
>> Suppose the original file is A, and we write a new file B which is composed by the
new data blocks, then we merge A and B to C which is the file we wanted
>> The obvious shortcoming of this method is wasting of network bandwidth 
>> I'm wondering whether there is a way to replace the old block by the new block directly.
>> Any thoughts?
>> -- 
>> Best Wishes!
>> Yours, Zesheng
View raw message