hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zesheng Wu <wuzeshen...@gmail.com>
Subject Re: Replace a block with a new one
Date Fri, 18 Jul 2014 01:47:45 GMT
How about write a new block with new checksum file, and replace the old
block file and checksum file both?


2014-07-17 19:34 GMT+08:00 Wellington Chevreuil <
wellington.chevreuil@gmail.com>:

> Hi,
>
> there's no way to do that, as HDFS does not provide file updates features.
> You'll need to write a new file with the changes.
>
> Notice that even if you manage to find the physical block replica files on
> the disk, corresponding to the part of the file you want to change, you
> can't simply update it manually, as this would give a different checksum,
> making HDFS mark such blocks as corrupt.
>
> Regards,
> Wellington.
>
>
>
> On 17 Jul 2014, at 10:50, Zesheng Wu <wuzesheng86@gmail.com> wrote:
>
> > Hi guys,
> >
> > I recently encounter a scenario which needs to replace an exist block
> with a newly written block
> > The most straightforward way to finish may be like this:
> > Suppose the original file is A, and we write a new file B which is
> composed by the new data blocks, then we merge A and B to C which is the
> file we wanted
> > The obvious shortcoming of this method is wasting of network bandwidth
> >
> > I'm wondering whether there is a way to replace the old block by the new
> block directly.
> > Any thoughts?
> >
> > --
> > Best Wishes!
> >
> > Yours, Zesheng
>
>


-- 
Best Wishes!

Yours, Zesheng

Mime
View raw message