hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Baldeschwieler <eri...@yahoo-inc.com>
Subject Re: fsck with the move option
Date Tue, 02 May 2006 20:37:16 GMT
So the goal is to be able to map/reduce from input files with missing  
blocks?

On May 1, 2006, at 11:23 PM, Andrzej Bialecki wrote:

> Eric Baldeschwieler wrote:
>> Why not go after the FS APIs so you can simply read the remaining  
>> good blocks?  Might it work already if you had a client that  
>> skipped a block if it found a missing one (via error/exception)?   
>> Or create empty blocks...
>
> Well, this version of fsck is an interim solution anyway. Currently  
> corrupted files can't be processed, so we need a way to weed them  
> out, and possible recover as much data as possible from the leftovers.
>
> My point is that we have no other way to do it now. If/when the  
> missing block handling improves so that it doesn't kill jobs, we  
> can change this part of fsck to simply move files; for now I think  
> a middle-ground solution (additional -recover option) would be better.
>
> -- 
> Best regards,
> Andrzej Bialecki     <><
> ___. ___ ___ ___ _ _   __________________________________
> [__ || __|__/|__||\/|  Information Retrieval, Semantic Web
> ___|||__||  \|  ||  |  Embedded Unix, System Integration
> http://www.sigram.com  Contact: info at sigram dot com
>
>


Mime
View raw message