hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Torsten Curdt <tcu...@apache.org>
Subject Re: still getting "is valid, and cannot be written to"
Date Tue, 28 Aug 2007 20:53:40 GMT

On 28.08.2007, at 20:06, Raghu Angadi wrote:

> This means that a datanode is asked to receive a new block but the  
> block already exists.

That's what I thought when I looked at the code ..but why would it  
receive such a request? (Especially that often!) Shouldn't the name  
node know what blocks are where?

> One case where I have seen this happen is when Namenode is trying  
> to replicate blocks. Say replication increased from 3 to 20,  
> Namenode asks might ask multiple datanodes to transfer a block to  
> the same datanode. Smaller the cluster the cluster more probability  
> that this happens.
> What context are you seeing this?

Good question ...we have seen that come and go. Somehow related to  
the execution of some of our jobs. We haven't found the exact cause  
yet. At some stage we have seen hundreds of these exceptions within a  
few minutes. So far we never increased the replication. But even on  
our 0.14 test cluster we have seen them now. And it's not like we  
have been doing any stress tests on there (yet).


View raw message