hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Shvachko <...@yahoo-inc.com>
Subject Re: dfs incompatibility .3 and .4-dev?
Date Tue, 06 Jun 2006 22:44:44 GMT
Hi Stefan,

I'm trying to investigate the problem.
Could you give some more information about what happened to you.
- what is your cluster size
- amount of data
- how long did dfs ran without restarting the name node before upgrading

> we discussed the three  kinds of data looses, hardware, software or  
> human errors.
> I would love to figure out what was my problem today. :)

Looks like you are not alone :-(

>
> Scenario:
> + updated from hadoop .2.1 to .4.
> + problems to get all datanodes started

what was the problem with datanodes?

> + downgrade to hadoop .3.1
> + error message of incompatible dfs (I guess . already had started to  
> write to the log)

What is the message?

> All transaction done with .2 during last hours lost. Means the data I  
> created and copy was not in the dfs any more.
> I guess the update / downgrade process destroyed the log but the  
> image itself was still ok.
>
> I ended up with a complete corrupted dfs - I guess since the lost dfs  
> namenode transaction log.
>
> Wouldn't be better in case we discover a version conflict in the dfs,  
> that the user need to do manually confirm that the data have to  
> converted into a new format.
> Any thoughts?

Thanks,
Konstantin

Mime
View raw message