hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: distcp problems going from hadoop-0.20.1 to -0.20.2
Date Sat, 23 Apr 2011 18:13:11 GMT
Errr really? Well shipping a bunch of hard drives should be faster.

J-D
On Apr 23, 2011 12:17 AM, "Jonathan Disher" <jdisher@parad.net> wrote:
> Aha, that works.
>
> Any ideas what kind of throughput I can expect, or suggestions for making
this run as fast as possible? Obviously exact numbers will depend on cluster
config, I won't bore you with the details, but... 10mbit? 100mbit? A
gigabit? I've got about 112TB of data to move from the East coast to the
West coast, and sooner would be better than later :)
>
> -j
>
> On Apr 22, 2011, at 10:38 PM, Jean-Daniel Cryans wrote:
>
>> See "Copying between versions of HDFS":
>> http://hadoop.apache.org/common/docs/r0.20.2/distcp.html#cpver
>>
>> J-D
>>
>> On Fri, Apr 22, 2011 at 10:37 PM, Jonathan Disher <jdisher@parad.net>
wrote:
>>> I have an existing cluster running hadoop-0.20.1, and I am migrating
most of the data to a new cluster running -0.20.2. I am seeing this in the
namenode logs when I try to run a distcp:
>>>
>>> @400000004db263bf29c77134 WARN ipc.Server: Incorrect header or version
mismatch from newNN:46111 got version 4 expected version 3
>>> 2011-04-23 05:30:55,999 WARN org.apache.hadoop.ipc.Server: Incorrect
header or version mismatch from oldNN:48750 got version 3 expected version 4
>>>
>>> When I run my distcp, on either side, it dies with a
java.io.IOException/java.io.EOFException.
>>>
>>> Ideas? Am I screwed? I really don't want to drop my new cluster down to
0.20.1.
>>>
>>> -j
>

Mime
View raw message