hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sagar Shukla <sagar_shu...@persistent.co.in>
Subject RE: Hadoop Compatibility and EMR
Date Thu, 25 Mar 2010 04:28:51 GMT
Hi Ilayaraja,
      Hadoop HDFS has a utility distcp using which it should be possible to copy data between
two different versions of hadoop. Though I am not quite sure if this utility would be able
to read data from hadoop 15.5 . More information is available at the link - http://hadoop.apache.org/common/docs/r0.19.1/distcp.html#cpver

There is no information on supported versions at this URL, but it could be worth a try.

You can also create a local environment to push your HDFS data from older version to newer
version. And then finally, move it on Amazon EMR.


-----Original Message-----
From: ilayaraja [mailto:ilayaraja@rediff.co.in] 
Sent: Sunday, March 21, 2010 12:53 PM
To: hadoop-user; hadoop-dev
Subject: Hadoop Compatibility and EMR


We 've been using hadoop 15.5 in our production environment where we have about 10 TB of data
stored on the dfs.
The files were generated as mapreduce output. We want to move our env. to Amazon Elastic Map
Reduce (EMR) which throws the following questions to us:

1. EMR supports only hadoop 19.0 and above. Is it possible to use the current data that were
generated with hadoop 15.5 from hadoop 19.0?

2. Or how can we make it possible to use or update to hadoop 19.0 from hadoop 15.5? What are
the issues expected while doing so?


This e-mail may contain privileged and confidential information which is the property of Persistent
Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed.
If you are not the intended recipient, you are not authorized to read, retain, copy, print,
distribute or use this message. If you have received this communication in error, please notify
the sender and delete all copies of this message. Persistent Systems Ltd. does not accept
any liability for virus infected mails.

View raw message