crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Dinu <jond...@gmail.com>
Subject Crunch Hadoop version support
Date Wed, 15 Aug 2012 22:58:17 GMT
Hi,

I have been using Crunch and starting to love it more and more.  I have been using it with
Hadoop-cdh3u4 and it has been swell but as Josh points out (http://mail-archives.apache.org/mod_mbox/incubator-crunch-user/201207.mbox/%3CCAH29n6N3GXkQ6B-NT3w3JwK8aMo7vDr9ip2=1xx0UF3KiKJ4ZA@mail.gmail.com%3E)
it depends on Hadoop being able to write multiple output files.  I have tried my code with
multiple versions of hadoop and it seems like 1.0.1 is the earliest version that I got my
tests to successful complete with.  0.20.x is not compatible because of the file output issue.
 

I was curious if it is a goal of Crunch as it goes through Apache incubation to become compatible
with the 0.20.x versions of Hadoop (or more versions), if it is actually possible to make
it compatible with these versions, and if it is something that is desired by others.

I have not yet contributed to Crunch, but I would be interested in helping to make it compatible
with the 0.20.x versions of Hadoop if the rest of the community thinks this is a worthwhile
endeavor.  Just wanted to get the opinion of the community and creators of Crunch.

Best,
Jonathan
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message