Mailing-List: contact crunch-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: crunch-dev@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of josh.wills@gmail.com
 designates 209.85.212.171 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <9FF58E1C-9197-43A1-A84D-43FA3F806CF6@gmail.com>
References: <9FF58E1C-9197-43A1-A84D-43FA3F806CF6@gmail.com>
From: Josh Wills <josh.wills@gmail.com>
Date: Wed, 15 Aug 2012 17:00:43 -0700
Message-ID: 
 <CANb5z2KVG6ES591u3JDusiPzyDCBk63DZN6Bq4_HSZend_2W9Q@mail.gmail.com>
Subject: Re: Crunch Hadoop version support
To: crunch-dev@incubator.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

In my mind, it's a function of how much surgery we would need to do in
order to support 0.20.2-- if it's just the multiple outputs stuff,
that seems like a small price to pay in terms of code duplication vs.
wider support. But I don't have a feel for how deep the rabbit hole
goes. If you wouldn't mind taking a crack at it and reporting back, we
would be much obliged-- but if you find yourself shaving a yak, you
should bail out. :)

J

On Wed, Aug 15, 2012 at 3:58 PM, Jonathan Dinu <jondinu@gmail.com> wrote:
> Hi,
>
> I have been using Crunch and starting to love it more and more.  I have b=
een using it with Hadoop-cdh3u4 and it has been swell but as Josh points ou=
t (http://mail-archives.apache.org/mod_mbox/incubator-crunch-user/201207.mb=
ox/%3CCAH29n6N3GXkQ6B-NT3w3JwK8aMo7vDr9ip2=3D1xx0UF3KiKJ4ZA@mail.gmail.com%=
3E) it depends on Hadoop being able to write multiple output files.  I have=
 tried my code with multiple versions of hadoop and it seems like 1.0.1 is =
the earliest version that I got my tests to successful complete with.  0.20=
.x is not compatible because of the file output issue.
>
> I was curious if it is a goal of Crunch as it goes through Apache incubat=
ion to become compatible with the 0.20.x versions of Hadoop (or more versio=
ns), if it is actually possible to make it compatible with these versions, =
and if it is something that is desired by others.
>
> I have not yet contributed to Crunch, but I would be interested in helpin=
g to make it compatible with the 0.20.x versions of Hadoop if the rest of t=
he community thinks this is a worthwhile endeavor.  Just wanted to get the =
opinion of the community and creators of Crunch.
>
> Best,
> Jonathan