crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <josh.wi...@gmail.com>
Subject Re: Exception with AvroPathPerKeyTarget
Date Fri, 28 Mar 2014 10:41:04 GMT
On Fri, Mar 28, 2014 at 1:22 AM, Gabriel Reid <gabriel.reid@gmail.com>wrote:

> Hi Jeremy,
>
> On Thu, Mar 27, 2014 at 3:26 PM, Jeremy Lewi <jeremy@lewi.us> wrote:
> > Hi
> >
> > I'm hitting the exception pasted below when using AvroPathPerKeyTarget.
> > Interestingly, my code works just fine when I run on a small dataset
> using
> > the LocalJobTracker. However, when I run on a large dataset using a
> hadoop
> > cluster I hit the exception.
> >
>
> Have you ever been able to successfully use the AvroPathPerKeyTarget
> on a real cluster, or is this the first try with it?
>
> I'm wondering if this could be a problem that's always been around (as
> the integration test for AvroPathPerKeyTarget also runs in the local
> jobtracker), or if this could be something new.
>

+1-- Jeremy, if you force the job to run w/a single reducer on the cluster
(i.e., via groupByKey(1)), does it work?


>
> - Gabriel
>

Mime
View raw message