Mailing-List: contact user-help@crunch.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@crunch.apache.org
Received-SPF: unknown (athena.apache.org: error in processing during lookup of
 admin@nikore.net)
MIME-Version: 1.0
From: Matt Christiansen <admin@nikore.net>
Date: Wed, 7 Aug 2013 17:37:48 -0700
Message-ID: 
 <CA+HR8ZNDWMd_hp+heBW9OKv1gA1xT8j3CTyTJbDzw3R4DHtUVg@mail.gmail.com>
Subject: Crunch Errors while running jobs on YARN
To: user@crunch.apache.org
Content-Type: text/plain; charset=ISO-8859-1

Hey guys, im getting this error while running a crunch job on our YARN
cluster. It doesn't seem to be serious buts generating a lot of QA
questions:


ERROR exec.MRExecutor: Exception thrown fetching job counters for
stage: com.rr.crunch.ads.KeyValuePairsAggregator:
Avro(/QA/event_logs/2013_08_07/pixel_tracking)+S0+pre-distinct+GBK+post-distinct+S1+SeqFile(/tmp/crunch-1725623982/p1)
java.io.IOException
     at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:317)
     at org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:337)
     at org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:476)
     at org.apache.hadoop.mapreduce.Job$8.run(Job.java:757)
     at org.apache.hadoop.mapreduce.Job$8.run(Job.java:754)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:415)
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
     at org.apache.hadoop.mapreduce.Job.getCounters(Job.java:754)
     at org.apache.crunch.impl.mr.exec.MRExecutor.execute(MRExecutor.java:69)
     at org.apache.crunch.impl.mr.MRPipeline.run(MRPipeline.java:153)
     at com.rr.crunch.ads.KeyValuePairsAggregator.process(KeyValuePairsAggregator.java:79)
     at com.rr.crunch.ads.KeyValuePairsAggregator.main(KeyValuePairsAggregator.java:63)

This is with crunch 0.5.0 but I have tried 0.7.0 and get the same
general error (no message about it being job counters but same strack
trace and in the code its at the job.getJob().getCounters() part) I
have tried both the hadoop-1 and hadoop-2 jars; I was wondering if any
one had come acrossed this before or had any idea