Hi Brian, I'm using hadoop-2.3.0-cdh5.1.0, there is gridmix jar in this
package.
step 1: using Rumen to generate the job trace file.
sudo -u yarn java -cp `hadoop classpath`
org.apache.hadoop.tools.rumen.TraceBuilder
file:///tmp/jobhistory_log/job-trace.json
file:///tmp/jobhistory_log/topology.output
file:///tmp/jobhistory_log/15/000375/
step 2: run gridmix on yarn+mrv2
sudo -u yarn hadoop dfs -put /tmp/jobhistory_log/job-trace.json /tmp
sudo -u yarn hadoop jar /usr/lib/hadoop-mapreduce/hadoop-gridmix.jar
-Dgridmix.min.file.size=10485760
-Dgridmix.job-submission.use-queue-in-trace=true
-Dgridmix.distributed-cache-emulation.enable=false -generate 133120m
hdfs:///user/yarn/foo/ hdfs:///tmp/job-trace.json
hope this can be helpful.
-Leitao
2014-08-01 20:08 GMT+08:00 Brian Husted <brian.husted@gmail.com>:
> Hi Guo,
>
> Can you please let me know if there was any specific configuration needed
> to get Gridmix working with YARN+MRv2. We are getting the following
> exception:
>
> INFO gridmix.JobSubmitter: Job org.apache.hadoop.mapreduce.Job@18a8ce2
> submission failed java.lang.ArithmeticException: / by zero
> at
>
> org.apache.hadoop.mapred.gridmix.GenerateData$GenDataFormat.getSplits(GenerateData.java:161)
> at
> org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:902)
>
> It seems to be very similar to this issue which relates to not having the
> tasktracker is not running
>
> https://issues.apache.org/jira/browse/MAPREDUCE-2016
>
>
>
>
>
> --------------------------------------------------------
> Yes, I'm using Gridmix running on YARN+MRv2.
>
|