hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Trevor Robinson <tre...@scurrilous.com>
Subject TestDFSIO job hangs (0.23.1-cdh4b2)
Date Mon, 14 May 2012 20:46:04 GMT
Would someone please give me some troubleshooting tips for TestDFSIO
hanging on a new 0.23.1-cdh4b2 cluster? I've tried both a 5-machine
cluster and just running everything on a single node. It's my first
time configuring YARN, so maybe I've misconfigured something. I don't
see anything suspicious in the logs for namenode, datanode,
resourcemanager, or nodemanager.

$ sudo su hdfs -c 'bin/hadoop --config etc/hadoop jar
./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-0.23.1-cdh4.0.0b2-tests.jar
TestDFSIO -write -nrFiles 10 -fileSize 1GB'
12/05/14 15:02:11 INFO fs.TestDFSIO: TestDFSIO.0.0.6
12/05/14 15:02:11 INFO fs.TestDFSIO: nrFiles = 10
12/05/14 15:02:11 INFO fs.TestDFSIO: fileSize (MB) = 1024.0
12/05/14 15:02:11 INFO fs.TestDFSIO: bufferSize = 1000000
12/05/14 15:02:11 INFO fs.TestDFSIO: baseDir = /benchmarks/TestDFSIO
12/05/14 15:02:11 INFO fs.TestDFSIO: creating control file: 1073741824
bytes, 10 files
12/05/14 15:02:12 INFO fs.TestDFSIO: created control files for: 10 files
12/05/14 15:02:13 INFO mapred.FileInputFormat: Total input paths to process : 10
12/05/14 15:02:13 INFO mapreduce.JobSubmitter: number of splits:10
12/05/14 15:02:13 WARN conf.Configuration: mapred.jar is deprecated.
Instead, use mapreduce.job.jar
12/05/14 15:02:13 WARN conf.Configuration: mapred.reduce.tasks is
deprecated. Instead, use mapreduce.job.reduces
12/05/14 15:02:13 WARN conf.Configuration: mapred.output.value.class
is deprecated. Instead, use mapreduce.job.output.value.class
12/05/14 15:02:13 WARN conf.Configuration:
mapred.used.genericoptionsparser is deprecated. Instead, use
mapreduce.client.genericoptionsparser.used
12/05/14 15:02:13 WARN conf.Configuration: mapred.job.name is
deprecated. Instead, use mapreduce.job.name
12/05/14 15:02:13 WARN conf.Configuration: mapred.input.dir is
deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
12/05/14 15:02:13 WARN conf.Configuration: mapred.output.dir is
deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
12/05/14 15:02:13 WARN conf.Configuration: mapred.map.tasks is
deprecated. Instead, use mapreduce.job.maps
12/05/14 15:02:13 WARN conf.Configuration: mapred.output.key.class is
deprecated. Instead, use mapreduce.job.output.key.class
12/05/14 15:02:13 WARN conf.Configuration: io.bytes.per.checksum is
deprecated. Instead, use dfs.bytes-per-checksum
12/05/14 15:02:13 WARN conf.Configuration: mapred.working.dir is
deprecated. Instead, use mapreduce.job.working.dir
12/05/14 15:02:13 INFO mapred.ResourceMgrDelegate: Submitted
application application_1337025701572_0001 to ResourceManager at
server1/10.10.130.30:8040
12/05/14 15:02:13 INFO mapreduce.Job: The url to track the job:
http://server1:8088/proxy/application_1337025701572_0001/
12/05/14 15:02:13 INFO mapreduce.Job: Running job: job_1337025701572_0001
<30 minutes pass - no significant CPU or disk activity>

Thanks,
Trevor

Mime
View raw message