hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brahma Reddy Battula <brahmareddy.batt...@huawei.com>
Subject RE: Benchmark Failure
Date Wed, 19 Mar 2014 06:51:34 GMT
Seems to be this is issue, which is logged..Please check following jira for same....Hope you
also facing same issue...



https://issues.apache.org/jira/browse/HDFS-4929







Thanks & Regards



Brahma Reddy Battula



________________________________
From: Lixiang Ao [aolixiang@gmail.com]
Sent: Tuesday, March 18, 2014 10:34 AM
To: user@hadoop.apache.org
Subject: Re: Benchmark Failure


the version is release 2.2.0

2014年3月18日 上午12:26于 "Lixiang Ao" <aolixiang@gmail.com<mailto:aolixiang@gmail.com>>写道:
Hi all,

I'm running jobclient tests(on single node), other tests like TestDFSIO, mrbench succeed except
nnbench.

I got a lot of Exceptions but without any explanation(see below).

Could anyone tell me what might went wrong?

Thanks!


14/03/17 23:54:22 INFO hdfs.NNBench: Waiting in barrier for: 112819 ms
14/03/17 23:54:23 INFO mapreduce.Job: Job job_local2133868569_0001 running in uber mode :
false
14/03/17 23:54:23 INFO mapreduce.Job:  map 0% reduce 0%
14/03/17 23:54:28 INFO mapred.LocalJobRunner: hdfs://0.0.0.0:9000/benchmarks/NNBench-aolx-PC/control/NNBench_Controlfile_10:0+125<http://0.0.0.0:9000/benchmarks/NNBench-aolx-PC/control/NNBench_Controlfile_10:0+125>
> map
14/03/17 23:54:29 INFO mapreduce.Job:  map 6% reduce 0%
14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op: Create/Write/Close
14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op: Create/Write/Close
14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op: Create/Write/Close
14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op: Create/Write/Close
14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op: Create/Write/Close
14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op: Create/Write/Close
14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op: Create/Write/Close
14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op: Create/Write/Close
(1000 Exceptions)
.
.
.
results:

File System Counters
FILE: Number of bytes read=18769411
FILE: Number of bytes written=21398315
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=11185
HDFS: Number of bytes written=19540
HDFS: Number of read operations=325
HDFS: Number of large read operations=0
HDFS: Number of write operations=13210
Map-Reduce Framework
Map input records=12
Map output records=95
Map output bytes=1829
Map output materialized bytes=2091
Input split bytes=1538
Combine input records=0
Combine output records=0
Reduce input groups=8
Reduce shuffle bytes=0
Reduce input records=95
Reduce output records=8
Spilled Records=214
Shuffled Maps =0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=211
CPU time spent (ms)=0
Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
Total committed heap usage (bytes)=4401004544
File Input Format Counters
Bytes Read=1490
File Output Format Counters
Bytes Written=170
14/03/17 23:56:18 INFO hdfs.NNBench: -------------- NNBench -------------- :
14/03/17 23:56:18 INFO hdfs.NNBench:                                Version: NameNode Benchmark
0.4
14/03/17 23:56:18 INFO hdfs.NNBench:                            Date & time: 2014-03-17
23:56:18,619
14/03/17 23:56:18 INFO hdfs.NNBench:
14/03/17 23:56:18 INFO hdfs.NNBench:                         Test Operation: create_write
14/03/17 23:56:18 INFO hdfs.NNBench:                             Start time: 2014-03-17 23:56:15,521
14/03/17 23:56:18 INFO hdfs.NNBench:                            Maps to run: 12
14/03/17 23:56:18 INFO hdfs.NNBench:                         Reduces to run: 6
14/03/17 23:56:18 INFO hdfs.NNBench:                     Block Size (bytes): 1
14/03/17 23:56:18 INFO hdfs.NNBench:                         Bytes to write: 0
14/03/17 23:56:18 INFO hdfs.NNBench:                     Bytes per checksum: 1
14/03/17 23:56:18 INFO hdfs.NNBench:                        Number of files: 1000
14/03/17 23:56:18 INFO hdfs.NNBench:                     Replication factor: 3
14/03/17 23:56:18 INFO hdfs.NNBench:             Successful file operations: 0
14/03/17 23:56:18 INFO hdfs.NNBench:
14/03/17 23:56:18 INFO hdfs.NNBench:         # maps that missed the barrier: 11
14/03/17 23:56:18 INFO hdfs.NNBench:                           # exceptions: 1000
14/03/17 23:56:18 INFO hdfs.NNBench:
14/03/17 23:56:18 INFO hdfs.NNBench:                TPS: Create/Write/Close: 0
14/03/17 23:56:18 INFO hdfs.NNBench: Avg exec time (ms): Create/Write/Close: Infinity
14/03/17 23:56:18 INFO hdfs.NNBench:             Avg Lat (ms): Create/Write: NaN
14/03/17 23:56:18 INFO hdfs.NNBench:                    Avg Lat (ms): Close: NaN
14/03/17 23:56:18 INFO hdfs.NNBench:
14/03/17 23:56:18 INFO hdfs.NNBench:                  RAW DATA: AL Total #1: 0
14/03/17 23:56:18 INFO hdfs.NNBench:                  RAW DATA: AL Total #2: 0
14/03/17 23:56:18 INFO hdfs.NNBench:               RAW DATA: TPS Total (ms): 1131
14/03/17 23:56:18 INFO hdfs.NNBench:        RAW DATA: Longest Map Time (ms): 1.395071776653E12
14/03/17 23:56:18 INFO hdfs.NNBench:                    RAW DATA: Late maps: 11
14/03/17 23:56:18 INFO hdfs.NNBench:              RAW DATA: # of exceptions: 1000
14/03/17 23:56:18 INFO hdfs.NNBench:

Mime
View raw message