hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lixiang Ao <aolixi...@gmail.com>
Subject Benchmark Failure
Date Mon, 17 Mar 2014 16:26:36 GMT
Hi all,

I'm running jobclient tests(on single node), other tests like TestDFSIO,
mrbench succeed except nnbench.

I got a lot of Exceptions but without any explanation(see below).

Could anyone tell me what might went wrong?

Thanks!


14/03/17 23:54:22 INFO hdfs.NNBench: Waiting in barrier for: 112819 ms
14/03/17 23:54:23 INFO mapreduce.Job: Job job_local2133868569_0001 running
in uber mode : false
14/03/17 23:54:23 INFO mapreduce.Job:  map 0% reduce 0%
14/03/17 23:54:28 INFO mapred.LocalJobRunner: hdfs://
0.0.0.0:9000/benchmarks/NNBench-aolx-PC/control/NNBench_Controlfile_10:0+125>
map
14/03/17 23:54:29 INFO mapreduce.Job:  map 6% reduce 0%
14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op:
Create/Write/Close
14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op:
Create/Write/Close
14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op:
Create/Write/Close
14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op:
Create/Write/Close
14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op:
Create/Write/Close
14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op:
Create/Write/Close
14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op:
Create/Write/Close
14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op:
Create/Write/Close
(1000 Exceptions)
.
.
.
results:

File System Counters
FILE: Number of bytes read=18769411
 FILE: Number of bytes written=21398315
FILE: Number of read operations=0
FILE: Number of large read operations=0
 FILE: Number of write operations=0
HDFS: Number of bytes read=11185
HDFS: Number of bytes written=19540
 HDFS: Number of read operations=325
HDFS: Number of large read operations=0
HDFS: Number of write operations=13210
 Map-Reduce Framework
Map input records=12
Map output records=95
 Map output bytes=1829
Map output materialized bytes=2091
Input split bytes=1538
 Combine input records=0
Combine output records=0
Reduce input groups=8
 Reduce shuffle bytes=0
Reduce input records=95
Reduce output records=8
 Spilled Records=214
Shuffled Maps =0
Failed Shuffles=0
 Merged Map outputs=0
GC time elapsed (ms)=211
CPU time spent (ms)=0
 Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
Total committed heap usage (bytes)=4401004544
 File Input Format Counters
Bytes Read=1490
File Output Format Counters
 Bytes Written=170
14/03/17 23:56:18 INFO hdfs.NNBench: -------------- NNBench --------------
:
14/03/17 23:56:18 INFO hdfs.NNBench:
 Version: NameNode Benchmark 0.4
14/03/17 23:56:18 INFO hdfs.NNBench:                            Date &
time: 2014-03-17 23:56:18,619
14/03/17 23:56:18 INFO hdfs.NNBench:
14/03/17 23:56:18 INFO hdfs.NNBench:                         Test
Operation: create_write
14/03/17 23:56:18 INFO hdfs.NNBench:                             Start
time: 2014-03-17 23:56:15,521
14/03/17 23:56:18 INFO hdfs.NNBench:                            Maps to
run: 12
14/03/17 23:56:18 INFO hdfs.NNBench:                         Reduces to
run: 6
14/03/17 23:56:18 INFO hdfs.NNBench:                     Block Size
(bytes): 1
14/03/17 23:56:18 INFO hdfs.NNBench:                         Bytes to
write: 0
14/03/17 23:56:18 INFO hdfs.NNBench:                     Bytes per
checksum: 1
14/03/17 23:56:18 INFO hdfs.NNBench:                        Number of
files: 1000
14/03/17 23:56:18 INFO hdfs.NNBench:                     Replication
factor: 3
14/03/17 23:56:18 INFO hdfs.NNBench:             Successful file
operations: 0
14/03/17 23:56:18 INFO hdfs.NNBench:
14/03/17 23:56:18 INFO hdfs.NNBench:         # maps that missed the
barrier: 11
14/03/17 23:56:18 INFO hdfs.NNBench:                           #
exceptions: 1000
14/03/17 23:56:18 INFO hdfs.NNBench:
14/03/17 23:56:18 INFO hdfs.NNBench:                TPS:
Create/Write/Close: 0
14/03/17 23:56:18 INFO hdfs.NNBench: Avg exec time (ms):
Create/Write/Close: Infinity
14/03/17 23:56:18 INFO hdfs.NNBench:             Avg Lat (ms):
Create/Write: NaN
14/03/17 23:56:18 INFO hdfs.NNBench:                    Avg Lat (ms):
Close: NaN
14/03/17 23:56:18 INFO hdfs.NNBench:
14/03/17 23:56:18 INFO hdfs.NNBench:                  RAW DATA: AL Total
#1: 0
14/03/17 23:56:18 INFO hdfs.NNBench:                  RAW DATA: AL Total
#2: 0
14/03/17 23:56:18 INFO hdfs.NNBench:               RAW DATA: TPS Total
(ms): 1131
14/03/17 23:56:18 INFO hdfs.NNBench:        RAW DATA: Longest Map Time
(ms): 1.395071776653E12
14/03/17 23:56:18 INFO hdfs.NNBench:                    RAW DATA: Late
maps: 11
14/03/17 23:56:18 INFO hdfs.NNBench:              RAW DATA: # of
exceptions: 1000
14/03/17 23:56:18 INFO hdfs.NNBench:

Mime
View raw message