hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lixiang Ao <aolixi...@gmail.com>
Subject Re: Benchmark Failure
Date Tue, 18 Mar 2014 05:04:12 GMT
the version is release 2.2.0
2014年3月18日 上午12:26于 "Lixiang Ao" <aolixiang@gmail.com>写道:

> Hi all,
>
> I'm running jobclient tests(on single node), other tests like TestDFSIO,
> mrbench succeed except nnbench.
>
> I got a lot of Exceptions but without any explanation(see below).
>
> Could anyone tell me what might went wrong?
>
> Thanks!
>
>
> 14/03/17 23:54:22 INFO hdfs.NNBench: Waiting in barrier for: 112819 ms
> 14/03/17 23:54:23 INFO mapreduce.Job: Job job_local2133868569_0001 running
> in uber mode : false
> 14/03/17 23:54:23 INFO mapreduce.Job:  map 0% reduce 0%
> 14/03/17 23:54:28 INFO mapred.LocalJobRunner: hdfs://
> 0.0.0.0:9000/benchmarks/NNBench-aolx-PC/control/NNBench_Controlfile_10:0+125> map
> 14/03/17 23:54:29 INFO mapreduce.Job:  map 6% reduce 0%
> 14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op:
> Create/Write/Close
> 14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op:
> Create/Write/Close
> 14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op:
> Create/Write/Close
> 14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op:
> Create/Write/Close
> 14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op:
> Create/Write/Close
> 14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op:
> Create/Write/Close
> 14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op:
> Create/Write/Close
> 14/03/17 23:56:15 INFO hdfs.NNBench: Exception recorded in op:
> Create/Write/Close
> (1000 Exceptions)
> .
> .
> .
> results:
>
> File System Counters
> FILE: Number of bytes read=18769411
>  FILE: Number of bytes written=21398315
> FILE: Number of read operations=0
> FILE: Number of large read operations=0
>  FILE: Number of write operations=0
> HDFS: Number of bytes read=11185
> HDFS: Number of bytes written=19540
>  HDFS: Number of read operations=325
> HDFS: Number of large read operations=0
> HDFS: Number of write operations=13210
>  Map-Reduce Framework
> Map input records=12
> Map output records=95
>  Map output bytes=1829
> Map output materialized bytes=2091
> Input split bytes=1538
>  Combine input records=0
> Combine output records=0
> Reduce input groups=8
>  Reduce shuffle bytes=0
> Reduce input records=95
> Reduce output records=8
>  Spilled Records=214
> Shuffled Maps =0
> Failed Shuffles=0
>  Merged Map outputs=0
> GC time elapsed (ms)=211
> CPU time spent (ms)=0
>  Physical memory (bytes) snapshot=0
> Virtual memory (bytes) snapshot=0
> Total committed heap usage (bytes)=4401004544
>  File Input Format Counters
> Bytes Read=1490
> File Output Format Counters
>  Bytes Written=170
> 14/03/17 23:56:18 INFO hdfs.NNBench: -------------- NNBench --------------
> :
> 14/03/17 23:56:18 INFO hdfs.NNBench:
>  Version: NameNode Benchmark 0.4
> 14/03/17 23:56:18 INFO hdfs.NNBench:                            Date &
> time: 2014-03-17 23:56:18,619
> 14/03/17 23:56:18 INFO hdfs.NNBench:
> 14/03/17 23:56:18 INFO hdfs.NNBench:                         Test
> Operation: create_write
> 14/03/17 23:56:18 INFO hdfs.NNBench:                             Start
> time: 2014-03-17 23:56:15,521
> 14/03/17 23:56:18 INFO hdfs.NNBench:                            Maps to
> run: 12
> 14/03/17 23:56:18 INFO hdfs.NNBench:                         Reduces to
> run: 6
> 14/03/17 23:56:18 INFO hdfs.NNBench:                     Block Size
> (bytes): 1
> 14/03/17 23:56:18 INFO hdfs.NNBench:                         Bytes to
> write: 0
> 14/03/17 23:56:18 INFO hdfs.NNBench:                     Bytes per
> checksum: 1
> 14/03/17 23:56:18 INFO hdfs.NNBench:                        Number of
> files: 1000
> 14/03/17 23:56:18 INFO hdfs.NNBench:                     Replication
> factor: 3
> 14/03/17 23:56:18 INFO hdfs.NNBench:             Successful file
> operations: 0
> 14/03/17 23:56:18 INFO hdfs.NNBench:
> 14/03/17 23:56:18 INFO hdfs.NNBench:         # maps that missed the
> barrier: 11
> 14/03/17 23:56:18 INFO hdfs.NNBench:                           #
> exceptions: 1000
> 14/03/17 23:56:18 INFO hdfs.NNBench:
> 14/03/17 23:56:18 INFO hdfs.NNBench:                TPS:
> Create/Write/Close: 0
> 14/03/17 23:56:18 INFO hdfs.NNBench: Avg exec time (ms):
> Create/Write/Close: Infinity
> 14/03/17 23:56:18 INFO hdfs.NNBench:             Avg Lat (ms):
> Create/Write: NaN
> 14/03/17 23:56:18 INFO hdfs.NNBench:                    Avg Lat (ms):
> Close: NaN
> 14/03/17 23:56:18 INFO hdfs.NNBench:
> 14/03/17 23:56:18 INFO hdfs.NNBench:                  RAW DATA: AL Total
> #1: 0
> 14/03/17 23:56:18 INFO hdfs.NNBench:                  RAW DATA: AL Total
> #2: 0
> 14/03/17 23:56:18 INFO hdfs.NNBench:               RAW DATA: TPS Total
> (ms): 1131
> 14/03/17 23:56:18 INFO hdfs.NNBench:        RAW DATA: Longest Map Time
> (ms): 1.395071776653E12
> 14/03/17 23:56:18 INFO hdfs.NNBench:                    RAW DATA: Late
> maps: 11
> 14/03/17 23:56:18 INFO hdfs.NNBench:              RAW DATA: # of
> exceptions: 1000
> 14/03/17 23:56:18 INFO hdfs.NNBench:
>
>

Mime
View raw message