hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From EdwardKing <zhan...@neusoft.com>
Subject Where is hdfs result ?
Date Mon, 23 Jun 2014 06:36:17 GMT
 I use hadoop 2.2.0 under Redhat Linux,I run hadoop mapreduce examples like follows:

[yarn@localhost sbin]$ hadoop jar /opt/yarn/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar
pi -Dmapreduce.clientfactory.class.name=org.apache.hadoop.mapred.YarnClientFactory -libjars
/opt/yarn/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0.jar
16 10000
Number of Maps  = 16
Samples per Map = 10000
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Wrote input for Map #10
Wrote input for Map #11
Wrote input for Map #12
Wrote input for Map #13
Wrote input for Map #14
Wrote input for Map #15
Starting Job
14/06/22 19:57:58 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
14/06/22 19:57:59 INFO input.FileInputFormat: Total input paths to process : 16
14/06/22 19:57:59 INFO mapreduce.JobSubmitter: number of splits:16
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.job.classpath.files is deprecated.
Instead, use mapreduce.job.classpath.files
14/06/22 19:57:59 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.cache.files.filesizes is deprecated.
Instead, use mapreduce.job.cache.files.filesizes
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.cache.files is deprecated. Instead,
use mapreduce.job.cache.files
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is
deprecated. Instead, use mapreduce.map.speculative
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead,
use mapreduce.job.reduces
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.output.value.class is deprecated.
Instead, use mapreduce.job.output.value.class
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution
is deprecated. Instead, use mapreduce.reduce.speculative
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.used.genericoptionsparser is deprecated.
Instead, use mapreduce.client.genericoptionsparser.used
14/06/22 19:57:59 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead,
use mapreduce.job.map.class
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead,
use mapreduce.job.name
14/06/22 19:57:59 INFO Configuration.deprecation: mapreduce.reduce.class is deprecated. Instead,
use mapreduce.job.reduce.class
14/06/22 19:57:59 INFO Configuration.deprecation: mapreduce.inputformat.class is deprecated.
Instead, use mapreduce.job.inputformat.class
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead,
use mapreduce.input.fileinputformat.inputdir
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead,
use mapreduce.output.fileoutputformat.outputdir
14/06/22 19:57:59 INFO Configuration.deprecation: mapreduce.outputformat.class is deprecated.
Instead, use mapreduce.job.outputformat.class
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead,
use mapreduce.job.maps
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.cache.files.timestamps is deprecated.
Instead, use mapreduce.job.cache.files.timestamps
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.output.key.class is deprecated. Instead,
use mapreduce.job.output.key.class
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead,
use mapreduce.job.working.dir
14/06/22 19:58:00 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1403492007236_0001
14/06/22 19:58:01 INFO impl.YarnClientImpl: Submitted application application_1403492007236_0001
to ResourceManager at /0.0.0.0:8032
14/06/22 19:58:01 INFO mapreduce.Job: The url to track the job: http://localhost:8088/proxy/application_1403492007236_0001/
14/06/22 19:58:01 INFO mapreduce.Job: Running job: job_1403492007236_0001
14/06/22 19:58:22 INFO mapreduce.Job: Job job_1403492007236_0001 running in uber mode : false
14/06/22 19:58:22 INFO mapreduce.Job:  map 0% reduce 0%
14/06/22 20:01:45 INFO mapreduce.Job:  map 6% reduce 0%
14/06/22 20:01:52 INFO mapreduce.Job:  map 13% reduce 0%
14/06/22 20:01:53 INFO mapreduce.Job:  map 19% reduce 0%
14/06/22 20:01:56 INFO mapreduce.Job:  map 38% reduce 0%
14/06/22 20:02:36 INFO mapreduce.Job:  map 50% reduce 15%
14/06/22 20:02:37 INFO mapreduce.Job:  map 69% reduce 15%
14/06/22 20:02:39 INFO mapreduce.Job:  map 69% reduce 23%
14/06/22 20:03:05 INFO mapreduce.Job:  map 94% reduce 23%
14/06/22 20:03:06 INFO mapreduce.Job:  map 100% reduce 31%
14/06/22 20:03:08 INFO mapreduce.Job:  map 100% reduce 100%
14/06/22 20:03:09 INFO mapreduce.Job: Job job_1403492007236_0001 completed successfully
14/06/22 20:03:09 INFO mapreduce.Job: Counters: 43
 File System Counters
  FILE: Number of bytes read=358
  FILE: Number of bytes written=1387065
  FILE: Number of read operations=0
  FILE: Number of large read operations=0
  FILE: Number of write operations=0
  HDFS: Number of bytes read=4230
  HDFS: Number of bytes written=215
  HDFS: Number of read operations=67
  HDFS: Number of large read operations=0
  HDFS: Number of write operations=3
 Job Counters 
  Launched map tasks=16
  Launched reduce tasks=1
  Data-local map tasks=16
  Total time spent by all maps in occupied slots (ms)=1587720
  Total time spent by all reduces in occupied slots (ms)=71305
 Map-Reduce Framework
  Map input records=16
  Map output records=32
  Map output bytes=288
  Map output materialized bytes=448
  Input split bytes=2342
  Combine input records=0
  Combine output records=0
  Reduce input groups=2
  Reduce shuffle bytes=448
  Reduce input records=32
  Reduce output records=0
  Spilled Records=64
  Shuffled Maps =16
  Failed Shuffles=0
  Merged Map outputs=16
  GC time elapsed (ms)=45172
  CPU time spent (ms)=107260
  Physical memory (bytes) snapshot=2563190784
  Virtual memory (bytes) snapshot=6111346688
  Total committed heap usage (bytes)=1955135488
 Shuffle Errors
  BAD_ID=0
  CONNECTION=0
  IO_ERROR=0
  WRONG_LENGTH=0
  WRONG_MAP=0
  WRONG_REDUCE=0
 File Input Format Counters 
  Bytes Read=1888
 File Output Format Counters 
  Bytes Written=97
Job Finished in 311.613 seconds
Estimated value of Pi is 3.14127500000000000000


Then I check hdfs like follows:
[yarn@localhost sbin]$ hdfs dfs -ls 
[yarn@localhost sbin]

Why hdfs don't show any information? How to do it? Thanks

---------------------------------------------------------------------------------------------------
Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s)

is intended only for the use of the intended recipient and may be confidential and/or privileged
of 
Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication
is 
not the intended recipient, unauthorized use, forwarding, printing,  storing, disclosure or
copying 
is strictly prohibited, and may be unlawful.If you have received this communication in error,please

immediately notify the sender by return e-mail, and delete the original message and all copies
from 
your system. Thank you. 
---------------------------------------------------------------------------------------------------
Mime
View raw message