hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shi Yu <sh...@uchicago.edu>
Subject standalone ? mapred.LocalJobRunner
Date Mon, 06 Jun 2011 17:21:00 GMT
Hi, I am stuck in a basic problem but can't figure out. My previous 
verbose logging problem is the same as the one mentioned in the old post.

http://mail-archives.apache.org/mod_mbox/nutch-user/200901.mbox/%3C0ADBD67BD6811A4BB2144D805124714D03F754A447@KAEX1.Dom.Rastatt.de%3E

First quesiton, if I see a lot of logs on the screen like (as mentioned in Tom White's ``Hadoop:
The definitive Guide'' book, page 23):


09/04/07 12:34:35 INFO mapred.MapTask: numReduceTasks: 1
09/04/07 12:34:35 INFO mapred.MapTask: io.sort.mb = 100
09/04/07 12:34:35 INFO mapred.MapTask: data buffer = 79691776/99614720
09/04/07 12:34:35 INFO mapred.MapTask: record buffer = 262144/327680
09/04/07 12:34:35 INFO mapred.MapTask: Starting flush of map output
09/04/07 12:34:36 INFO mapred.MapTask: Finished spill 0

does it mean I am running in the standalone mode? I think in a real cluster mode I should
not see these. When I was running my code in real cluster model, I only see output like

Map 10% Reduce 0%

and all the logs are written to logs/userlogs folder.


So I guess I entered a LocalJob mode (standalone) mistakenly, but not in the real cluster
mode. However, I did setup the three xml files correctly I think, and I started up the MapReduce
daemons (start-dfs.sh, start-mapred.sh). So why the code is still running in standalone mode?
Anything else I should pay attention to? Thanks!


Shi




Mime
View raw message