hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arthur.hk.chan@gmail.com" <arthur.hk.c...@gmail.com>
Subject Hadoop Smoke Test: TERASORT
Date Wed, 10 Sep 2014 13:56:22 GMT

I am trying the smoke test for Hadoop (2.4.1).  About “terasort”, below is my test command,
the Map part was completed very fast because it was split into many subtasks, however the
Reduce part takes very long time and only 1 running Reduce job.  Is there a way speed up the
reduce phase by splitting the large reduce job into many smaller ones and run them across
the cluster like the Map part?

bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar  terasort /tmp/teragenout

Job ID							Name	State			Maps Total	Maps Completed		Reduce Total 			Reduce Complted
job_1409876705457_0002  	TeraSort	RUNNING 		22352		22352				1 					0



View raw message