harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From xuwenyang2005 <xuwenyang2...@126.com>
Subject hadoop testing on hs
Date Fri, 02 Jul 2010 06:52:29 GMT
Hello,
I am doing hadoop testing on the harmony select(r946981) and when I try to run some application,
I encountered some problems.Maybe some defect exists inside the harmony select caused these
problems and I will be very thankful if anybody could give me some advise to solve it.
I have configured 4 machine hadoop cluster(1 namenode, 1 jobtracker, 2 slaves), on which hadoop-0.20.2
could be started up successfully.
All of the 4 machines are linux(ubuntu) platform, JAVA_HOME is set to be the path of harmony
select, other configuration values are just set to be default.
After the hadoop cluster is started up, I ran the terasort, a sorting application, with 500M
input data.
During the testing, the job failed.
console messageļ¼š
10/07/02 13:23:44 INFO terasort.TeraSort: starting
10/07/02 13:23:45 INFO mapred.FileInputFormat: Total input paths to process : 2
10/07/02 13:23:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable
10/07/02 13:23:47 INFO compress.CodecPool: Got brand-new compressor
Making 1 from 100000 records
Step size is 100000.0
10/07/02 13:23:50 INFO mapred.JobClient: Running job: job_201007021322_0001
10/07/02 13:23:51 INFO mapred.JobClient:  map 0% reduce 0%
10/07/02 13:24:43 INFO mapred.JobClient:  map 7% reduce 0%
10/07/02 13:24:46 INFO mapred.JobClient:  map 12% reduce 0%
10/07/02 13:25:06 INFO mapred.JobClient:  map 13% reduce 0%
10/07/02 13:25:09 INFO mapred.JobClient:  map 15% reduce 0%
10/07/02 13:25:12 INFO mapred.JobClient:  map 16% reduce 0%
10/07/02 13:25:15 INFO mapred.JobClient:  map 18% reduce 0%
10/07/02 13:25:18 INFO mapred.JobClient:  map 19% reduce 0%
10/07/02 13:25:21 INFO mapred.JobClient:  map 20% reduce 0%
10/07/02 13:25:24 INFO mapred.JobClient:  map 22% reduce 0%
10/07/02 13:25:27 INFO mapred.JobClient:  map 23% reduce 0%
10/07/02 13:25:30 INFO mapred.JobClient:  map 24% reduce 0%
10/07/02 13:34:55 INFO mapred.JobClient:  map 12% reduce 0%
10/07/02 13:35:03 INFO mapred.JobClient: Task Id : attempt_201007021322_0001_m_000002_0, Status
: FAILED
Task attempt_201007021322_0001_m_000002_0 failed to report status for 601 seconds. Killing!
10/07/02 13:35:03 INFO mapred.JobClient: Task Id : attempt_201007021322_0001_m_000003_0, Status
: FAILED
Task attempt_201007021322_0001_m_000003_0 failed to report status for 602 seconds. Killing!
10/07/02 13:35:16 INFO mapred.JobClient:  map 17% reduce 0%
10/07/02 13:35:19 INFO mapred.JobClient:  map 23% reduce 0%
10/07/02 13:35:22 INFO mapred.JobClient:  map 24% reduce 0%
10/07/02 13:35:46 INFO mapred.JobClient:  map 12% reduce 0%
10/07/02 13:36:06 INFO mapred.JobClient: Task Id : attempt_201007021322_0001_m_000000_0, Status
: FAILED
Task attempt_201007021322_0001_m_000000_0 failed to report status for 602 seconds. Killing!
10/07/02 13:36:07 INFO mapred.JobClient: Task Id : attempt_201007021322_0001_m_000001_0, Status
: FAILED
Task attempt_201007021322_0001_m_000001_0 failed to report status for 600 seconds. Killing!
10/07/02 13:36:30 INFO mapred.JobClient:  map 13% reduce 0%
10/07/02 13:36:33 INFO mapred.JobClient:  map 16% reduce 0%
10/07/02 13:36:36 INFO mapred.JobClient:  map 18% reduce 0%
....

log :

2010-07-02 13:35:09,538 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics
with processName=MAP, sessionId=
2010-07-02 13:35:10,433 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 1
2010-07-02 13:35:10,459 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 100
2010-07-02 13:35:11,073 INFO org.apache.hadoop.mapred.MapTask: data buffer = 79691776/99614720
2010-07-02 13:35:11,074 INFO org.apache.hadoop.mapred.MapTask: record buffer = 262144/327680
2010-07-02 13:35:11,308 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
2010-07-02 13:35:11,311 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
2010-07-02 13:35:16,153 INFO org.apache.hadoop.mapred.MapTask: Spilling map output: record
full = true
2010-07-02 13:35:16,154 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 26214400;
bufvoid = 99614720
2010-07-02 13:35:16,154 INFO org.apache.hadoop.mapred.MapTask: kvstart = 0; kvend = 262144;
length = 327680


Look forward to any reply. 

Best Regards,
Cindy

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message