harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From xuwenyang2005 <xuwenyang2...@126.com>
Subject Re:Re: hadoop testing on hs
Date Tue, 06 Jul 2010 03:13:22 GMT
Thanks Jim,

In fact, no error message for the JRE could be observed during the test.
That's just what troubles me.
For the hdk, I will try it :)



在2010-07-05 13:23:39,"Jim Yu" <junjie0122@gmail.com> 写道:
>Hi Cindy,
>
>In the console and log message you provided, there is only trace info of
>Hadoop.  Did you see any error message for the JRE? As Harmony Select is a
>subset of HDK, any luck to try with the HDK? :-)
>
>在 2010年7月2日 下午2:52,xuwenyang2005 <xuwenyang2005@126.com>写道:
>
>> Hello,
>> I am doing hadoop testing on the harmony select(r946981) and when I try to
>> run some application, I encountered some problems.Maybe some defect exists
>> inside the harmony select caused these problems and I will be very thankful
>> if anybody could give me some advise to solve it.
>> I have configured 4 machine hadoop cluster(1 namenode, 1 jobtracker, 2
>> slaves), on which hadoop-0.20.2 could be started up successfully.
>> All of the 4 machines are linux(ubuntu) platform, JAVA_HOME is set to be
>> the path of harmony select, other configuration values are just set to be
>> default.
>> After the hadoop cluster is started up, I ran the terasort, a sorting
>> application, with 500M input data.
>> During the testing, the job failed.
>> console message:
>> 10/07/02 13:23:44 INFO terasort.TeraSort: starting
>> 10/07/02 13:23:45 INFO mapred.FileInputFormat: Total input paths to process
>> : 2
>> 10/07/02 13:23:47 WARN util.NativeCodeLoader: Unable to load native-hadoop
>> library for your platform... using builtin-java classes where applicable
>> 10/07/02 13:23:47 INFO compress.CodecPool: Got brand-new compressor
>> Making 1 from 100000 records
>> Step size is 100000.0
>> 10/07/02 13:23:50 INFO mapred.JobClient: Running job: job_201007021322_0001
>> 10/07/02 13:23:51 INFO mapred.JobClient:  map 0% reduce 0%
>> 10/07/02 13:24:43 INFO mapred.JobClient:  map 7% reduce 0%
>> 10/07/02 13:24:46 INFO mapred.JobClient:  map 12% reduce 0%
>> 10/07/02 13:25:06 INFO mapred.JobClient:  map 13% reduce 0%
>> 10/07/02 13:25:09 INFO mapred.JobClient:  map 15% reduce 0%
>> 10/07/02 13:25:12 INFO mapred.JobClient:  map 16% reduce 0%
>> 10/07/02 13:25:15 INFO mapred.JobClient:  map 18% reduce 0%
>> 10/07/02 13:25:18 INFO mapred.JobClient:  map 19% reduce 0%
>> 10/07/02 13:25:21 INFO mapred.JobClient:  map 20% reduce 0%
>> 10/07/02 13:25:24 INFO mapred.JobClient:  map 22% reduce 0%
>> 10/07/02 13:25:27 INFO mapred.JobClient:  map 23% reduce 0%
>> 10/07/02 13:25:30 INFO mapred.JobClient:  map 24% reduce 0%
>> 10/07/02 13:34:55 INFO mapred.JobClient:  map 12% reduce 0%
>> 10/07/02 13:35:03 INFO mapred.JobClient: Task Id :
>> attempt_201007021322_0001_m_000002_0, Status : FAILED
>> Task attempt_201007021322_0001_m_000002_0 failed to report status for 601
>> seconds. Killing!
>> 10/07/02 13:35:03 INFO mapred.JobClient: Task Id :
>> attempt_201007021322_0001_m_000003_0, Status : FAILED
>> Task attempt_201007021322_0001_m_000003_0 failed to report status for 602
>> seconds. Killing!
>> 10/07/02 13:35:16 INFO mapred.JobClient:  map 17% reduce 0%
>> 10/07/02 13:35:19 INFO mapred.JobClient:  map 23% reduce 0%
>> 10/07/02 13:35:22 INFO mapred.JobClient:  map 24% reduce 0%
>> 10/07/02 13:35:46 INFO mapred.JobClient:  map 12% reduce 0%
>> 10/07/02 13:36:06 INFO mapred.JobClient: Task Id :
>> attempt_201007021322_0001_m_000000_0, Status : FAILED
>> Task attempt_201007021322_0001_m_000000_0 failed to report status for 602
>> seconds. Killing!
>> 10/07/02 13:36:07 INFO mapred.JobClient: Task Id :
>> attempt_201007021322_0001_m_000001_0, Status : FAILED
>> Task attempt_201007021322_0001_m_000001_0 failed to report status for 600
>> seconds. Killing!
>> 10/07/02 13:36:30 INFO mapred.JobClient:  map 13% reduce 0%
>> 10/07/02 13:36:33 INFO mapred.JobClient:  map 16% reduce 0%
>> 10/07/02 13:36:36 INFO mapred.JobClient:  map 18% reduce 0%
>> ....
>>
>> log :
>>
>> 2010-07-02 13:35:09,538 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
>> Initializing JVM Metrics with processName=MAP, sessionId=
>> 2010-07-02 13:35:10,433 INFO org.apache.hadoop.mapred.MapTask:
>> numReduceTasks: 1
>> 2010-07-02 13:35:10,459 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb =
>> 100
>> 2010-07-02 13:35:11,073 INFO org.apache.hadoop.mapred.MapTask: data buffer
>> = 79691776/99614720
>> 2010-07-02 13:35:11,074 INFO org.apache.hadoop.mapred.MapTask: record
>> buffer = 262144/327680
>> 2010-07-02 13:35:11,308 WARN org.apache.hadoop.util.NativeCodeLoader:
>> Unable to load native-hadoop library for your platform... using builtin-java
>> classes where applicable
>> 2010-07-02 13:35:11,311 INFO org.apache.hadoop.io.compress.CodecPool: Got
>> brand-new decompressor
>> 2010-07-02 13:35:16,153 INFO org.apache.hadoop.mapred.MapTask: Spilling map
>> output: record full = true
>> 2010-07-02 13:35:16,154 INFO org.apache.hadoop.mapred.MapTask: bufstart =
>> 0; bufend = 26214400; bufvoid = 99614720
>> 2010-07-02 13:35:16,154 INFO org.apache.hadoop.mapred.MapTask: kvstart = 0;
>> kvend = 262144; length = 327680
>>
>>
>> Look forward to any reply.
>>
>> Best Regards,
>> Cindy
>>
>>
>
>
>-- 
>Best Regards,
>Jim, Jun Jie Yu
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message