hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shengkai Zhu" <geniusj...@gmail.com>
Subject Re: hadoop hanging (probably misconfiguration) assistance
Date Thu, 11 Sep 2008 07:36:16 GMT
Logs may probably tell what happened.

On Thu, Sep 11, 2008 at 3:20 PM, <damien.cooke@internode.on.net> wrote:

> Hi All,
> I have been trying to move from pseudo distributed hadoop cluster which
> worked perfectly well, to a real hadoop cluster.  I was able to execute
> the wordcount example on my pseudo cluster but my real cluster hangs at
> this point:
>
> # bin/hadoop jar hadoop*jar wordcount /myinput /myoutput
> 08/09/10 17:10:30 INFO mapred.FileInputFormat: Total input paths to process
> : 2
> 08/09/10 17:10:30 INFO mapred.FileInputFormat: Total input paths to process
> : 2
> 08/09/10 17:10:31 INFO mapred.JobClient: Running job: job_200809101706_0001
> 08/09/10 17:10:32 INFO mapred.JobClient:  map 0% reduce 0%
>
> The machines are doing nothing ie all processes at 0.0%
>
> I have changed the configuration a couple of times to see where the issue
> lies.  Currently I have 2 machines in the cluster the namenode and
> the jobtracker one one machine with the datanode on a separate machine.
>
> I have moved from named nodes to ip addresses with negligible improvement.
> The only errors in the logfiles are regarding flushing for log4j so I did
> not consider that to be relevant.
>
> If anyone has seen this or has any ideas where I might find the source of
> my issues I would be grateful.
>
> Regards
> Damien
>
> # cat hadoop-site.xml
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>
> <!-- Put site-specific property overrides in this file. -->
>
> <configuration>
>
>  <property>
>        <name>mapred.task.timeout</name>
>        <value>6000</value>
>  <description>The number of milliseconds before a task will be
>  terminated if it neither reads an input, writes an output, nor
>  updates its status string.
>  </description>
> </property>
>
> <property>
>    <name>fs.default.name</name>
>    <value>hdfs://10.7.3.164:54130/</value>
>  </property>
>
>   <property>
>    <name>dfs.replication</name>
>    <value>1</value>
>  </property>
>
>   <property>
>    <name>hadoop.logfile.size</name>
>    <value>1000000</value>
>  </property>
>
>   <property>
>    <name>hadoop.logfile.count</name>
>    <value>2</value>
>  </property>
>
>   <property>
>    <name>io.sort.mb</name>
>    <value>25</value>
>  </property>
>
>  <property>
>    <name>dfs.block.size</name>
>    <value>8388608</value>
>  </property>
>
>  <property>
>    <name>dfs.namenode.handler.count</name>
>    <value>5</value>
>  </property>
>
>  <property>
>    <name>mapred.job.tracker</name>
>    <value>10.7.3.164:54131</value>
>  </property>
>
>   <property>
>    <name>mapred.job.tracker.handler.count</name>
>    <value>3</value>
>  </property>
>
>  <property>
>    <name>mapred.tasktracker.map.tasks.maximum</name>
>    <value>2</value>
>  </property>
>
>  <property>
>    <name>mapred.tasktracker.reduce.tasks.maximum</name>
>    <value>2</value>
>  </property>
>
>   <property>
>    <name>mapred.child.java.opts</name>
>    <value>-Xmx128m</value>
>  </property>
>
>  <property>
>    <name>mapred.map.tasks.speculative.execution</name>
>    <value>false</value>
>  </property>
>
>  <property>
>    <name>mapred.reduce.tasks.speculative.execution</name>
>    <value>false</value>
>  </property>
>
>  <property>
>    <name>mapred.submit.replication</name>
>    <value>1</value>
>  </property>
>
>  <property>
>    <name>tasktracker.http.threads</name>
>    <value>4</value>
>  </property>
>
> </configuration>
>
>
>


-- 

朱盛凯

Jash Zhu

复旦大学软件学院

Software School, Fudan University
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message