hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From derevo <...@inbox.ru>
Subject two servers nutch hadoop
Date Mon, 30 Apr 2007 15:02:51 GMT

hi, when i try

[nutch@master ~/search]$ bin/nutch crawl urls -dir crawled -depth 3

i have this error in log file 

master server

==> hadoop-nutch-tasktracker-master.log <==
2007-04-30 08:53:52,318 WARN  mapred.TaskRunner - task_0002_r_000000_0 copy
failed: task_0002_m_000004_0 from slave1
2007-04-30 08:53:52,319 WARN  mapred.TaskRunner - java.io.IOException: File
/home/nutch/fs/mapred/local/task_0002_r_000000_0/map_4.out-6 not created
        at
org.apache.hadoop.mapred.ReduceTaskRunner$MapOutputCopier.copyOutput(ReduceTaskRunner.java:322)
        at
org.apache.hadoop.mapred.ReduceTaskRunner$MapOutputCopier.run(ReduceTaskRunner.java:274)

2007-04-30 08:53:52,319 WARN  mapred.TaskRunner - task_0002_r_000000_0
adding host slave1 to penalty box, next contact in 73 seconds


==> hadoop-nutch-secondarynamenode-master.log <==
2007-04-30 08:47:31,766 WARN  NameNode.Secondary - Checkpoint done. Image
Size:16 Edit Size:1055 New Image Size:941

 slave1 server 
2007-04-30 08:55:02,493 WARN  mapred.TaskRunner - task_0001_r_000000_0 copy
failed: task_0001_m_000015_0 from master
2007-04-30 08:55:02,494 WARN  mapred.TaskRunner - java.io.IOException: File
/home/nutch/fs/mapred/local/task_0001_r_000000_0/map_15.out-1 not created
        at
org.apache.hadoop.mapred.ReduceTaskRunner$MapOutputCopier.copyOutput(ReduceTaskRunner.java:322)
        at
org.apache.hadoop.mapred.ReduceTaskRunner$MapOutputCopier.run(ReduceTaskRunner.java:274)

2007-04-30 08:55:02,494 WARN  mapred.TaskRunner - task_0001_r_000000_0
adding host master to penalty box, next contact in 84 seconds

master# netstat -an | grep LIST
tcp4       0      0  *.50050                *.*                    LISTEN
tcp4       0      0  *.50060                *.*                    LISTEN
tcp4       0      0  *.50030                *.*                    LISTEN
tcp4       0      0  *.50090                *.*                    LISTEN
tcp4       0      0  216.32.77.34.9001      *.*                    LISTEN
tcp4       0      0  *.50075                *.*                    LISTEN
tcp4       0      0  *.50010                *.*                    LISTEN
tcp4       0      0  216.32.77.34.9000      *.*                    LISTEN
tcp4       0      0  *.50070                *.*                    LISTEN
tcp4       0      0  *.21                   *.*                    LISTEN
tcp4       0      0  127.0.0.1.25           *.*                    LISTEN
tcp4       0      0  *.22                   *.*                    LISTEN
tcp6       0      0  *.22                   *.*                    LISTEN



slave1# netstat -an | grep LIST
tcp4       0      0  *.50050                *.*                    LISTEN
tcp4       0      0  *.50060                *.*                    LISTEN
tcp4       0      0  *.50075                *.*                    LISTEN
tcp4       0      0  *.50010                *.*                    LISTEN
tcp4       0      0  *.21                   *.*                    LISTEN
tcp4       0      0  127.0.0.1.25           *.*                    LISTEN
tcp4       0      0  *.22                   *.*                    LISTEN
tcp6       0      0  *.22                   *.*                    LISTEN


[nutch@master ~/search]$ bin/slaves.sh uptime
xxx.32.77.xxx:  8:56AM  up 2 days,  2:50, 2 users, load averages: 0.08,
0.14, 0.11
xxx.21.41.xxx:  8:57AM  up 2 days,  2:50, 2 users, load averages: 0.04,
0.08, 0.04

OS: freebsd 6.2



-- 
View this message in context: http://www.nabble.com/two-servers--nutch--hadoop-tf3670129.html#a10254968
Sent from the Hadoop Users mailing list archive at Nabble.com.


Mime
View raw message