nutch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sahasranaman M S <sahasrana...@ideas2it.com>
Subject Re: Random 'Connection Refused' errors when running Nutch 1.14 on Hadoop 3.0.0
Date Tue, 27 Feb 2018 15:30:16 GMT
This got resolved.  I had to set hostname on all machines using:
    sudo hostname -v hadoop-master
etc.

Thanks,
Sahas

On Tue, Feb 27, 2018 at 5:36 PM, Sahasranaman M S <sahasranaman@ideas2it.com
> wrote:

> Hi,
>
> I setup 3 machine Hadoop cluster using the instructions in:
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/
> ClusterSetup.html
>
> When hadoop and yarn are running,  if I run 'jps' on the three nodes, I
> see on hadoop-master:
>
> 19696 SecondaryNameNode
> 20068 ResourceManager
> 20374 NodeManager
> 19383 DataNode
> 19257 NameNode
>
> and on the two slaves:
>
> 17812 DataNode
> 18164 NodeManager
>
> When I run inject using below command,  it mostly succeeds, shows number
> of urls injected.
>     apache-nutch-1.14/runtime/deploy/bin/nutch inject
> /nutch/crawl/crawlDb  /nutch/urls
>
> When I run generate,  it very often fails with a connection refused to an
> unknown port.  Sometimes,  the 'select from crawlDb' job of generate
> succeeds,  and the 'partition' job fails.  Sometimes,  the 'select from
> crawlDb' fails.  All of them with the same connection refused to a random
> port in localhost.  The port number keeps changing with every attempt.
>     apache-nutch-1.14/runtime/deploy/bin/nutch generate
> /nutch/crawl/crawlDb /nutch/crawl/segments
>
> I have checked the configs multiple times,  couldn't find the issue.
> Could anyone please give me a pointer?
>
> 2018-02-27 11:09:51,955 INFO  [main] mapreduce.Job
> (Job.java:monitorAndPrintJob(1619)) - Running job: job_1519727841906_0003
> 2018-02-27 11:15:55,004 INFO  [main] mapreduce.Job
> (Job.java:monitorAndPrintJob(1640)) - Job job_1519727841906_0003 running
> in uber mode : false
> 2018-02-27 11:15:55,006 INFO  [main] mapreduce.Job
> (Job.java:monitorAndPrintJob(1647)) -  map 0% reduce 0%
> 2018-02-27 11:15:55,016 INFO  [main] mapreduce.Job
> (Job.java:monitorAndPrintJob(1660)) - Job job_1519727841906_0003 failed
> with state FAILED due to: Application application_1519727841906_0003 failed
> 2 times due to Error launching appattempt_1519727841906_0003_000002. Got
> exception: java.net.ConnectException: *Call From localhost/127.0.0.1
> <http://127.0.0.1> to localhost:39093 failed on connection exception:
> java.net.ConnectException: Connection refused*; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
>
> Thanks,
> Sahas
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message