This got resolved. I had to set hostname on all machines using:
sudo hostname -v hadoop-master
etc.
Thanks,
Sahas
On Tue, Feb 27, 2018 at 5:36 PM, Sahasranaman M S <sahasranaman@ideas2it.com
> wrote:
> Hi,
>
> I setup 3 machine Hadoop cluster using the instructions in:
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/
> ClusterSetup.html
>
> When hadoop and yarn are running, if I run 'jps' on the three nodes, I
> see on hadoop-master:
>
> 19696 SecondaryNameNode
> 20068 ResourceManager
> 20374 NodeManager
> 19383 DataNode
> 19257 NameNode
>
> and on the two slaves:
>
> 17812 DataNode
> 18164 NodeManager
>
> When I run inject using below command, it mostly succeeds, shows number
> of urls injected.
> apache-nutch-1.14/runtime/deploy/bin/nutch inject
> /nutch/crawl/crawlDb /nutch/urls
>
> When I run generate, it very often fails with a connection refused to an
> unknown port. Sometimes, the 'select from crawlDb' job of generate
> succeeds, and the 'partition' job fails. Sometimes, the 'select from
> crawlDb' fails. All of them with the same connection refused to a random
> port in localhost. The port number keeps changing with every attempt.
> apache-nutch-1.14/runtime/deploy/bin/nutch generate
> /nutch/crawl/crawlDb /nutch/crawl/segments
>
> I have checked the configs multiple times, couldn't find the issue.
> Could anyone please give me a pointer?
>
> 2018-02-27 11:09:51,955 INFO [main] mapreduce.Job
> (Job.java:monitorAndPrintJob(1619)) - Running job: job_1519727841906_0003
> 2018-02-27 11:15:55,004 INFO [main] mapreduce.Job
> (Job.java:monitorAndPrintJob(1640)) - Job job_1519727841906_0003 running
> in uber mode : false
> 2018-02-27 11:15:55,006 INFO [main] mapreduce.Job
> (Job.java:monitorAndPrintJob(1647)) - map 0% reduce 0%
> 2018-02-27 11:15:55,016 INFO [main] mapreduce.Job
> (Job.java:monitorAndPrintJob(1660)) - Job job_1519727841906_0003 failed
> with state FAILED due to: Application application_1519727841906_0003 failed
> 2 times due to Error launching appattempt_1519727841906_0003_000002. Got
> exception: java.net.ConnectException: *Call From localhost/127.0.0.1
> <http://127.0.0.1> to localhost:39093 failed on connection exception:
> java.net.ConnectException: Connection refused*; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
>
> Thanks,
> Sahas
>
|