Also, the cluster is on AWS. Security group set to allow all inbound and
outbound traffic...
Any ideas?...
On 08/16/2017 12:37 PM, Michael Chen wrote:
>
> Hi,
>
> I've run into a ZooKeeper connection error during the execution of a
> Nutch hadoop job. The tasks stall on connection error to ZooKeeper
> server. Here's what I know:
>
> 1. ZK connection error is the only known problem, other logs report no
> issue
>
> 2. Error message on YARN NodeManager on one of the slaves is:
>
> 2017-08-16 19:03:42,280 INFO [main-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn:
Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate
using SASL (unknown error)
> 2017-08-16 19:03:42,281 WARN [main-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn:
Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
> java.net.ConnectException: Connection refused
>
> The connection keeps failing until it hits the 10min limit and the
> task fails.
>
> 3. ZooKeeper Server is deployed only on master
>
> 4. Cluster managed by CloudEra Manager 5.12.
>
> Could a configuration on Nutch side or CloudEra Manager side be
> missing? There are no ZK servers on the slaves and the NodeManager
> should be connecting to the ZK server on the master, instead of
> localhost:2181.
>
> Any suggestion or help is greatly appreciated!
>
> Thank you,
>
> Michael
>
|