spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jürgen Thomann <juergen_thom...@linfre.de>
Subject Timeout between driver and application master (Thrift Server)
Date Mon, 04 Mar 2019 12:49:19 GMT
Hi,

I'm using the Spark Thrift Server and after some time the driver and 
application master are shutting down because of timeouts. There is a firewall 
in between and there is no traffic between them as it seems. Is there a way to 
configure TCP keep alive for the connection or some other way to make the 
firewall happy?

Environment:
CentOS 7, HDP 2.6.5 with Spark 2.3.0

The Error on the driver is "ERROR YarnClientSchedulerBackend: Yarn application 
has already exited with state finished" and a bit later there are some 
Exceptions with ClosedChannelException.

The application master has the following message:
WARN TransportChannelHandler: Exception in connection from <driver Host>
java.io.IOException: Connection timed out
... Stacktrace omitted
The messages are at the same time (same second, sadly no milliseconds in the 
logs).

Thanks,
Jürgen



---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message