spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sea" <261810...@qq.com>
Subject InvalidAuxServiceException in dynamicAllocation
Date Wed, 18 Mar 2015 02:07:05 GMT
Hi, all:


Spark1.3.0 hadoop2.2.0


I put the following params in the spark-defaults.conf 


spark.dynamicAllocation.enabled true
spark.dynamicAllocation.minExecutors 20
spark.dynamicAllocation.maxExecutors 300
spark.dynamicAllocation.executorIdleTimeout 300
spark.shuffle.service.enabled true‍



I started the thriftserver and do a query. Exception happened!
I find it in JIRA https://issues.apache.org/jira/browse/SPARK-5759‍ 
It says fixed version 1.3.0


Caused by: org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:spark_shuffle
does not exist 	at sun.reflect.GeneratedConstructorAccessor28.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:513) 	at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:152)
	at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
	at org.apache.hadoop.yarn.client.api.impl.NMClientImpl.startContainer(NMClientImpl.java:203)
	at org.apache.spark.deploy.yarn.ExecutorRunnable.startContainer(ExecutorRunnable.scala:113)
	... 4 more‍
Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message