spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ulanov, Alexander" <alexander.ula...@hpe.com>
Subject Too many executors are created
Date Tue, 29 Sep 2015 19:23:53 GMT
Dear Spark developers,

I have created a simple Spark application for spark submit. It calls a machine learning library
from Spark MLlib that is executed in a number of iterations that correspond to the same number
of task in Spark. It seems that Spark creates an executor for each task and then removes it.
The following messages indicate this in my log:

15/09/29 12:21:02 INFO AppClient$ClientEndpoint: Executor updated: app-20150929120924-0000/24463
is now RUNNING
15/09/29 12:21:02 INFO AppClient$ClientEndpoint: Executor updated: app-20150929120924-0000/24463
is now EXITED (Command exited with code 1)
15/09/29 12:21:02 INFO SparkDeploySchedulerBackend: Executor app-20150929120924-0000/24463
removed: Command exited with code 1
15/09/29 12:21:02 INFO SparkDeploySchedulerBackend: Asked to remove non-existent executor
24463
15/09/29 12:21:02 INFO AppClient$ClientEndpoint: Executor added: app-20150929120924-0000/24464
on worker-20150929120330-16.111.35.101-46374 (16.111.35.101:46374) with 12 cores
15/09/29 12:21:02 INFO SparkDeploySchedulerBackend: Granted executor ID app-20150929120924-0000/24464
on hostPort 16.111.35.101:46374 with 12 cores, 30.0 GB RAM
15/09/29 12:21:02 INFO AppClient$ClientEndpoint: Executor updated: app-20150929120924-0000/24464
is now LOADING
15/09/29 12:21:02 INFO AppClient$ClientEndpoint: Executor updated: app-20150929120924-0000/24464
is now RUNNING
15/09/29 12:21:02 INFO AppClient$ClientEndpoint: Executor updated: app-20150929120924-0000/24464
is now EXITED (Command exited with code 1)
15/09/29 12:21:02 INFO SparkDeploySchedulerBackend: Executor app-20150929120924-0000/24464
removed: Command exited with code 1
15/09/29 12:21:02 INFO SparkDeploySchedulerBackend: Asked to remove non-existent executor
24464
15/09/29 12:21:02 INFO AppClient$ClientEndpoint: Executor added: app-20150929120924-0000/24465
on worker-20150929120330-16.111.35.101-46374 (16.111.35.101:46374) with 12 cores
15/09/29 12:21:02 INFO SparkDeploySchedulerBackend: Granted executor ID app-20150929120924-0000/24465
on hostPort 16.111.35.101:46374 with 12 cores, 30.0 GB RAM
15/09/29 12:21:02 INFO AppClient$ClientEndpoint: Executor updated: app-20150929120924-0000/24465
is now LOADING
15/09/29 12:21:02 INFO AppClient$ClientEndpoint: Executor updated: app-20150929120924-0000/24465
is now EXITED (Command exited with code 1)
15/09/29 12:21:02 INFO SparkDeploySchedulerBackend: Executor app-20150929120924-0000/24465
removed: Command exited with code 1
15/09/29 12:21:02 INFO SparkDeploySchedulerBackend: Asked to remove non-existent executor
24465
15/09/29 12:21:02 INFO AppClient$ClientEndpoint: Executor added: app-20150929120924-0000/24466
on worker-20150929120330-16.111.35.101-46374 (16.111.35.101:46374) with 12 cores
15/09/29 12:21:02 INFO SparkDeploySchedulerBackend: Granted executor ID app-20150929120924-0000/24466
on hostPort 16.111.35.101:46374 with 12 cores, 30.0 GB RAM
15/09/29 12:21:02 INFO AppClient$ClientEndpoint: Executor updated: app-20150929120924-0000/24466
is now LOADING
15/09/29 12:21:02 INFO AppClient$ClientEndpoint: Executor updated: app-20150929120924-0000/24466
is now RUNNING

It end up creating and removing thousands of executors. Is this a normal behavior?

If I run the same code within spark-shell, this does not happen. Could you suggest what might
be wrong in my setting?

Best regards, Alexander

Mime
View raw message