ambari-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From cs user <acldstk...@gmail.com>
Subject Fwd: pyspark - spark history server
Date Fri, 05 Feb 2016 14:14:12 GMT
Hi All,

I sent this to the spark mailing list, sending it here as well in the hope
someone might know why this is not working....

Thanks,

Ryan
---------- Forwarded message ----------
From: cs user <acldstkusr@gmail.com>
Date: Fri, Feb 5, 2016 at 2:08 PM
Subject: pyspark - spark history server
To: user@spark.apache.org


Hi All,

I'm having trouble getting a job to use the spark history server. We have a
cluster configured with Ambari, if I run the job from one of the nodes
within the Ambari configured cluster, everything works fine, the job
appears in the spark history server.

If I configure a client external to the cluster, running the same job, the
history server is not used.

When the job completes fine, I see these lines appear in the log:


16/02/05 11:57:22 INFO history.YarnHistoryService: Starting
YarnHistoryService for application application_1453893909110_0108 attempt
Some(appattempt_1453893909110_0108_000001); state=1; endpoint=
http://somehost:8188/ws/v1/timeline/; bonded to ATS=false; listening=false;
batchSize=10; flush count=0; total number queued=0, processed=0; attempted
entity posts=0 successful entity posts=0 failed entity posts=0; events
dropped=0; app start event received=false; app end event received=false;
16/02/05 11:57:22 INFO history.YarnHistoryService: Spark events will be
published to the Timeline service at http://somehost:8188/ws/v1/timeline/


On the client which is external to the cluster, these lines do not appear
in the logs. I have printed out spark context and attempted to match what
is configured on the working job, with the failing job, all seems fine.

These are the job settings:

conf.set('spark.speculation','true')
conf.set('spark.dynamicAllocation.enabled','false')
conf.set('spark.shuffle.service.enabled','false')
conf.set('spark.executor.instances', '4')
conf.set('spark.akka.threads','4')
conf.set('spark.dynamicAllocation.initialExecutors','4')
conf.set('spark.history.provider','org.apache.spark.deploy.yarn.history.YarnHistoryProvider')
conf.set('spark.yarn.services','org.apache.spark.deploy.yarn.history.YarnHistoryService')
conf.set('spark.history.ui.port','18080')
conf.set('spark.driver.extraJavaOptions','-Dhdp.version=2.3.4.0-3485')
conf.set('spark.yarn.containerLauncherMaxThreads','25')
conf.set('spark.yarn.driver.memoryOverhead','384')
conf.set('spark.yarn.executor.memoryOverhead','384')
conf.set('spark.yarn.historyServer.address','somehost:18080')
conf.set('spark.yarn.max.executor.failures','3')
conf.set('spark.yarn.preserve.staging.files','false')
conf.set('spark.yarn.queue','default')
conf.set('spark.yarn.scheduler.heartbeat.interval-ms','5000')
conf.set('spark.yarn.submit.file.replication','3')
conf.set('spark.yarn.am.extraJavaOptions','-Dhdp.version=2.3.4.0-3485')
conf.set('spark.blockManager.port','9096')
conf.set('spark.driver.port','9095')
conf.set('spark.fileserver.port','9097')

I am using the following tar.gz file to install spark on the node external
to the cluster:

http://www.apache.org/dyn/closer.lua/spark/spark-1.5.2/spark-1.5.2-bin-hadoop2.6.tgz

Will this version of spark have everything required to talk correctly to
yarn and the spark history service?

So it comes down to, the spark context settings appear to be exactly the
same, there are no errors in the logs pointing to the job not being able to
connect to anything, none of the ports are blocked, why is this not working
when run external to the cluster....?

There is no kerberos security configured on the cluster.

Thanks!

Mime
View raw message