spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetić <otis.gospodne...@gmail.com>
Subject Re: Spark metrics when running with YARN?
Date Tue, 30 Aug 2016 12:43:26 GMT
Hi Mich and Vijay,

Thanks!  I forgot to include an important bit - I'm looking for a
*programmatic* way to get Spark metrics when running Spark under YARN - so
JMX or API of some kind.

Thanks,
Otis
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/


On Tue, Aug 30, 2016 at 6:59 AM, Mich Talebzadeh <mich.talebzadeh@gmail.com>
wrote:

> Spark UI regardless of deployment mode Standalone, yarn etc runs on port
> 4040 by default that can be accessed directly
>
> Otherwise one can specify a specific port with --conf "spark.ui.port=55555"
> for example 55555
>
> HTH
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
> On 30 August 2016 at 11:48, Vijay Kiran <mail@vijaykiran.com> wrote:
>
>>
>> From Yarm RM UI, find the spark application Id, and in the application
>> details, you can click on the “Tracking URL” which should give you the
>> Spark UI.
>>
>> ./Vijay
>>
>> > On 30 Aug 2016, at 07:53, Otis Gospodnetić <otis.gospodnetic@gmail.com>
>> wrote:
>> >
>> > Hi,
>> >
>> > When Spark is run on top of YARN, where/how can one get Spark metrics?
>> >
>> > Thanks,
>> > Otis
>> > --
>> > Monitoring - Log Management - Alerting - Anomaly Detection
>> > Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>>
>>
>

Mime
View raw message