spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vijay Kiran <m...@vijaykiran.com>
Subject Re: Spark metrics when running with YARN?
Date Tue, 30 Aug 2016 12:52:15 GMT
Hi Otis,

Did you check the REST API as documented in  http://spark.apache.org/docs/latest/monitoring.html


Regards,
Vijay

> On 30 Aug 2016, at 14:43, Otis Gospodnetić <otis.gospodnetic@gmail.com> wrote:
> 
> Hi Mich and Vijay,
> 
> Thanks!  I forgot to include an important bit - I'm looking for a programmatic way to
get Spark metrics when running Spark under YARN - so JMX or API of some kind.
> 
> Thanks,
> Otis
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> 
> 
> On Tue, Aug 30, 2016 at 6:59 AM, Mich Talebzadeh <mich.talebzadeh@gmail.com> wrote:
> Spark UI regardless of deployment mode Standalone, yarn etc runs on port 4040 by default
that can be accessed directly
> 
> Otherwise one can specify a specific port with --conf "spark.ui.port=55555" for example
55555
> 
> HTH
> 
> Dr Mich Talebzadeh
>  
> LinkedIn  https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>  
> http://talebzadehmich.wordpress.com
> 
> Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage
or destruction of data or any other property which may arise from relying on this email's
technical content is explicitly disclaimed. The author will in no case be liable for any monetary
damages arising from such loss, damage or destruction.
>  
> 
> On 30 August 2016 at 11:48, Vijay Kiran <mail@vijaykiran.com> wrote:
> 
> From Yarm RM UI, find the spark application Id, and in the application details, you can
click on the “Tracking URL” which should give you the Spark UI.
> 
> ./Vijay
> 
> > On 30 Aug 2016, at 07:53, Otis Gospodnetić <otis.gospodnetic@gmail.com> wrote:
> >
> > Hi,
> >
> > When Spark is run on top of YARN, where/how can one get Spark metrics?
> >
> > Thanks,
> > Otis
> > --
> > Monitoring - Log Management - Alerting - Anomaly Detection
> > Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> >
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
> 
> 
> 


---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message