spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-12803) Consider adding ability to profile specific instances of executors in spark
Date Wed, 13 Jan 2016 22:04:39 GMT

    [ https://issues.apache.org/jira/browse/SPARK-12803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097097#comment-15097097
] 

Sean Owen commented on SPARK-12803:
-----------------------------------

What does it hurt though? enabling a profiling agent in the JVM doesn't do anything if nothing
connects.

> Consider adding ability to profile specific instances of executors in spark
> ---------------------------------------------------------------------------
>
>                 Key: SPARK-12803
>                 URL: https://issues.apache.org/jira/browse/SPARK-12803
>             Project: Spark
>          Issue Type: Improvement
>          Components: Java API
>            Reporter: Rajesh Balamohan
>
> It would be useful to profile specific instances of executors as opposed to adding profiler
details to all executors via "spark.executor.extraJavaOptions".  
> Setting the number of executors to just 1 and profiling wouldn't be much useful (in some
cases, most of the time with single executor mode would be spent in terms of reading data
from remote node).  At the same time, setting profiling option to all executors could just
create too many number of snapshots; making it harder to analyze.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message