spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Will Benton <wi...@redhat.com>
Subject Re: Profiling Spark tests with YourKit (or something else)
Date Tue, 15 Jul 2014 03:43:46 GMT
Sure thing:

   https://issues.apache.org/jira/browse/SPARK-2486
   https://github.com/apache/spark/pull/1413

best,
wb


----- Original Message -----
> From: "Aaron Davidson" <ilikerps@gmail.com>
> To: dev@spark.apache.org
> Sent: Monday, July 14, 2014 8:38:16 PM
> Subject: Re: Profiling Spark tests with YourKit (or something else)
> 
> Would you mind filing a JIRA for this? That does sound like something bogus
> happening on the JVM/YourKit level, but this sort of diagnosis is
> sufficiently important that we should be resilient against it.
> 
> 
> On Mon, Jul 14, 2014 at 6:01 PM, Will Benton <willb@redhat.com> wrote:
> 
> > ----- Original Message -----
> > > From: "Aaron Davidson" <ilikerps@gmail.com>
> > > To: dev@spark.apache.org
> > > Sent: Monday, July 14, 2014 5:21:10 PM
> > > Subject: Re: Profiling Spark tests with YourKit (or something else)
> > >
> > > Out of curiosity, what problems are you seeing with Utils.getCallSite?
> >
> > Aaron, if I enable call site tracking or CPU profiling in YourKit, many
> > (but not all) Spark test cases will NPE on the line filtering out
> > "getStackTrace" from the stack trace (this is Utils.scala:812 in the
> > current master).  I'm not sure if this is a consequence of
> > Thread#getStackTrace including bogus frames when running instrumented or if
> > whatever instrumentation YourKit inserts relies on assumptions that don't
> > always hold for Scala code.
> >
> >
> > best,
> > wb
> >
> 

Mime
View raw message