incubator-kato-spec mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicholas Sterling <Nicholas.Sterl...@Sun.COM>
Subject Re: Tool Prioritisation
Date Tue, 13 Jan 2009 08:02:38 GMT


Steve Poole wrote:
> Greetings.
>
> We haven't quite closed on the list of example tools but I think its
> important that we start to prioritise the list we have already.  I'd like to
> spend the rest of the month expanding the requirements we have for our top 3
> or 4  tools.   I would ask you all to respond with your view on the
> prioritisation of the list below.   Highest being most important.
> Additionally , if you have an opinion, please indicate as a comment if the
> tool seem to be relying on  a technical aspect  that is not feasible to
> implement.  (For instance the Native Memory Analysis Tool proposal assumes
> that getting access to memory allocation info from a dump is practicable -
> we have some work to do to figure out what is really achieveable)
>   
...
> 1: Process Explorer
>
> An Eclipse plugin which allows presentation, navigation and simple querying
> of the elements of a Process Dump.   This tool will demonstrate how to
> explore a dump in an efficient and high performing manner.   Key
> characteristics will include  fast startup time, handling of large
> quantities of data (including summarization), effective navigation to areas
> of interest.
>
>
> 2: Native Memory Analyser Tool
>
> A program that can retrieve native memory allocations by (or on behalf of) a
> Java Runtime and provide trace back to the  Java objects that hold the
> allocation reference.   The tool will be able to display what memory
> allocations exist, the contents of the allocation, and conservatively
> identify which entities hold references to that allocation.  Ideally this
> tool will be able to point to specific fields within Java objects that hold
> the references.   This tool will demonstrate the capabilities   of the API
> to find and display native memory  from a memory manager.    Key
> characteristics will include the performance of the API in  exhaustively
> scanning a dump (for memory allocation handles) and the ability to resolve
> an arbitrary location within the dump into a Java object or similar entity
>   
It appears that at least in our org, folks seem happy enough with 
libmalloc, etc. for debugging native memory problems.  So I would give 
this demo a somewhat lower priority, although I think it's a useful 
feature to offer.
>
> 3: Java Runtime Explorer
>
> Similar to the Process Explorer above this Eclipse plugin will allow the
> presentation, navigation and  simple querying of the elements of a Java
> Runtime dump. This tool will demonstrate how to explore a Java runtime dump
> in an efficient and high performing manner.   Ideally the plugin will also
> demonstrate the APIs ability to  provide some support for  virtualisation of
> Java runtime objects so that implementation specifics concerning objects
> within  the java.lang and java.util packages can be hidden.   Key
> characteristics will include  fast startup time, handling of large
> quantities of data (including summarization), effective navigation to areas
> of interest, useful abstraction of key Java object implementation specifics
>   
Top priority, since this will show people how to use the basic features.
#1
>
> 4:  Runtime Investigator
>
> A program that can examine a dump and provide guidance on common aliments.
> This tool will provide analysis modules that can report on such items as
> deadlock analysis,  heap occupancy etc The tool will provide extension
> points that will allow others to contribute new analysis modules.  Key
> characteristics of this tool will include handling large quantities of data
> efficiently  (probably via a query language of some type) , ensuring the API
> is generally consumable by programmers and ensuring the API provides the
> data that is actually required to analyze real problems.
>   
This could grow into something really cool, almost a separate layer of 
technology on top of the API.  As such I don't think I would do it 
first, but it would be cool.
#4
>
> 5: Java Runtime Trend Analyzer
>
> A tool that can compare multiple dumps and provide trend analysis.  This
> tool will provide analysis modules that can report on such items as  heap
> growth etc The tool will provide extension points that will allow others to
> contribute new analysis modules.  Key characteristics of this tool will
> include exercising  the creation of well formed dumps,  fast startup time,
> correlation between dump objects and handling large quantities of data
> efficiently  (probably via a query language of some type) , ensuring the API
> is generally consumable by programmers and ensuring the API provides the
> data that is actually required to analyze real problems.
>   
Yes, I think this will be a good test of whether we have the right 
interfaces and performs well, and there is a large class of problems 
that are best addressed by analyzing what has changed between two 
dumps.  Cool.
#3
>
> 6: Java Debug Interface (JDI) Connector
>
> An adapter that allows a Java debugger to interrogate the contents of a Java
> Runtime diagnostic artifact.  This connector will enable similar
> capabilities that exist today with other debuggers than can debug corefiles
> or similar process diagnostic artifacts.   This tool will demonstrate  key
> characteristics such as effective navigation to areas of interest, useful
> abstraction of key Java object implementation specifics and that the API
> provides  the data that required to analyze real problems.
>   
I'm not sure I understand what extra value this provides versus the 
explorer.  A debugger allows you to
 - explore state
 - modify state
 - control execution
The first is what the explorer does, and the last two don't apply to 
dump artifacts.  I'm sure I'm missing something...
>
> 7: Memory Analyser Tool (MAT) Adapter
>
> MAT (http://www.eclipse.org/mat/) is an open source project that consumes
> HPROF and DTFJ supported dumps.  MAT is designed to help find memory leaks
> and reduce memory consumption.  An adapter for MAT will be developed that
> allows MAT to consume HPROF and other dumps via the JSR 326 API.   Key
> characteristics of this adapter will include  handling large quantities of
> data efficiently,  useful abstraction of key Java object implementation
> specifics and  dump type identification.
>   
Sounds like a relatively easy project that would effectively address 
common problems -- tasty low-hanging fruit.
#2
>
> 8 : Retrospector
>
> One of the rationales behind the JSR-326 was complexity of failure &
> performance analysis increasing with CPU core count growth.  This tool will
> showcase this rationale. The tool will present a number of landmarks or
> checkpoints in the code and display  how their mutual temporal ordering and
> latency change (a) over time (b) depending on the number of threads (c)
> depending on the number of available CPU cores etc
>
> Checkpoints can be of different kinds:
>             - well-known events defined by JVMTI (monitor enter/exits,
> method entry/exits, class loads, code generation, thread lifecycle
> events,...)
>            - new kinds of events helping solve particular problems. For
> example, interpreted -> compiled code execution mode change for a method,
> reaching a safepoint (in Hotspot terms), native memory allocation in from
> JNI code, arbitrary checkpoint specified by a user as <method, bytecode
> position> pair (with optional additional refinements or filtering), etc.
>
> This tool will likely require additions within the JRE such that   when a
> checkpoint is reached by a thread, the JRE internally quickly logs this fact
> by adding a record <checkpoint ID, thread ID, timestamp, pc, ...> into a
> buffer, which, at appropriate time, is processed and made available via the
> JSR-326 API. [Note: The logging should be extremely efficient to minimize
> the "observer effect", which seems quite possible with thread-local
> checkpoint event buffer.].
>   
DTrace?
>        The retrospector tool will be able to load the 2 or more checkpoint
> logs and
>        - split up each of them into per-thread events
>        - visualize the checkpoint events mapped onto the timescale
>        - visualize the difference in temporal behavior between two threads
> within the same run of the analyzed app, between two threads within
> different runs on the analyzed app
>   
This sounds like an interesting tool, but a lot of work.  I think I'd 
save it for last.

Nicholas


Mime
View raw message