From kato-spec-return-30-apmail-incubator-kato-spec-archive=incubator.apache.org@incubator.apache.org Sun Feb 01 21:34:11 2009 Return-Path: Delivered-To: apmail-incubator-kato-spec-archive@minotaur.apache.org Received: (qmail 17286 invoked from network); 1 Feb 2009 21:34:11 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 1 Feb 2009 21:34:11 -0000 Received: (qmail 68017 invoked by uid 500); 1 Feb 2009 21:34:11 -0000 Delivered-To: apmail-incubator-kato-spec-archive@incubator.apache.org Received: (qmail 68003 invoked by uid 500); 1 Feb 2009 21:34:11 -0000 Mailing-List: contact kato-spec-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: kato-spec@incubator.apache.org Delivered-To: mailing list kato-spec@incubator.apache.org Received: (qmail 67992 invoked by uid 99); 1 Feb 2009 21:34:11 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Feb 2009 13:34:11 -0800 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of spoole167@googlemail.com designates 74.125.46.158 as permitted sender) Received: from [74.125.46.158] (HELO yw-out-1718.google.com) (74.125.46.158) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Feb 2009 21:33:59 +0000 Received: by yw-out-1718.google.com with SMTP id 5so448572ywr.0 for ; Sun, 01 Feb 2009 13:33:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=ee6nkqyWgeU5MQSBGqg+LzDuUNWNX57M6Kw1dlsXGbQ=; b=Ifs6f3SgcVIMmPMUAZlcI6ev3fL/4J63NscLDD9pQxBbZKvUsst1Y9YwFUccWselqF fZDDLdbzv9X6OWgbX8TNAkQv2kCETOpK11YnXzpTT3Ao5yRxhXKBwrlA7heBr+uXG2be Cldc0DxoZdsfGNBUoXhQjcdro6EWucVUanE0c= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=YSV6AdIdVb3EkCTnDEnDMyZcK7NiM6AjvgRxtf2s3QdKlhZ4ulTn/wi5JczVG/JIzg oaqDq7TP1sDZei9aN7YCe5cclw3tBs0i4GA0l++8Fq+fhCp3vXqpxLNanflrzyMreogn /zhT8EXez+/K2zRtrPccttl0J0CeHmpM9Zz/c= MIME-Version: 1.0 Received: by 10.150.196.5 with SMTP id t5mr257623ybf.142.1233524017851; Sun, 01 Feb 2009 13:33:37 -0800 (PST) In-Reply-To: <497EE58A.3080809@glatschke.com> References: <496C4A9E.6080101@Sun.COM> <497EE58A.3080809@glatschke.com> Date: Sun, 1 Feb 2009 21:33:37 +0000 Message-ID: Subject: Re: Tool Prioritisation From: Steve Poole To: kato-spec@incubator.apache.org Content-Type: multipart/alternative; boundary=000e0cd47e54e7ef740461e23038 X-Virus-Checked: Checked by ClamAV on apache.org --000e0cd47e54e7ef740461e23038 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On Tue, Jan 27, 2009 at 10:44 AM, Christian Glatschke < christian@glatschke.com> wrote: > Hi all, > > here is my priorisation list: > > 1. Native Memory Analyser Tool > This should be a tool that collects runtime metrics about leaking and > growing objects. > It can be realized by collecting gc metrics, like heap usage in the Gen 0, > 1 and 2, if Gen 2 is growing constantly, > count the number of Objects and there instance names promoted to the Gen 2 > space, to identify the memory leak > After a crash the user should get something like a graphical snapshot about > leaking objects. > Hi Christian, this is not what I imagined the tool doing :-) I think you've invented a new one. Can we use your usecase as something to consider for the Trend Analysis tool? I think what I had imagined the NMAT tool doing was just an analysis of a single diagnostic artifact. > 2. Retrospector > > As you know I am working since more than four years in the area of bytecode > monitoring, but on life systems. > I would love to integrate bytecode manipulation during runtime to have > metrics and artefacts after a application crashes for any reasons. > This can include metrics about objects that allocates massive CPU time or > threads. We should explore your ideas in more detail. I'd really like to get an idea about the sort of data that is needed beyond what you've mentioned here. What are the key bits of data that are needed to be inserted into the bytecode to help solve customer problems? > > > 3. Thread Monitoring > > This should include runtime informations about all Threads, like a snapshot > with detailled thread dump informations, at the moment a application goes > down. > Maybe this is part of the Process Explorer? > Yes I think it is. > > Hope that helps, > > Cheers, > > Christian Glatschke > > > 1: Process Explorer > >> >>> An Eclipse plugin which allows presentation, navigation and simple >>> querying >>> of the elements of a Process Dump. This tool will demonstrate how to >>> explore a dump in an efficient and high performing manner. Key >>> characteristics will include fast startup time, handling of large >>> quantities of data (including summarization), effective navigation to >>> areas >>> of interest. >>> >>> >>> 2: Native Memory Analyser Tool >>> >>> A program that can retrieve native memory allocations by (or on behalf >>> of) >>> a >>> Java Runtime and provide trace back to the Java objects that hold the >>> allocation reference. The tool will be able to display what memory >>> allocations exist, the contents of the allocation, and conservatively >>> identify which entities hold references to that allocation. Ideally this >>> tool will be able to point to specific fields within Java objects that >>> hold >>> the references. This tool will demonstrate the capabilities of the >>> API >>> to find and display native memory from a memory manager. Key >>> characteristics will include the performance of the API in exhaustively >>> scanning a dump (for memory allocation handles) and the ability to >>> resolve >>> an arbitrary location within the dump into a Java object or similar >>> entity >>> >>> >>> > It appears that at least in our org, folks seem happy enough with >> libmalloc, etc. for debugging native memory problems. So I would give >> this >> demo a somewhat lower priority, although I think it's a useful feature to >> offer. >> >> > >> >> >> 3: Java Runtime Explorer >> >> >> >> Similar to the Process Explorer above this Eclipse plugin will allow >> the >> >> presentation, navigation and simple querying of the elements of a Java >> >> Runtime dump. This tool will demonstrate how to explore a Java runtime >> >> dump >> >> in an efficient and high performing manner. Ideally the plugin will >> also >> >> demonstrate the APIs ability to provide some support for >> virtualisation >> >> of >> >> Java runtime objects so that implementation specifics concerning >> objects >> >> within the java.lang and java.util packages can be hidden. Key >> >> characteristics will include fast startup time, handling of large >> >> quantities of data (including summarization), effective navigation to >> >> areas >> >> of interest, useful abstraction of key Java object implementation >> >> specifics >> >> >> >> >> Top priority, since this will show people how to use the basic features. >> #1 >> >> > >> >> >> 4: Runtime Investigator >> >> >> >> A program that can examine a dump and provide guidance on common >> aliments. >> >> This tool will provide analysis modules that can report on such items >> as >> >> deadlock analysis, heap occupancy etc The tool will provide extension >> >> points that will allow others to contribute new analysis modules. Key >> >> characteristics of this tool will include handling large quantities of >> >> data >> >> efficiently (probably via a query language of some type) , ensuring >> the >> >> API >> >> is generally consumable by programmers and ensuring the API provides >> the >> >> data that is actually required to analyze real problems. >> >> >> >> >> This could grow into something really cool, almost a separate layer of >> technology on top of the API. As such I don't think I would do it first, >> but it would be cool. >> #4 >> >> > >> >> >> 5: Java Runtime Trend Analyzer >> >> >> >> A tool that can compare multiple dumps and provide trend analysis. >> This >> >> tool will provide analysis modules that can report on such items as >> heap >> >> growth etc The tool will provide extension points that will allow >> others >> >> to >> >> contribute new analysis modules. Key characteristics of this tool will >> >> include exercising the creation of well formed dumps, fast startup >> time, >> >> correlation between dump objects and handling large quantities of data >> >> efficiently (probably via a query language of some type) , ensuring >> the >> >> API >> >> is generally consumable by programmers and ensuring the API provides >> the >> >> data that is actually required to analyze real problems. >> >> >> >> >> Yes, I think this will be a good test of whether we have the right >> interfaces and performs well, and there is a large class of problems that >> are best addressed by analyzing what has changed between two dumps. Cool. >> #3 >> >> > >> >> >> 6: Java Debug Interface (JDI) Connector >> >> >> >> An adapter that allows a Java debugger to interrogate the contents of a >> >> Java >> >> Runtime diagnostic artifact. This connector will enable similar >> >> capabilities that exist today with other debuggers than can debug >> >> corefiles >> >> or similar process diagnostic artifacts. This tool will demonstrate >> key >> >> characteristics such as effective navigation to areas of interest, >> useful >> >> abstraction of key Java object implementation specifics and that the >> API >> >> provides the data that required to analyze real problems. >> >> >> >> >> I'm not sure I understand what extra value this provides versus the >> explorer. A debugger allows you to >> - explore state >> - modify state >> - control execution >> The first is what the explorer does, and the last two don't apply to dump >> artifacts. I'm sure I'm missing something... >> >> > > When I ask Java developers how they would want to see data from a crashed > system presented this is usually top of the list > We'd be producing a connector that allowed a readonly view of the state of > the system. No ability to modify data and no concept > of allowing continued execution. Stuart Monteith is interested in this > area and I've asked him to post something more about the tool and its > usecases. (In fact I've asked all the team to start threads for fleshing > out these tools!) > > > > >> > >> >> >>> >> 7: Memory Analyser Tool (MAT) Adapter >>> >> >>> >> MAT (http://www.eclipse.org/mat/) is an open source project that >>> consumes >>> >> HPROF and DTFJ supported dumps. MAT is designed to help find memory >>> leaks >>> >> and reduce memory consumption. An adapter for MAT will be developed >>> that >>> >> allows MAT to consume HPROF and other dumps via the JSR 326 API. Key >>> >> characteristics of this adapter will include handling large >>> quantities of >>> >> data efficiently, useful abstraction of key Java object >>> implementation >>> >> specifics and dump type identification. >>> >> >>> >> >>> >>> >> > Sounds like a relatively easy project that would effectively address >> common >> > problems -- tasty low-hanging fruit. >> > #2 >> > >> >> >>> >> >>> >> 8 : Retrospector >>> >> >>> >> One of the rationales behind the JSR-326 was complexity of failure & >>> >> performance analysis increasing with CPU core count growth. This tool >>> >> will >>> >> showcase this rationale. The tool will present a number of landmarks >>> or >>> >> checkpoints in the code and display how their mutual temporal >>> ordering >>> >> and >>> >> latency change (a) over time (b) depending on the number of threads >>> (c) >>> >> depending on the number of available CPU cores etc >>> >> >>> >> Checkpoints can be of different kinds: >>> >> - well-known events defined by JVMTI (monitor enter/exits, >>> >> method entry/exits, class loads, code generation, thread lifecycle >>> >> events,...) >>> >> - new kinds of events helping solve particular problems. For >>> >> example, interpreted -> compiled code execution mode change for a >>> method, >>> >> reaching a safepoint (in Hotspot terms), native memory allocation in >>> from >>> >> JNI code, arbitrary checkpoint specified by a user as >> bytecode >>> >> position> pair (with optional additional refinements or filtering), >>> etc. >>> >> >>> >> This tool will likely require additions within the JRE such that >>> when a >>> >> checkpoint is reached by a thread, the JRE internally quickly logs >>> this >>> >> fact >>> >> by adding a record into >>> a >>> >> buffer, which, at appropriate time, is processed and made available >>> via >>> >> the >>> >> JSR-326 API. [Note: The logging should be extremely efficient to >>> minimize >>> >> the "observer effect", which seems quite possible with thread-local >>> >> checkpoint event buffer.]. >>> >> >>> >> >>> >>> >> > DTrace? >> > >> >> >>> >> The retrospector tool will be able to load the 2 or more >>> checkpoint >>> >> logs and >>> >> - split up each of them into per-thread events >>> >> - visualize the checkpoint events mapped onto the timescale >>> >> - visualize the difference in temporal behavior between two >>> threads >>> >> within the same run of the analyzed app, between two threads within >>> >> different runs on the analyzed app >>> >> >>> >> >>> >>> >> Steve Poole schrieb: > > On Tue, Jan 13, 2009 at 8:02 AM, Nicholas Sterling < >> Nicholas.Sterling@sun.com> wrote: >> >> >> >>> Steve Poole wrote: >>> >>> >>> >>>> Greetings. >>>> >>>> We haven't quite closed on the list of example tools but I think its >>>> important that we start to prioritise the list we have already. I'd >>>> like >>>> to >>>> spend the rest of the month expanding the requirements we have for our >>>> top >>>> 3 >>>> or 4 tools. I would ask you all to respond with your view on the >>>> prioritisation of the list below. Highest being most important. >>>> Additionally , if you have an opinion, please indicate as a comment if >>>> the >>>> tool seem to be relying on a technical aspect that is not feasible to >>>> implement. (For instance the Native Memory Analysis Tool proposal >>>> assumes >>>> that getting access to memory allocation info from a dump is practicable >>>> - >>>> we have some work to do to figure out what is really achieveable) >>>> >>>> >>>> >>>> >>> ... >>> >>> >>> >>>> 1: Process Explorer >>>> >>>> An Eclipse plugin which allows presentation, navigation and simple >>>> querying >>>> of the elements of a Process Dump. This tool will demonstrate how to >>>> explore a dump in an efficient and high performing manner. Key >>>> characteristics will include fast startup time, handling of large >>>> quantities of data (including summarization), effective navigation to >>>> areas >>>> of interest. >>>> >>>> >>>> 2: Native Memory Analyser Tool >>>> >>>> A program that can retrieve native memory allocations by (or on behalf >>>> of) >>>> a >>>> Java Runtime and provide trace back to the Java objects that hold the >>>> allocation reference. The tool will be able to display what memory >>>> allocations exist, the contents of the allocation, and conservatively >>>> identify which entities hold references to that allocation. Ideally >>>> this >>>> tool will be able to point to specific fields within Java objects that >>>> hold >>>> the references. This tool will demonstrate the capabilities of the >>>> API >>>> to find and display native memory from a memory manager. Key >>>> characteristics will include the performance of the API in exhaustively >>>> scanning a dump (for memory allocation handles) and the ability to >>>> resolve >>>> an arbitrary location within the dump into a Java object or similar >>>> entity >>>> >>>> >>>> >>>> >>> It appears that at least in our org, folks seem happy enough with >>> libmalloc, etc. for debugging native memory problems. So I would give >>> this >>> demo a somewhat lower priority, although I think it's a useful feature to >>> offer. >>> >>> >>> >>>> 3: Java Runtime Explorer >>>> >>>> Similar to the Process Explorer above this Eclipse plugin will allow the >>>> presentation, navigation and simple querying of the elements of a Java >>>> Runtime dump. This tool will demonstrate how to explore a Java runtime >>>> dump >>>> in an efficient and high performing manner. Ideally the plugin will >>>> also >>>> demonstrate the APIs ability to provide some support for >>>> virtualisation >>>> of >>>> Java runtime objects so that implementation specifics concerning objects >>>> within the java.lang and java.util packages can be hidden. Key >>>> characteristics will include fast startup time, handling of large >>>> quantities of data (including summarization), effective navigation to >>>> areas >>>> of interest, useful abstraction of key Java object implementation >>>> specifics >>>> >>>> >>>> >>>> >>> Top priority, since this will show people how to use the basic features. >>> #1 >>> >>> >>> >>>> 4: Runtime Investigator >>>> >>>> A program that can examine a dump and provide guidance on common >>>> aliments. >>>> This tool will provide analysis modules that can report on such items as >>>> deadlock analysis, heap occupancy etc The tool will provide extension >>>> points that will allow others to contribute new analysis modules. Key >>>> characteristics of this tool will include handling large quantities of >>>> data >>>> efficiently (probably via a query language of some type) , ensuring the >>>> API >>>> is generally consumable by programmers and ensuring the API provides the >>>> data that is actually required to analyze real problems. >>>> >>>> >>>> >>>> >>> This could grow into something really cool, almost a separate layer of >>> technology on top of the API. As such I don't think I would do it first, >>> but it would be cool. >>> #4 >>> >>> >>> >>>> 5: Java Runtime Trend Analyzer >>>> >>>> A tool that can compare multiple dumps and provide trend analysis. This >>>> tool will provide analysis modules that can report on such items as >>>> heap >>>> growth etc The tool will provide extension points that will allow others >>>> to >>>> contribute new analysis modules. Key characteristics of this tool will >>>> include exercising the creation of well formed dumps, fast startup >>>> time, >>>> correlation between dump objects and handling large quantities of data >>>> efficiently (probably via a query language of some type) , ensuring the >>>> API >>>> is generally consumable by programmers and ensuring the API provides the >>>> data that is actually required to analyze real problems. >>>> >>>> >>>> >>>> >>> Yes, I think this will be a good test of whether we have the right >>> interfaces and performs well, and there is a large class of problems that >>> are best addressed by analyzing what has changed between two dumps. >>> Cool. >>> #3 >>> >>> >>> >>>> 6: Java Debug Interface (JDI) Connector >>>> >>>> An adapter that allows a Java debugger to interrogate the contents of a >>>> Java >>>> Runtime diagnostic artifact. This connector will enable similar >>>> capabilities that exist today with other debuggers than can debug >>>> corefiles >>>> or similar process diagnostic artifacts. This tool will demonstrate >>>> key >>>> characteristics such as effective navigation to areas of interest, >>>> useful >>>> abstraction of key Java object implementation specifics and that the API >>>> provides the data that required to analyze real problems. >>>> >>>> >>>> >>>> >>> I'm not sure I understand what extra value this provides versus the >>> explorer. A debugger allows you to >>> - explore state >>> - modify state >>> - control execution >>> The first is what the explorer does, and the last two don't apply to dump >>> artifacts. I'm sure I'm missing something... >>> >>> >>> >> >> When I ask Java developers how they would want to see data from a crashed >> system presented this is usually top of the list >> We'd be producing a connector that allowed a readonly view of the state of >> the system. No ability to modify data and no concept >> of allowing continued execution. Stuart Monteith is interested in this >> area and I've asked him to post something more about the tool and its >> usecases. (In fact I've asked all the team to start threads for fleshing >> out these tools!) >> >> >> >>> >>> >>>> 7: Memory Analyser Tool (MAT) Adapter >>>> >>>> MAT (http://www.eclipse.org/mat/) is an open source project that >>>> consumes >>>> HPROF and DTFJ supported dumps. MAT is designed to help find memory >>>> leaks >>>> and reduce memory consumption. An adapter for MAT will be developed >>>> that >>>> allows MAT to consume HPROF and other dumps via the JSR 326 API. Key >>>> characteristics of this adapter will include handling large quantities >>>> of >>>> data efficiently, useful abstraction of key Java object implementation >>>> specifics and dump type identification. >>>> >>>> >>>> >>>> >>> Sounds like a relatively easy project that would effectively address >>> common >>> problems -- tasty low-hanging fruit. >>> #2 >>> >>> >>> >>>> 8 : Retrospector >>>> >>>> One of the rationales behind the JSR-326 was complexity of failure & >>>> performance analysis increasing with CPU core count growth. This tool >>>> will >>>> showcase this rationale. The tool will present a number of landmarks or >>>> checkpoints in the code and display how their mutual temporal ordering >>>> and >>>> latency change (a) over time (b) depending on the number of threads (c) >>>> depending on the number of available CPU cores etc >>>> >>>> Checkpoints can be of different kinds: >>>> - well-known events defined by JVMTI (monitor enter/exits, >>>> method entry/exits, class loads, code generation, thread lifecycle >>>> events,...) >>>> - new kinds of events helping solve particular problems. For >>>> example, interpreted -> compiled code execution mode change for a >>>> method, >>>> reaching a safepoint (in Hotspot terms), native memory allocation in >>>> from >>>> JNI code, arbitrary checkpoint specified by a user as >>> position> pair (with optional additional refinements or filtering), etc. >>>> >>>> This tool will likely require additions within the JRE such that when >>>> a >>>> checkpoint is reached by a thread, the JRE internally quickly logs this >>>> fact >>>> by adding a record into a >>>> buffer, which, at appropriate time, is processed and made available via >>>> the >>>> JSR-326 API. [Note: The logging should be extremely efficient to >>>> minimize >>>> the "observer effect", which seems quite possible with thread-local >>>> checkpoint event buffer.]. >>>> >>>> >>>> >>>> >>> DTrace? >>> >>> >>> >>>> The retrospector tool will be able to load the 2 or more checkpoint >>>> logs and >>>> - split up each of them into per-thread events >>>> - visualize the checkpoint events mapped onto the timescale >>>> - visualize the difference in temporal behavior between two threads >>>> within the same run of the analyzed app, between two threads within >>>> different runs on the analyzed app >>>> >>>> >>>> >>>> >>> This sounds like an interesting tool, but a lot of work. I think I'd >>> save >>> it for last. >>> >>> Nicholas >>> >>> >>> >>> >> >> >> > > --000e0cd47e54e7ef740461e23038--