From kato-spec-return-92-apmail-incubator-kato-spec-archive=incubator.apache.org@incubator.apache.org Wed Apr 08 01:20:07 2009 Return-Path: Delivered-To: apmail-incubator-kato-spec-archive@minotaur.apache.org Received: (qmail 95749 invoked from network); 8 Apr 2009 01:20:07 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 8 Apr 2009 01:20:07 -0000 Received: (qmail 81868 invoked by uid 500); 8 Apr 2009 01:20:07 -0000 Delivered-To: apmail-incubator-kato-spec-archive@incubator.apache.org Received: (qmail 81838 invoked by uid 500); 8 Apr 2009 01:20:07 -0000 Mailing-List: contact kato-spec-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: kato-spec@incubator.apache.org Delivered-To: mailing list kato-spec@incubator.apache.org Received: (qmail 81828 invoked by uid 99); 8 Apr 2009 01:20:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Apr 2009 01:20:07 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of dpj@us.ibm.com designates 32.97.182.139 as permitted sender) Received: from [32.97.182.139] (HELO e9.ny.us.ibm.com) (32.97.182.139) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Apr 2009 01:19:56 +0000 Received: from d01relay04.pok.ibm.com (d01relay04.pok.ibm.com [9.56.227.236]) by e9.ny.us.ibm.com (8.13.1/8.13.1) with ESMTP id n3819jKd012686 for ; Tue, 7 Apr 2009 21:09:46 -0400 Received: from d01av01.pok.ibm.com (d01av01.pok.ibm.com [9.56.224.215]) by d01relay04.pok.ibm.com (8.13.8/8.13.8/NCO v9.2) with ESMTP id n381JZnS142852 for ; Tue, 7 Apr 2009 21:19:35 -0400 Received: from d01av01.pok.ibm.com (loopback [127.0.0.1]) by d01av01.pok.ibm.com (8.12.11.20060308/8.13.3) with ESMTP id n381JZsF014540 for ; Tue, 7 Apr 2009 21:19:35 -0400 Received: from d01mc084.pok.ibm.com (d01mc084.pok.ibm.com [9.63.9.226]) by d01av01.pok.ibm.com (8.12.11.20060308/8.12.11) with ESMTP id n381JYfl014537 for ; Tue, 7 Apr 2009 21:19:34 -0400 In-Reply-To: References: <49D9987F.9060001@Sun.COM> <49DA5707.3040301@Sun.COM> <85b044530904061331g9e382ach938b5287bab47ccb@mail.gmail.com> <49DA69EA.7000908@Sun.COM> Subject: Re: Kato API javadoc X-KeepSent: 0271BFA0:1101024E-85257592:00054761; type=4; name=$KeepSent To: kato-spec@incubator.apache.org X-Mailer: Lotus Notes Release 8.0.1 HF105 April 10, 2008 Message-ID: From: Daniel Julin Date: Tue, 7 Apr 2009 21:19:23 -0400 X-MIMETrack: Serialize by Router on D01MC084/01/M/IBM(Release 8.0.1|February 07, 2008) at 04/07/2009 21:19:34 MIME-Version: 1.0 Content-type: multipart/alternative; Boundary="0__=0ABBFF01DF96C1F18f9e8a93df938690918c0ABBFF01DF96C1F1" Content-Disposition: inline X-Virus-Checked: Checked by ClamAV on apache.org --0__=0ABBFF01DF96C1F18f9e8a93df938690918c0ABBFF01DF96C1F1 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: quoted-printable Some thoughts about these iterators, also based on earlier experiences = with the DumpAnalyzer tool: (1) The set of entries returned by some of these iterators can be quite= large. Especially for JavaHeap.getObjects(), and also, to a lesser degr= ee, JavaClassLoader.getDefinedClasses(), ImageModule.getSymbols(), etc. If = we go to a model that allows random access, can we do it in a way that avo= ids forcing a pre-loading of every instance of every entry in the set, onto= the heap of the JVM in which we are running the analyzer tool. (2) The API does not actually define any specific order in which the entries are returned by the various iterators, and I wish it would. For= example: * When walking through all the objects on a heap, some heap analysis functions (and some file formats like PHD) require that the objects be returned in the order corresponding to the increasing addresses where t= hey are located on that heap. It turns out that, in all the implementation= s that I've seen so far, JavaHeap.getObjects() does provide that order, b= ut nowhere does it say that I can count on it in every future implementati= on. * When writing unit tests for analysis tools, it is very convenient to = be able to count on a stable order returned by every iterator. Many tests = can be written by printing all the output from some function, and simply comparing that output from one run of the tool to the next to detect regressions. This stable order was actually not provided by many iterat= ors in current DTFJ implementations, and has caused considerable headache i= n the early days of the implementation of the DumpAnalyzer tool. We ended= -up adding our own sort routines on top of many DTFJ iterators... which of course conflicts with point (1) above, to avoid pre-loading every entry= . (3) At various times in the past, we had some discussions about potenti= ally using a cursor-style API rather than an iterator, for large sets like t= he one from JavaHeap.getObjects(), to avoid excessive object allocation in= the analyzer tool JVM. How do you feel about that? -- Daniel -- Steve Poole wrote on 2009-04-06 05:19:07 PM:= > > On Mon, Apr 6, 2009 at 9:45 PM, Nicholas Sterling > wrote: > > > > > > > Carmine Cristallo wrote: > > > >> Hi all!A number of potential work items are starting to emerge fro= m this > >> thread. I'll try to enumerate them: > >> > >> 1. refactor the API to use generics. Basically, every method that = now > >> returns an Iterator should have its signature modified. And while we're at > >> it... is Iterator<...> the right "thing" to return? Wouldn't a collection > >> (List<...>?) suit better? > >> > >> > > Ah, good point, although there might be some cases where the number= of > > values would be huge (like objects on the heap), and for those you probably > > do just want to provide an Iterator. > > > > The main problem with iterators is providing any sort of random acces= s. You > end up with repeated walks through the iterator. Having lists seeme= d like > a good alternative. I took a copy of DTFJ last year and replaced the= > iterators with lists - then I easily added a jxpath ( > http://commons.apache.org/jxpath/) layer on top. It was pretty cool= - if > we agree on providing a random access mechanism of any sort it should= be > fairly simple to redo the jxpath experiment. > > > > > That reminds me -- Steve, weren't you talking about an event-driven= > > (SAX-like) parser at one point, in which one specifies the classes = of > > interest in the target heap and the code to be executed for each? I= suppose > > that could, and probably should be, build on *top* of an Iterator. > > > > Yes - I was wondering out loud if the "DOM" approach we have with the= API is > sufficent. Its obvious when you start implementing the underlying support > for this sort of API just how important it is to be lazy. A "SAX" lik= e > approach could possibly provide oppotunities to be even lazier!. > > > > > Nicholas > > > > > > 2. give a better implementation - as the one suggested by Nicholas= - to > >> ImageThread#getRegister(). How about renaming it to "getRegisterMa= p ()", > >> returning a RegisterMap interface? > >> 3. refactor the TCK to decouple the setup classes from the test classes, > >> as > >> suggested by Stuart. > >> > >> Steve... should we start to open Jira work items for the above activities? > >> > >> > >> Carmine > >> > >> On Mon, Apr 6, 2009 at 8:24 PM, Nicholas Sterling < > >> Nicholas.Sterling@sun.com > >> > >> > >>> wrote: > >>> > >>> > >> > >> > >> > >>> It's very nice being able to look at this javadoc -- thanks! > >>> > >>> It might help to have a little introductory text in some of the k= ey > >>> classes > >>> giving some context, something along the lines of the DTFJ exampl= e on > >>> your > >>> web site that opens a core dump and iterates through the threads.= > >>> > >>> I wonder if ImageThread should return interface RegisterSet, of w= hich > >>> there > >>> would be various implementations for various CPU types, each containing a > >>> map from a Register enum to a RegisterValue. > >>> > >>> I hadn't realized until I started looking through this javadoc ho= w much > >>> easier the use of generics makes it to understand an API. For example, > >>> under JavaClassLoader I see methods getCachedClasses() and > >>> getDefinedClasses(), but I can't tell from their signatures wheth= er they > >>> return the same type or not. That info is in the method descriptions, > >>> but > >>> it's a lot more work to flip back and forth between the Summary a= nd the > >>> Detail. > >>> > >>> Nicholas > >>> > >>> > >>> > >>> Steve Poole wrote: > >>> > >>> > >>> > >>>> On Mon, Apr 6, 2009 at 6:51 AM, Nicholas Sterling < > >>>> Nicholas.Sterling@sun.com > >>>> > >>>> > >>>> > >>>> > >>>>> wrote: > >>>>> > >>>>> > >>>>> > >>>>> > >>>> > >>>> > >>>> > >>>>> Great! I've passed this on to the HotSpot folks for comment. > >>>>> > >>>>> I think I remember us talking about there being some provision = for > >>>>> accessing the vendor-specific VM constructs that implement the heap, > >>>>> etc., > >>>>> in addition to the Java objects in it. Will that be done by, f= or > >>>>> example, > >>>>> casting a JavaVM to a HotSpotVM and using the latter's extra methods? > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>> That's the most obvious solution I think. > >>>> > >>>> > >>>> > >>>> > >>>> > >>>>> Also, I'm seeing methods that return Iterator with no type (e.g= . in > >>>>> JavaMethod). Is that just a temporary placeholder which will > >>>>> ultimately > >>>>> get > >>>>> a type? > >>>>> > >>>>> > >>>>> > >>>>> > >>>> That's an interesting question. The reason for there being no = type > >>>> info > >>>> is > >>>> that the API was designed to compile and run on 1.4.2. > >>>> We need to decide if that still makes sense. I know that 1.4 i= s out of > >>>> support by Sun and IBM. What about Oracle? > >>>> > >>>> > >>>> > >>>> > >>>> > >>>>> Nicholas > >>>>> > >>>>> > >>>>> > >>>>> Steve Poole wrote: > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>>> Well at last ! - we actually have the API javdoc available - it's > >>>>>> here > >>>>>> > >>>>>> > >>>>>> http://hudson.zones.apache.org/hudson/view/Kato/job/kato.api- > head/javadoc/ > >>>>>> > >>>>>> I'm certainly not going to hold this up as a the greatest java= doc in > >>>>>> the > >>>>>> world but its a good place to start. I do feel that we have finally > >>>>>> arrived :-) > >>>>>> > >>>>>> The API has lots of "DTFJ"ness to it that needs to go but I'm really > >>>>>> interested in intitial reactions to the javadoc - is the form= of the > >>>>>> API > >>>>>> what you expected? > >>>>>> > >>>>>> > >>>>>> Moving on - there is still code needed to make the API work (w= e need > >>>>>> to > >>>>>> get > >>>>>> the hprof support working) but we can make progress in the interim. > >>>>>> I > >>>>>> want to move quickly towards having a regular heat beat where = we are > >>>>>> moving > >>>>>> through the usecases that we have. To do that we need to get= up to > >>>>>> speed > >>>>>> with the API shape as it stands today. Stuart has published= some > >>>>>> info > >>>>>> on > >>>>>> the API but its not really sufficent for educational needs :-= ) > >>>>>> > >>>>>> Is it worth holding a conference call so that we can walk thro= ugh the > >>>>>> API > >>>>>> to > >>>>>> explain why its the shape it is or is everyone comfortable wit= h just > >>>>>> more > >>>>>> doc? > >>>>>> > >>>>>> Cheers > >>>>>> > >>>>>> Steve > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>> > >>>> > >>> > >> > >> > >= --0__=0ABBFF01DF96C1F18f9e8a93df938690918c0ABBFF01DF96C1F1--