incubator-kato-spec mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stuart Monteith <stuk...@stoo.me.uk>
Subject Re: New draft versin of spec
Date Tue, 28 Jul 2009 16:37:29 GMT
Hi,
    Seamonkey doesn't help much, so replies as before :-)


Bobrovsky, Konstantin S wrote:
> Hi Stuart,
>
> Some more of my comments are also inline below (marked with '[Bobrovsky, Konstantin S]'
- that's how Outlook helps :-) )
>
> Thanks,
> Konst
>  
> Intel Novosibirsk
> Closed Joint Stock Company Intel A/O
> Registered legal address: Krylatsky Hills Business Park, 
> 17 Krylatskaya Str., Bldg 4, Moscow 121614, 
> Russian Federation
>  
>
>   
>> -----Original Message-----
>> From: Stuart Monteith [mailto:stukato@stoo.me.uk]
>> Sent: Monday, July 27, 2009 11:26 PM
>> To: kato-spec@incubator.apache.org
>> Subject: Re: New draft versin of spec
>>
>> Hi Konstantin,
>>    Thanks for you comments, I've added mine inline below.
>>
>> Regards,
>>    Stuart
>>
>> Bobrovsky, Konstantin S wrote:
>>     
>>> Hi Steve, all,
>>>
>>> FYI: I added some comments to the Kato javadoc @
>>>       
>> http://cwiki.apache.org/confluence/display/KATO/JavaDoc+comments. I'm
>> resending them to this list for convenience.
>>     
>>> Thanks,
>>> Konst
>>>
>>> Intel Novosibirsk
>>> Closed Joint Stock Company Intel A/O
>>> Registered legal address: Krylatsky Hills Business Park,
>>> 17 Krylatskaya Str., Bldg 4, Moscow 121614,
>>> Russian Federation
>>>
>>> Top-level
>>> =========
>>>
>>> (1) kato.java seems a specialization of kato.runtime, so
>>>       
>> kato.runtime.java could be used instead.
>>     
>>>       
>> +1
>>
>>     
>>> (2) I noticed that there are API elements in the form of List
>>>       
>> get[SomeEntityName]() and javadoc does not specify objects of which type
>> will be returned in the resulting list. E.g. ImageThread.getStackFrames().
>>     
>>>       
>> I'll fix these up when I come across them. I've been reviewing the Java
>> runtime.
>>
>>     
>>> (3) It would be nice to have an opportunity to create an Image* object
>>>       
> >from a pointer into an image (having ImagePointer instance which describes
>   
>> the start of the Image* entity)
>>     
>>>       
>> Can you give an example? I can see how this might be useful for the java
>> API, but I don't see how it'd be useful for image.
>>
>>     
>
> [Bobrovsky, Konstantin S] 
> e.g.
> ImageStackFrame ImageFactory.getImageStackFrameAt(ImagePointer addr)
> ImageThread ImageFactory.getImageThreadAt(ImagePointer addr)
>
> And yes, this would be useful for the kato.java API too - I agree.
>
>   
[Monteith, S]

I think I can understand how this applies to 
getImageStackFrameAt(ImagePointer) - a pointer that lies within
the stack could be returned. It might be better though to implement 
ImageStackFrame.getSections() and search
through them. This would also apply to IA64 split stacks.

>>> org.apache.kato.image
>>> =====================
>>>
>>> ImageModule
>>> -----------
>>>
>>> There should also be a way to get the address the module was loaded at
>>>       
>> and, if possible, its size.
>>     
>>>       
>> I think this is a JavaDoc issue. ImageModule.getSections() returns
>> ImageSections that contain the information you are asking for. It is not
>> clear from
>> the doc that this is the case.
>>     
>>> ImageRegister
>>> -------------
>>>
>>> register names for different CPUs should probably follow the naming
>>>       
>> convention adopted in the official specification of this CPU. This way
>> tools would work uniformly with different Kato implementations.
>>     
>>>       
>> I think having us specify the register set for every possible processor
>> architecture is inappropriate, so I agree.
>>     
>>> 'Long' or 'Integer' might be too vague for specifying register type. How
>>>       
>> about '64 bit' or '32 bit' plus an additional characteristic such as
>> 'general purpose/floating point/application/flags/...' ?
>>     
>> The "Type" information is there to describe which subclass of Number
>> will be returned by ImageRegister.getValue(). But you're right, it
>> doesn't actually describe
>> what the register contains.
>>     
>>> The register sets specified here should be complete, otherwise we could
>>>       
>> get implementation/tool incompatibilites for the registers missing from
>> this javadoc. One of the alternative ways for specifying them could be just
>> a link to the corresponding external specification.
>>     
>>>       
>> I have to agree with linking to the external specification. There has to
>> be at least some agreement to the set of registers available.
>>     
>>> org.apache.kato.java
>>> ====================
>>>
>>> JavaReference
>>> -------------
>>>
>>> (1)
>>> sometimes it is very helpful to know where the heap root actually resides
>>>       
>> - the corresponding CPU register and/or memory address (in a memory stack
>> frame or register stack frame), and this info is usually available in Java
>> runtimes.
>>     
>>>       
>> So, this would be something like: ImagePointer
>> JavaReference.getAddress() and ImageRegister
>> JavaReference.getRegister(), with a boolean for both?
>>
>>     
>
> [Bobrovsky, Konstantin S] 
> getAddress() and getRegister() look satisfactory. What the boolean is for?
> Can getRegister() return just null if the reference resides in memory only?
>
>   
[Monteith, S]
I think I was just being verbose. The boolean is just to indicate 
whether or not the reference is at an address and/or a register. With 
null's then
it would be redundant, I agree.

>>> (2)
>>> It seems the API here lacks a REFERENCE_DIRIVED reference type to
>>>       
>> describe derived references (pointers), which point to an interrior of a
>> Java object - to an array element, for example. Derived references are
>> reported to Garbage Collector in pairs with references to their base Java
>> objects.
>>     
>>>
>>>       
>> The JavaReference types are the same as what you would get from JVMTI,
>> so that's why we have that limited list.
>>     
>
> [Bobrovsky, Konstantin S] 
> So it seems JVMTI does not fully cover the variety of references.
>
>   
[Monteith, S]
Well, notionally a derived reference is a reference to the object the 
element/field is in as it is that it is keeping alive.
You could have a description of "Derived reference to element 42" from 
the JavaReference.
You will want to know explicitly that it is a reference of a specific 
type on your specific JVM, and so the specificity is important.
My question is, what other reference types are there?

>> If I were to implement this, I would have had  the reference just be
>> like any other reference to a field or element.
>> But if we were to expand it, I'd suggest something like the following:
>>
>>    REFERENCE_DERIVED_ARRAY_ELEMENT
>>    REFERENCE_DERIVED_OBJECT_FIELD
>>    REFERENCE_DERIVED_CLASS_FIELD
>>
>>     
>
> [Bobrovsky, Konstantin S] This looks good to me.
>
>   
>>> JavaRuntime
>>> -----------
>>>
>>> (1) We should have somewhere in the API means to iterate trough generated
>>>       
>> code sequences other than managed methods - so-called 'stubs'. Each stub is
>> similar to a method: it has a name, (possibly, multiple) code sections.
>> This interface (JavaRuntime) seems an appropriate place for this API.
>>     
>
> [Bobrovsky, Konstantin S] 
> (To develop this thought a bit...)
> Maybe we could have
>   interface CodeStub {
>     String getName();
>     ImageSection getCodeSections();
>   }
>
> And
> List JavaRuntime.getCodeStubs();
>
>   
[Monteith, S]
I think I'd like to see how these are used. Presumably then they would 
be returned in the stacks.
Presumably for our integrated stack frames we'd have a class hierarchy like:

StackElement--+----ImageStackFrame
              |
              +----JavaStackFrame
              |
              +----CodeStub

and:
     List<StackElement> JavaThread.getStackElements()

That would put the code stubs in some sort of context.
>>> (2) Not all methods within a Java runtime are necessarily compiled. Many
>>>       
>> may remain interpreted, others might not even have a bytecode. So I would
>> suggest to replace 'getCompiledMethods' with smth like 'getJavaMethods' or
>> add 'getInterpretedMethods', 'getMethodsWithoutCode'.
>>     
>>>       
>> The intention isn't for all of the methods to be compiled, the method
>> will just list those methods that are compiled. I would argue for us
>> actually removing the JavaRuntime.getCompiledMethods() if we don't have
>> a reason for including it.
>>     
>
> [Bobrovsky, Konstantin S] 
> I thought of this method as an iterator through all java methods known to the system
- its javadoc implies this: 'This is short cut method. The same result can be found by iterating
over all methods in all class loaders in all classes'. This iteration will also yield methods
which are not compiled.
>
> So we can have all
>   getCompiledMethods()
>   getInterpretedMethods()
>   getMethodsWithoutCode()
> and
>   getJavaMethods()
>
> or just
>   List getJavaMethods(MethodFilter f)
>
> where f defines the acceptance criteria.
>
>   
[Monteith, S]
Yes, the implication in the JavaDoc was that getCompiledMethods() would 
return every JavaMethod that returns something for 
JavaMethod.getCompiledSections() other than an empty list.
I'd settle on either removing the convenience method or looking at 
patterns we could apply to the whole API.
>> Apparently we may also have native methods that have their stubs
>> compiled too.
>>     
>>> JavaStackFrame
>>> --------------
>>>
>>> (1)
>>> Usually, Java stack frames contain information about Java monitors held
>>>       
>> in this frame. This is necessary because Java spec requires "forced"
>> release of all monitors held by a method in case it "completes abruptly".
>> So something like
>>     
>>> List getHeldMonitors()
>>> would be really useful.
>>>
>>>
>>>       
>> I agree, having the API explictly show where monitors are held would
>> take people closer towards diagnosing deadlocks, as opposed to having
>> them read through their code and work out where they were taken. It may
>> not always be obvious from reading code which monitor has actually been
>> taken.
>>
>>     
>>> (2)
>>> API also needs to support 'split' stack frames on Itanium - every frame
>>>       
>> has a traditinal memory stack frame and additionally, a register stack
>> frame where Register Stack Engine (RSE) spills current register frame
>> during a call (and restores from upon return).
>>     
>>> Maybe this could be done via delegating responsibility for such support
>>>       
>> to an Itanium-specific implementaion of the spec?
>>     
>>>       
>> The API isn't very good at exposing the native information. With the
>> currently JavaThread.getStackSections() could return two ImageSections
>> if you are on IA64.
>> Certainly that's what we do for ImageThread.getStackSections() on IBM's
>> 1.4.2 IA64 Linux implementation.
>>     
>
> [Bobrovsky, Konstantin S] 
> Yes, this is a good catch. Stack is split on ia64 at the thread level too.
> Though, I meant slightly different thing: on IA64, we'd like to have information about
where register stack for a particular stack frame (rather than a thread) starts in memory.
>
>   
[Monteith, S]
I can see how there'd be difficulties, there is certainly a case for 
having ImageSections for Java and Image StackFrames, to show where they 
are and their extent.
One thing that troubles me is how we'd handle the NaT bits - the 
cleanest option would be to leave an 8 byte whole where they've been 
spilled. Ideally
the NaT bits could be reconstructed for the spilled registers, but that 
is a distinctly odd feature of IA64.

>> One of the the unresolved problems I have with the API is how to relate
>> all of the different frame types. Just now Java frame could be on it's
>> own "Java stack",
>> on the native stack as interpreted or compiled code, could be a native
>> method with a stub calling a C/C++ function, and we don't help with this.
>> For example, if you have a stack like:
>>
>> C function frame
>> stub frame
>> native method frame
>> compiled method frame (on native stack)
>> interpreted method frame (on own stack)
>>
>> how do you derive that information from the Kato API? I don't think we can.
>>     
>
> [Bobrovsky, Konstantin S] 
> As I understand the question, the problem is in apparent complexity of writing a Kato
implementation which would be able to iterate through various kinds of frames on the a thread's
stack, and for each frame report its type, extent and other information. ?
>
> I think any Java runtime should be able to do this (Hotspot is), at least when collecting
garbage. And if a Java runtime knows how to do this in a live VM, Java code (Kato implementation
for this runtime) which does this for a dead VM should look similar.
>   
[Monteith, S]
See my comment above. It's not so much the implementation complexity I 
was worried about, but rather how we represent it in the API. We 
currently don't
provide a way, that I can see, of sensibly communicating the layout of 
the stack.
>   
>>> (3)
>>> while getLocation() can provide information about what method this frame
>>>       
>> belongs to, this method may have several compiled or interpreted
>> "incarnations". So, I would suggest to add the following:
>>     
>>> boolean isCompiledFrame()
>>> boolean isInterpretedFrame()
>>>
>>> /** @return an ImageSection containing the compiled code to which this
>>>       
>> stack frame corresponds */
>>     
>>> ImageSection getCode()
>>>
>>>
>>>       
>> I agree with this, there can be several incarnations of a method
>> inflight that we aren't presenting. Contrary to your suggestion, should
>> we make it easy to
>> get all java method incarnations? I might suggest that we have:
>>
>> List<JavaMethodIncarnation> JavaMethod.getIncarnations()
>>
>> JavaMethodIncarnation JavaLocation.getMethodIncarnation()
>>
>> In JavaMethodIncarnation:
>>    JavaMethod getMethod()
>>    boolean isJITCompiled()
>>    boolean isNative()
>>    int compilationLevel()
>>    List<ImageSection> getSections();
>>
>>     
>
> [Bobrovsky, Konstantin S]
> I think this is an interesting idea overall. But not solve the <stack frame> ->
<method incarnation> mapping problem there should additionally be JavaMethodIncarnation
JavaStackFrame.getMethodIncarnation()
>
>   
[Monteith, S]
Yes, it's important to match them up.
> Here is another angle to look from.
> compilationLevel() and getSections() don't apply to an incarnation of a native method
(which will always have one incarnation), or to an interpreted incarnation of a usual method.
What will only be used for such incarnations is getMethod(), isJITCompiled() and isNative().
So the API looks a little 'irregular' to me. It seems the 'incarnation' semantics is more
close to 'compiled code incarnation' of a particular method. So I would develop the suggested
API into a little more specific one:
>
> JavaMethod JavaStackFrame.getMethod()
> boolean JavaStackFrame.isCompiled()
> boolean JavaStackFrame.isInterpreted()  
> boolean JavaStackFrame.isNative()  
>
> // returns a non-null value only if isCompiled() is true
> CompiledMethodIncarnation JavaStackFrame.getCompiledMethodIncarnation()
>
> interface CompiledMethodIncarnation {
>     JavaMethod getMethod();
>     int compilationLevel()
>     List<ImageSection> getSections();
>     int getBytecodeIndexForEntryPoint(); // repeats your suggestion below
>     ...
> }
>
> And all the details for native and interpreted stack frames (which can have only one
'incarnation') will be queried from the returned JavaMethod - such as bytecode, function address,
etc.
>
>   
[Monteith, S]
That looks good. It leaves a simple route for more basic uses (i.e. 
getting the name of the method and it's class), and it is sufficiently 
general to cover the various JVM implementations.
>>> (4)
>>> note that sometimes methods are compiled with the entry point being not
>>>       
>> the first method's bytecode (compilation for so-called on-stack-replacement
>> or OSR). So each compiled version of a method should also be annotated with
>> bytecode index where the entry point for this compilation is. In this
>> sense, ImageSection is not enough to describe a compiled code.
>>     
>>>       
>> Sure,
>>    ImagePointer JavaMethodIncarnation.getEntryPoint()
>> ?
>>
>>     
>
> [Bobrovsky, Konstantin S] 
> Yes, this approach looks good to me, with a slight change I described above.
>
>   
>> JavaMethodIncarnation is a placeholder for perhaps a more sensible name.
>>     
>>> -----Original Message-----
>>> From: Steve Poole [mailto:spoole167@googlemail.com]
>>> Sent: Sunday, July 26, 2009 12:17 AM
>>> To: kato-spec@incubator.apache.org
>>> Subject: New draft versin of spec
>>>
>>> I've just posted to the wiki a docbook based version of the
>>>       
>> specification.
>>     
>>> Still needs work but a lot better than it was.
>>>
>>>
>>> http://cwiki.apache.org/confluence/download/attachments/2326547/spec-edr-
>>>       
>> 25july2009.pdf
>>     
>>> Cheers
>>>
>>> Steve
>>>
>>>       
>> --
>> Stuart Monteith
>> http://blog.stoo.me.uk/
>>     
>
>   

Thanks,
    Stuart

-- 
Stuart Monteith
http://blog.stoo.me.uk/


Mime
View raw message