incubator-kato-spec mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bobrovsky, Konstantin S" <konstantin.s.bobrov...@intel.com>
Subject RE: Do we need to change the scope of the API design?
Date Fri, 27 Nov 2009 10:35:09 GMT
Steve,

I see what you mean, thank you for the clarifications. I agree that having an RI where a big
chunk of the API is unimplemented can be quite frustrating for end users. My only concern
is the potential decrease in usability of Kato-based products when the Image API is removed.
	When there is an internal VM error such as wrong JIT-generated code
	leading to a crash, we can not know beforehand what data will be 
	required to diagnose the problem. Having all available data (core
	dump) is ideal.
In the beginning of the JSR many use cases have been developed, and it would be interesting
to estimate what share of them will be dropped with dropping the Image API. What leaves a
hope when Image API is gone :-) is remaining opportunity for JVM vendors to still produce
Kato implementations which can read core dumps and still plug into the Kato-conforming set
of tools developed by 3rd parties. The loss would be a thick non-standardized layer where
external tools developers can't adhere to.

> The runtime restrictions on the SA really effect it's usefulness for end 
> users of our API.

Are these impossibility to sell tools based on Kato implementation which use SA source code
or binaries? We have probably already discussed these restrictions in the past, but I forgot,
sorry.

> The cost of writing a corefile reader for hotspot is also
> quite large - it could easily take someone 6 months to write or more.

That's pretty pessimistic, IMO :) (at least for the part covering Image API). 3-5 months would
be enough, in my estimation. But that's still big chunk of work, I agree.

Thanks,
Konst


>-----Original Message-----
>From: Steve Poole [mailto:spoole167@googlemail.com]
>Sent: Friday, November 27, 2009 3:34 PM
>To: kato-spec@incubator.apache.org
>Subject: Re: Do we need to change the scope of the API design?
>
>On Thu, Nov 26, 2009 at 5:02 AM, Bobrovsky, Konstantin S <
>konstantin.s.bobrovsky@intel.com> wrote:
>
>> Hi Steve,
>>
>> I think the first option would be ideal of course. But I assume the
>> initiative to develop RI based on Sun's Serviceability Agent (which would
>> lower the implementation cost dramatically) has failed because of
>> GPL-related issues. I wish there was a volunteer from the open-source who
>> could adapt SA to the KATO API...
>>
>> That would be nice Konst. To be honest though, if I was wishing for help
>like this I would prefer to have someone work with us on creating the
>extended JVMTI or new C API interface as I wrote about previously.    The
>runtime restrictions on the SA really effect it's usefulness for end users
>of our API.     The cost of writing a corefile reader for hotspot is also
>quite large - it could easily take someone 6 months to write or more.
>Therefore I'd rather be pragmatic about this and focus on what can be done
>by us and what people say they want.
>
>>From the other two, the second one (remove Image API) IMO would very much
>> discord with the original message of the JSR (*postmortem* analysis,
>which
>> means digging through the core dumps). So I'd vote for leaving as much as
>> possible of the Image API in the spec, making parts optional as needed.
>>
>
>I understand about changing the message but I feel uncomfortable to say the
>least to have a large part of our API not supported by the Reference
>Implementation.    I will put together a proposal and explain what I think
>needs to be done.
>
>You can look at this from anoher point of view.   We know that core file
>sizes are rising ( 8-30GB production heap sizes are now common) and we
>agreed thereforet at some point core files would become untenable.   That's
>why we have the "snapshot" api  idea.  It seems that for most people the
>usefulness of corefiles is already in doubt so  lets focus on producing a
>new modern dump mechanism that is fast, scaleable,  content configurable
>and generally cross JVM.   Since we will be developing a reference
>implementation to go with the design we will effectively be producing a
>modern replacement for HPROF and since the format for this dump is not part
>of the specification (as we have agreed that data formats are always
>constraining) we will not get rocks thrown at us by JVM vendors who want to
>implement their own alternatives.
>
>I think developing the HPROF alternative is the right place to go.  Users
>will understand the concept and we might actually gain input from existing
>HPROF users who want to see improvements made.
>
>We are building the current implementation just using JVMTI.  Thats not
>good
>enough and we need to find ways of getting more data out of a JVM.  Hence
>the idea of improving JVMTI or adding some new API.     That is probably
>not
>the right way either but lets discuss..
>
>
>> Thanks,
>> Konst
>>
>> Intel Novosibirsk
>> Closed Joint Stock Company Intel A/O
>> Registered legal address: Krylatsky Hills Business Park,
>> 17 Krylatskaya Str., Bldg 4, Moscow 121614,
>> Russian Federation
>>
>>
>> >-----Original Message-----
>> >From: Steve Poole [mailto:spoole167@googlemail.com]
>> >Sent: Wednesday, November 25, 2009 7:44 PM
>> >To: kato-spec@incubator.apache.org
>> >Subject: Do we need to change the scope of the API design?
>> >
>> >Stuart's note on diagnosing JNI problems (thanks for the review Stuart)
>> has
>> >touched on a key point  concerning what we want to do and what we can
>do.
>> >
>> >I think its time we discuss where we are and what happens next.
>> >
>> >Our desire to specify an API that helps people solve problems has to be
>> >tempered with  practical implementation considerations.   The API design
>> >that IBM contributed to the Kato project is satisfiable by IBM JVMs (of
>> >course),  but for us to develop a matching reference implementation we
>> know
>> >now that we need to get much more information from the Sun JVM than we
>> have
>> >today.  Developing a full solution to match the API as it stands today
>is
>> >difficult for the Kato project as the code we would need to access is in
>> >OpenJDK and is under the GPL.
>> >
>> >Practically, we have few choices.  We could
>> >
>> >1 - Get a JVM Vendor to produce , or commit to produce  an
>implementation
>> >that fully covers the Image API side.
>> >2 - Remove all the Image API side from the design
>> >3 - Find a halfway point where some of the Image API stays and is
>> optional.
>> >
>> >I've had many conversations with people concerning option 1 and the
>> >response
>> >is generally that its too expensive for the potential return.  Both for
>> >implementation and ongoing maintenance.
>> >
>> >Option 2 seems very draconian -  the ability to get at some of the
>native
>> >information (such as the native thread for a Java thread) just seems too
>> >useful to remove.
>> >
>> >I'd opt for us moving the API design towards option 3 - there are some
>> >basic
>> >changes we could do such as extracting the native access methods into a
>> >separate interface and allowing the implementer the choice to use it.
>> >
>> >We still need to get access to more data from the JVM (and perhaps in
>> >better
>> >ways) than we have today.  We need to get this data in such a manner as
>> not
>> >to require accessing GPL code and in a way that is cheap to maintain as
>> the
>> >JVM evolves.
>> >
>> >My idea for resolving this conundrum is to propose that we include in
>our
>> >API design either a new JVM 'C' API or  extensions to JVMTI. We would
>not
>> >implement these additions in the RI but would rely on others to provide
>an
>> >implementation.  (We would design the RI and the API to make this extra
>> >information optional)
>> >
>> >Comments?
>> >
>> >
>> >--
>> >Steve
>>
>
>
>
>--
>Steve

Mime
View raw message