incubator-kato-spec mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Poole <spoole...@googlemail.com>
Subject Re: JavaStackFrame/JavaLocation local variable support
Date Tue, 30 Jun 2009 12:47:39 GMT
On Tue, Jun 30, 2009 at 12:53 PM, Bobrovsky, Konstantin S <
konstantin.s.bobrovsky@intel.com> wrote:

> > I mostly agree with Steve.
>
> Sorry, that was Stuart :-)
>

Darn!

>
> Thanks,
> Konst
>
> Intel Novosibirsk
> Closed Joint Stock Company Intel A/O
> Registered legal address: Krylatsky Hills Business Park,
> 17 Krylatskaya Str., Bldg 4, Moscow 121614,
> Russian Federation
>
>
> -----Original Message-----
> From: Bobrovsky, Konstantin S
> Sent: Tuesday, June 30, 2009 6:50 PM
> To: 'kato-spec@incubator.apache.org'
> Subject: RE: JavaStackFrame/JavaLocation local variable support
>
> Hi all,
>
> --- citation start ---
> My experience of the JIT is somewhat limited, but certainly when
> debugging C programs with optimization,
> it is usual that variables are optimized out, loops unrolled, code
> reordered, such that  the variable contents and
> line numbers don't match the source. Having said that, I'm sure there
> are others who could make more authoritative
> comments on this area.
>
> Take:
>    for(int a=0, b=0; a<10; a++) {
>       b = a*2;
>       array[a][b] = array2[a];
>    }
>
> if the compiler did this:
>
>    for(int a=0; a<10;a++) {
>       array[a][a*2] = array2[a];
>    }
>
> Then the local variable "b" would no longer exist in any meaningful
> sense. My suggestion would be to remove "b" from the
> variable table. Of course, we could have two stack frames in the same
> method with different levels of optimization, but I believe that's
> probably still an issue anyhow.
> --- citation end ---
>
> (not pretending to be 'more authoritative', just 2 cents :-) ) I mostly
> agree with Steve. I would only add that the full code transformation chain
> looks like : source -> javac -> bytecode -> JIT -> code. And javac may
> sometimes do some subtle transformations which lead to certain local
> variables in the source code having no counterpart in the bytecode. JIT, in
> turn may of course optimize out many more things. But if we talk about
> Hotspot JIT, for many places in the code called 'safepoints' JIT retains the
> entire JVM state - in particular, values of live local variables - i.e. it
> is deducible from register/stack values at that place in the code. A call is
> always a safepoint. So at least for Hotspot VM, values of live local
> variables for all frames in a stack trace, except for maybe the starting
> one, should be deducible. But this is hardly true for an arbitrary JVM. (I'd
> be curious to know whether J9 has such feature).
>
> Thanks,
> Konst
>
> Intel Novosibirsk
> Closed Joint Stock Company Intel A/O
> Registered legal address: Krylatsky Hills Business Park,
> 17 Krylatskaya Str., Bldg 4, Moscow 121614,
> Russian Federation
>
>
> -----Original Message-----
> From: Stuart Monteith [mailto:stukato@stoo.me.uk]
> Sent: Friday, June 26, 2009 9:56 PM
> To: kato-spec@incubator.apache.org
> Subject: Re: JavaStackFrame/JavaLocation local variable support
>
>
>
> Steve Poole wrote:
> > On Fri, Jun 26, 2009 at 11:41 AM, Stuart Monteith <stukato@stoo.me.uk
> >wrote:
> >
> >
> >> Hi,
> >>   I was wondering what peoples thoughts were regarding program counters,
> >> line number table and variable tables.
> >> There is a tension between most users of the Kato API and the JDI
> connector
> >> and its obligations towards supplying the information JDWP requires.
> >> JDWP, for the most part, would like to know the location of a stack
> frame,
> >> i.e. a program counter normally, and using that to look up the variable
> >> tables and line number tables.
> >>
> >>
> >
> > I do see JDWP as a major use case  for us so we must make sure that our
> JDI
> > connector is first class.
> >
> >
> Agreed. I know that the katoView tomcat commands would benefit too - the
> FFDC scenario.
> >> Some changes have been made to the API to supply the local variables
> >> (through JavaStackFrame.getVariable(int)) and their locations/types
> through
> >> JavaMethod.getVariables().
> >> However, we haven't resolved the issue of the program counter or the
> line
> >> numbers.
> >>
> >> Just now the line numbers are available through
> >> JavaLocation.getLineNumber(), where available. However, JDWP never asks
> for
> >> a stack frame's line number, it maps from a stack frames
> >> location to a line number using the line number table from a stack
> frame's
> >> method.
> >>
> >>
> >
> >
> >> So, should we forgo having the simple JavaLocation.getLineNumber() and
> only
> >> supply the line number table (where appropriate)?
> >>
> >
> >
> > I was thinking "So what's the use of the getLineNumber method? "   but
> > outside the JDWP scenerio it does enable simple access to the linenumbers
> > (ie via the xpath approach)   The question is how much use that is and
> what
> > we'd be encombering the implementations with.       Since what the  the
> JDI
> > does is "standard" in its mapping then the RI could provide that code for
> > implementors to use.
> >
> >
> I think the concern I have is that not all implementations would be able
> to supply a program counter or a line number table.
> For instance, the hprof file stores only the line numbers. However, we
> shouldn't get too hung up on this as our implementation
> for hprof was always going to be limited.
> It is important that we supply all of the necessary information, and
> supply either helper methods on top or within API to make
> it more digestible for the majority of implementations.
>
> >
> >> Of course, having a "getProgramCounter()" method would be useful, but
> what
> >> should we do for compiled methods? There is a strong requirement for us
> to
> >> return the contents
> >> of local variables in compiled methods as well as interpreted methods.
> >> However, that requires synthesizing a bytecode program counter to
> retrieve
> >> the correct variables, which implies
> >> that line numbers could be generated too. However, as with C, etc, the
> >> debugging information derived from optimized code is usually inaccurate.
> >>
> >
> >
> >
> >> For line numbers, I imagine we'd either have the line numbers or not if
> >> they are inaccurate. But for local variables, it would be sensible to
> alter
> >> the variable table information to suit the
> >> optimized code, to give a consistent picture.
> >>
> >>
> >
> > I think we need to examine this in more detail -  got an example?
> >
> >
> >
> My experience of the JIT is somewhat limited, but certainly when
> debugging C programs with optimization,
> it is usual that variables are optimized out, loops unrolled, code
> reordered, such that  the variable contents and
> line numbers don't match the source. Having said that, I'm sure there
> are others who could make more authoritative
> comments on this area.
>
> Take:
>    for(int a=0, b=0; a<10; a++) {
>       b = a*2;
>       array[a][b] = array2[a];
>    }
>
> if the compiler did this:
>
>    for(int a=0; a<10;a++) {
>       array[a][a*2] = array2[a];
>    }
>
> Then the local variable "b" would no longer exist in any meaningful
> sense. My suggestion would be to remove "b" from the
> variable table. Of course, we could have two stack frames in the same
> method with different levels of optimization, but I believe that's
> probably still an issue anyhow.
>
>
> >
> >> Regards,
> >>   Stuart
> >>
> >>
> >> Stuart Monteith wrote:
> >>
> >>
> >>> Hi,
> >>>   I've been looking at local variables in relation to the JDI
> connector.
> >>> For the BOF at JavaOne we'd like for there to be a prototype of  local
> >>> variable support in the API. I've been looking at what JDWP requires as
> we
> >>> would have to be able to satisfy its queries using the Kato API. This
> has
> >>> made me lean towards exposing the variable table and have us retrieve
> the
> >>> local variables from the stack frames by slot number.
> >>>
> >>> So my suggestion for the API is this:
> >>>
> >>> ---------------------------------
> >>>
> >>> JavaMethod
> >>> -------------
> >>>
> >>> // returns all local variables
> >>> // empty if there are no variables.
> >>> Iterator<JavaVariable> getVariable() throws DataUnavailable;
> >>>
> >>> JavaVariable
> >>> -------------
> >>>
> >>> // Local variable's name
> >>> // throws DataUnavailable if the variable was derived from bytecode and
> so
> >>> the name is unknown. Caller is free to make a name up.
> >>> String getName() throws DataUnavailable;
> >>>
> >>> // The local variable's signature in JNI format.
> >>> String getSignature();
> >>>
> >>> // The start of the local variable's scope within the bytecode.
> >>> int getStart();
> >>>
> >>> // The number of bytes this variables scope covers over the bytecode.
> >>> int getLength();
> >>>
> >>> // The slot this variable occupies. Passed to
> JavaStackFrame.getVariable()
> >>> to retrieve the contents.
> >>> int getSlot();
> >>>
> >>>
> >>> JavaStackFrame
> >>> ------------------
> >>>
> >>> // Gets the value of a variable from a stack frame.
> >>> // Returns a JavaObject for an object reference, null for a null object
> >>> reference. Primitives are returned as boxed primitives.
> >>> // throws CorruptDataException if object reference is incorrect, or if
> the
> >>> float or double are set to invalid values.
> >>> // throws DataUnavailable if this method is not supported or if stack
> not
> >>> in correct state to return variables.
> >>> // throws IndexOutOfBoundsException if an invalid slot number if
> passed.
> >>> Object getVariable(int slot) throws CorruptDataException,
> DataUnavailable,
> >>> IndexOutOfBoundsException;
> >>>
> >>>
> >>> ---------------------------------
> >>>
> >>> The bytecode offset can be calculated with:
> >>>   JavaLocation.getAddress() - (
> >>> JavaMethod.getBytecodeSections().next().getBase().getAddress())
> >>> but I think that might be a little too tedious, and doesn't allow
> >>> cleverness with JITted frames. So we will probably have to add:
> >>>
> >>> // Return program counter in bytecode.
> >>> int JavaLocation.getBytecodePC();
> >>>
> >>> alternatively the JavaVariable.getStart() would use absolute addresses,
> >>> which could conceivably work with JITed frames, if the tables are
> maintained
> >>> during compilation.
> >>>
> >>> We should also expose the line number table too as that will aid class
> >>> file reproduction and queries for line numbers based on bytecode
> program
> >>> counters.
> >>>
> >>> A slightly different scheme would have the
> JavaStackFrame.getVariable(int
> >>> slot) method look like:
> >>>   Object getVariable(JavaVariable var);
> >>> but I don't think it gains us much.
> >>>
> >>> Retrieving all of the variables would therefore look something like
> this:
> >>>
> >>> void dumpVariables(JavaThread thread) throws Exception {
> >>>   Iterator frames = thread.getStackFrames();
> >>>   while (frames.hasNext()) {
> >>>      JavaStackFrame frame = (JavaStackFrame) frames.next();
> >>>      JavaLocation location = frame.getLocation();
> >>>      JavaMethod method = location.getMethod();
> >>>      int pc = location.getBytecodePC();
> >>>           System.out.println(location.toString()+":");
> >>>
> >>>      Iterator variables = method.getVariables();
> >>>      while (variables.hasNext()) {
> >>>         JavaVariable variable = (JavaVariable) variables.next();
> >>>
> >>>         if (pc >= variable.getStart() && pc <=
> >>> variable.getStart()+variables.getLength()) {
> >>>            Object value = frame.getVariable( variable.getSlot());
> >>>                       System.out.println("\t"+
> variable.getSignature()+"
> >>> "+variable.getName()+" = "+ value.toString());
> >>>         }
> >>>      }
> >>>   }
> >>> }
> >>>
> >>> Let me know what you think,
> >>>   Stuart
> >>>
> >>> Stuart Monteith wrote:
> >>>
> >>>
> >>>> Hello,
> >>>>   With Steve's work on JVMTI/python coming along, the issue of what
to
> do
> >>>> about local methods is coming up. Currently there is no means to
> determine
> >>>> the names and values of local variables through the current API.
> >>>>
> >>>> The most obvious way of implementing this is to have the API do all
of
> >>>> the processing by exposing the variables as name and value pairs.
> >>>>
> >>>> For example:
> >>>> interface JavaStackFrame {
> >>>>   List<LocalVariable> getLocalVariables();
> >>>> }
> >>>>
> >>>> where:
> >>>>
> >>>> interface LocalVariable {
> >>>>   String getName();
> >>>>   Object getValue();
> >>>> }
> >>>>
> >>>> Where the value is a JavaObject, or an boxed primitive.
> >>>>
> >>>> The other extreme is for the necessary information to be made
> available
> >>>> for the callers of the API to generate this information themselves.
> >>>> This would mean properly exposing:
> >>>>   Program Counter - currently we have JavaLocation.getAddress(), which
> is
> >>>> an address in memory, rather than a bytecode program counter. For
> JITted
> >>>> frames we'd still need the bytecode program counter.
> >>>>   Local variable table - this is to determine which variables there
> are,
> >>>> their types and their indexes into the local variable array
> >>>>   Local variable array - the contents of the local variables need to
> be
> >>>> exposed, and their proper types should be returnable (JavaObject, int,
> etc).
> >>>>
> >>>> Doing it that way might be beneficial for more user stories, there is
> >>>> more information available to reconstruct the class file, for
> instance.
> >>>> There is also the small matter of what to do when the local variable
> >>>> table is not available. When the API exposes all that it knows the
> values
> >>>> might still be retrievable, although I have my doubts as to how useful
> that
> >>>> would be if you don't know the types.
> >>>>
> >>>> Thoughts?
> >>>>   Stuart
> >>>>
> >>>>
> >>>>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message