harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ilya Berezhniuk" <ilya.berezhn...@gmail.com>
Subject Re: Thoughts... (was: Re: [vote] HARMONY-4689 - JVMTI Extension: Native Code Access Interface implementation)
Date Sun, 14 Oct 2007 22:46:01 GMT
Hello Vasily,

Both these approaches (NCAI and external debugger) have their own
advantages. I'll describe a couple of major NCAI advantages:

1) Using two debuggers at once looks risky for me; one debugger
extending JVMTI functionality is safer. There are several areas of
possible conflicts between the debuggers. It's breakpoints handling -
some caught breakpoints should be processed by the external debugger,
and some of them should be delivered to JVMTI. I guess there can be
possible conflicts while working with threads - only VM internal code
can take care of Java threads specifics. There can be other
conflicting areas.

2) AFAIK, gdb cannot enumerate the whole stack of Java thread
correctly, because it cannot continue unwinding native stack below
Java frames. Also gdb is unable to unwind native stack when the code
is called from JVMTI breakpoint/single step handling routine.
Moreover, gdb often provides native stack that is worse than the stack
provided by the heuristic stack walker I've implemented for NCAI.
(This stack walker is already integrated for using by the crash
handler, so NCAI Frame functions are simply wrappers.)
To tell the truth, the above statement is correct for IA-32; AMD64
calling conventions prevent heuristic stack unwinding. AFAIU, GDB can
use dwarf stack info for unwinding - so GDB could provide better stack
on x86_64.


2007/10/14, Alexei Fedotov <alexei.fedotov@gmail.com>:
> Hello Vasily,
>
> I like your constructive overview. I agree with you that traversing
> managed frames is quite VM specific and proves to be a problem for
> native debuggers.
>
> To my personal point of view, the complexity of using external thread
> libraries is underestimated in your analysis since VM uses several
> self-made synchronization primitives which do most of the job. For
> example, not far from now gdb on IPF crashed in a simple test.
>
> Windows threading model may prove some unexpected challenges. For
> example, Ivan Volosyuk spent a day trying to understand that our VM
> thread local data should be prefixed with unused field to made
> debugging possible. I faced a number of gdb crashes on Windows.
>
> As for your questions of alternatives, a year ago I investigated cross
> platform alternatives for native frame enumeration. I considered gdb
> and a debugger from Intel compiler. I talked to people who developed
> the debuggers, and then decided to use our internal solution Ilya was
> finalizing.
>
> Thank you for your attention.
>
> BTW, I believe we should not mix two things in our discussion. I like
> if someone will come up with better solution: many places of DRLVM can
> be improved. From the other side casting a vote
>
> BTW2, the file pthread_dbg.h you've provided a reference to doesn't
> have a pure BSD license, see the third item.
>
> On 10/13/07, Vasily Levchenko <vasily.v.levchenko@gmail.com> wrote:
> > Originally,  Java developers using JNI have to debug their application using
> > two debugger for accessing java and native world(
> > http://www-128.ibm.com/developerworks/java/library/j-jnidebug/index.html,
> > http://developers.sun.com/learning/javaoneonline/2006/tools/TS-1011.pdf).
> > This concept should be the foundation for "mixed" debugger. Native debugger
> > (gdb, adb or other) to debug java world need information how to deal with
> > java stack, other jdb functionality comes almost automatically. Let me start
> > from the final stage of the good "mixed  mode debugger". Ideal mixed mode
> > debugger should open core or dump file of java application  crush  restoring
> > internal state like it do for other application, it should attach and start
> > application in debugging mode. If look on debuggers' history we can see
> > similar task that was done before, e.g. thread handling in debugger. How
> > debugger operates with thread? How it get TSD and TLS from the thread on
> > remote application or core file? A long-long time ago Solaris or Sun OS
> > introduced thread debugging interface libc_db
> > http://docs.sun.com/app/docs/doc/816-5173/6mbb8adsh?l=en&a=view and then it
> > migrated to Linux world (
> > http://sources.redhat.com/cgi-bin/cvsweb.cgi/libc/nptl_db/?cvsroot=glibc)
> > and BSD (http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libpthread_dbg/) world
> > (thread_db). This library inform debugger how to fetch thread information
> > from core file or debugee process. Similar approach could be done with java
> > world but before we need several baby steps.
> >
> > Short-term:
> > 1. Using JVMTI possibilities provide information for debugger to operate
> > with the java stack
> > 2. Provide debugger operations throw JVMTI (break points, watch points and
> > so on)
> >
> > Result construction could used as a source for a long solution described as
> > ideal mixed-mode-debugger.
> >
> > BTW: here some links about ready to use short-term solutions:
> > http://www.hp.com/products1/unix/java/pdfs/07_06Tools_NativeDebug.pdf
> > http://docs.sun.com/source/819-3683/Java_debug.html
> >
> > On 10/12/07, Tim Ellison <t.p.ellison@gmail.com> wrote:
> > >
> > > It looks like there is some good discussion going on around this topic,
> > > and possible alternatives being explored.  Rather than prematurely
> > > forcing the issue with a vote, maybe we should stop the vote and
> > > continue the discussion for a while until we reach a consensus, then
> > > restart the vote.
> > >
> > > What do you think?
> > >
> > > Regards,
> > > Tim
> > >
> > > Vasily Levchenko wrote:
> > > > On 10/12/07, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> > > >> Vasily,
> > > >>
> > > >> You wrote,
> > > >>> As I understand my project is single user of NCAI
> > > >> I believe this is not the only use case for NCAI. Keeping internal
> > > >> layer of debugging interfaces is important for VM development itself.
> > > >
> > > >
> > > > Why are you so sure that it will help you in internal VM development?
> > > you
> > > > can used it for building some kind viewer of state in JNI wrapper, but
> > > not
> > > > more. Your "viewer" can't get debugging services provided by kernel.
> > > You're
> > > > breaking the model that people build for a long time ago and develop new
> > > one
> > > > instead.  You haven't (not personally you but harmony community I guess)
> > > > even try  re-use  the existing solutions.
> > > >
> > > >
> > > > Let me assure you that I'm far from convincing you to support NCAI in
> > > >> your stepchild tool if you don't want to.
> > > >>
> > > >> Thanks.
> > > >>
> > > >> On 10/12/07, Vasily Levchenko < vasily.v.levchenko@gmail.com>
wrote:
> > > >>> On 10/12/07, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> > > >>>> Vasily,
> > > >>>>
> > > >>>> 1.
> > > >>>> You use capitalized wordings like "Mixed Mode" debugging or
> > > >>>> "Integrated Debugger was transferred to me". This sound like
names of
> > >
> > > >>>> internal projects you are participating. It would be great
if you
> > > >>>> start a new thread and disclose your activity to the community
(and
> > > >>>> your humble servant). Or if they are top secret projects,
I have
> > > never
> > > >>>> seen these references on the dev list. :-) Thank you in advance.
> > > >>>
> > > >>>
> > > >>> Yes it's public available semi-product that available on
> > > >>> http://whatif.intel.com, and actually I'm the owner of Integrated
> > > >> debugger
> > > >>> for Java/JNI environments. As I understand my project is single
user
> > > of
> > > >> NCAI
> > > >>> and will available during Q4.
> > > >>>
> > > >>>
> > > >>> 2.
> > > >>>> Sorry for being unaware of projects like libthread_db.so.
I've tried
> > > >>>> to google it and the first search item didn't describe this
project
> > > >>>> well:
> > > >>>>> 'Crashes in /lib/libthread_db.so.1' thread - MARC
> > > >>>> Could you please support your words with references to these
> > > >>>> alternative projects and their licenses?
> > > >>>
> > > >>> libthread_db.so is usual part of system libraries e.g. for GLIbc
is
> > > LGPL
> > > >>> in BSD world  it's  BSDL and so on.
> > > >>>
> > > >>>
> > > >>> 3.
> > > >>>> In any case I believe we need to take the code now, and than
compare
> > > >>>> it with alternative when it is ready. Technologies quickly
change and
> > > >>>> replace each other, and I don't think we should reject a contribution
> > > >>>> counting on its possible future obsolescence.
> > > >>>
> > > >>>
> > > >>> I don't really undestand what harry for?
> > > >>>
> > > >>>
> > > >>> Contrary, our mission is to make today's code a good base for
the new
> > > >>>> development.
> > > >>>
> > > >>> If you really want good code for development, read NetBSD source
tree
> > > >> and
> > > >>> polish current code ;)
> > > >>>
> > > >>>
> > > >>> Any code will become obsolete one day. I believe
> > > >>>> interfaces are the easiest thing to be replaced. Other things
in this
> > > >>>> implementation such as stack frame enumeration will live much
longer
> > > >>>> because there are not too many sane maniacs which are able
to
> > > >>>> understand all our calling conventions.
> > > >>>>
> > > >>>> Thanks!
> > > >>>>
> > > >>>>
> > > >>>> On 10/12/07, Vasily Levchenko < vasily.v.levchenko@gmail.com>
wrote:
> > > >>>>> On 10/12/07, Alexei Fedotov <alexei.fedotov@gmail.com>
wrote:
> > > >>>>>> Hello Vasily,
> > > >>>>>>
> > > >>>>>> Now I see your attitude towards this donation. :-)
When I initally
> > > >>>>>> looked through interface specification, I was not
very supportive
> > > >> and
> > > >>>>>> replied with the same argument about gdb.
> > > >>>>>>
> > > >>>>>> Unfortunately, gdb is not BSD-licensed or Apache compatible.
So we
> > > >>>>>> cannot link with it. It is important if we want to
debug
> > > >> application
> > > >>>>>> in a JVMTI-like way from the same process.
> > > >>>>>
> > > >>>>> It's really doesn't matter you can find the sample of
GDB agents
> > > >> under
> > > >>>> LGPL
> > > >>>>> and BSD license, actually good example libthread_db.so
that provide
> > > >>>> access
> > > >>>>> to libpthread.so internals fo GDB usually it's the same
version as
> > > >> the
> > > >>>> rest
> > > >>>>> of the system libraries.
> > > >>>>>
> > > >>>>>
> > > >>>>> If you like solutions with external debuggers, you may
come up with
> > > >>>>>> one as well as Salikh did two years ago, and this
proved to be
> > > >> useful.
> > > >>>>>> This just does not help in two areas I've initially
outlined:
> > > >>>>>> defensive programming and crash handling.
> > > >>>>>
> > > >>>>>
> > > >>>>> It were actually one of the lack of good crash handling
that we
> > > >>>> discussed
> > > >>>>> when the Integrated Debugger was transfered to me.
> > > >>>>>
> > > >>>>>
> > > >>>>> Having a level of internal
> > > >>>>>> reflection from inside VM is very important to express
assertions
> > > >>>>>> about correct VM state for a debug build.
> > > >>>>>
> > > >>>>>
> > > >>>>> Yes, it's true but it should be done in similar way as
thread_db
> > > >>>> interface
> > > >>>>> which could provide even core file analyzing, but with
in-process
> > > >>>> debugger
> > > >>>>> it is impossible or very hard to implement.
> > > >>>>>
> > > >>>>>
> > > >>>>> Finally I want to address your concern of a "pure viewing"
nature of
> > >
> > > >>>>>> NCAI. NCAI allows changes in runtime environment.
For example, you
> > > >> can
> > > >>>>>> change values of variables, or any memory location.
> > > >>>>>
> > > >>>>>
> > > >>>>> GDB or any other out-of-process debugger provides the
same
> > > >> functionality
> > > >>>>> without any side-effects.
> > > >>>>>
> > > >>>>>
> > > >>>>> Thanks.
> > > >>>>>>
> > > >>>>>> On 10/12/07, Vasily Levchenko <vasily.v.levchenko@gmail.com>
> > > >> wrote:
> > > >>>>>>> On 10/12/07, Alexei Fedotov < alexei.fedotov@gmail.com>
wrote:
> > > >>>>>>>> +1
> > > >>>>>>>> NCAI adds very useful architectural level
for VM internal
> > > >>>> debugging
> > > >>>>>>>> and defensive programming. The best example
is a crash
> > > >> handler,
> > > >>>> but
> > > >>>>>>>> other utilities such us self-detection of
deadlocks, is also
> > > >> quite
> > > >>>>>>>> interesting.
> > > >>>>>>>
> > > >>>>>>> NCAI could be interface to build some kind of
viewer, but no
> > > >> real
> > > >>>>>> debugger.
> > > >>>>>>> 1. It couldn't be very useful  just because it's
single process
> > > >>>>>> debugging
> > > >>>>>>> model and if you set breakpoint incorrectly you
could lock whole
> > > >>>> JVM.
> > > >>>>>>> 2. building debugger from the very beginning you
try to invent
> > > >> wheel
> > > >>>>>> once
> > > >>>>>>> again, you have to develop new mechanisms that
already exists in
> > > >>>> native
> > > >>>>>>> debuggers.
> > > >>>>>>> 3. building infrastructure for Mixed  Mode debugging
using
> > > >> native
> > > >>>>>> debugger
> > > >>>>>>> could be implemented in 1-2 month and it'll be
really powerful
> > > >> and
> > > >>>>>> useful
> > > >>>>>>> solution especially JNI developers.
> > > >>>>>>>
> > > >>>>>>> HP(gdb) and Sun (dbx) already introduced native
debugger-centric
> > > >>>>>> solutions
> > > >>>>>>> for debugging in mixed environments. So NCAI seams
to be
> > > >> potential
> > > >>>> dead
> > > >>>>>> code
> > > >>>>>>> and I won't be surprised if year or two later
you'll be voting
> > > >> for
> > > >>>>>> removing
> > > >>>>>>> this code.
> > > >>>>>>>
> > > >>>>>>> On 10/12/07, Vasily Levchenko <vasily.v.levchenko@gmail.com>
> > > >> wrote:
> > > >>>>>>>>> Hi Mikhail,
> > > >>>>>>>>> Have you got any idea how to use it? Except
"Integrated
> > > >> Debugger
> > > >>>> for
> > > >>>>>>>>> Java/JNI environment" (former MMD).
> > > >>>>>>>>>
> > > >>>>>>>>> On 10/12/07, Mikhail Loenko < mloenko@gmail.com>
wrote:
> > > >>>>>>>>>> We now have all requisite paperwork
in place for
> > > >>>>>>>>>> http://issues.apache.org/jira/browse/HARMONY-4689,
so
> > > >> let's
> > > >>>> vote
> > > >>>>>> to
> > > >>>>>>>>>> accept:
> > > >>>>>>>>>>
> > > >>>>>>>>>> [ ] +1 Accept this contribution
> > > >>>>>>>>>> [ ] -1 Reject because...
> > > >>>>>>>>>>
> > > >>>>>>>>>> As usual, 3 days unless someone complains
they need more
> > > >> time.
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> --
> > > >>>>>>>>> --vvl
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> --
> > > >>>>>>>> With best regards,
> > > >>>>>>>> Alexei,
> > > >>>>>>>> ESSD, Intel
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> --
> > > >>>>>>> --vvl
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>> --
> > > >>>>>> With best regards,
> > > >>>>>> Alexei,
> > > >>>>>> ESSD, Intel
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> --
> > > >>>>> --vvl
> > > >>>>>
> > > >>>>
> > > >>>> --
> > > >>>> With best regards,
> > > >>>> Alexei,
> > > >>>> ESSD, Intel
> > > >>>>
> > > >>>
> > > >>>
> > > >>> --
> > > >>> --vvl
> > > >>>
> > > >>
> > > >> --
> > > >> With best regards,
> > > >> Alexei,
> > > >> ESSD, Intel
> > > >>
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > --vvl
> >
>
>
> --
> With best regards,
> Alexei,
> ESSD, Intel
>


-- 

Ilya

Mime
View raw message