harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vasily Levchenko" <vasily.v.levche...@gmail.com>
Subject Re: Thoughts...
Date Tue, 16 Oct 2007 06:25:31 GMT
Hello Ilya,

On 10/15/07, Ilya Berezhniuk <ilya.berezhniuk@gmail.com> wrote:
>
> Hello Vasily,
>
> Both these approaches (NCAI and external debugger) have their own
> advantages. I'll describe a couple of major NCAI advantages:
>
> 1) Using two debuggers at once looks risky for me; one debugger
> extending JVMTI functionality is safer. There are several areas of
> possible conflicts between the debuggers. It's breakpoints handling -
> some caught breakpoints should be processed by the external debugger,
> and some of them should be delivered to JVMTI. I guess there can be
> possible conflicts while working with threads - only VM internal code
> can take care of Java threads specifics. There can be other
> conflicting areas.


Conflicts you've mentioned caused by breakpoint in native debugger that
prevents continue of  execution in JVM and stop the java world, but this
conflicts easy to resolve by continue JVM execution. When you place debugger
inside JVM you can set up the breakpoint in the crossing area of JNI and JVM
code that cause blocking of whole VM and that will require restarting of
application and JVM itself, that is a reason why in Integrated Debugger for
Java/JNI environments we don't allow to set breakpoint anywhere outside of
JNI library. That why NCAI still is not the cure from two debugger scenario
and we have  to attach with native debugger to debug real native library.
But of course, assuming that JNI library in the most cases is wrapper for
well-designed and tested library we don't need additional debugger.


2) AFAIK, gdb cannot enumerate the whole stack of Java thread
> correctly, because it cannot continue unwinding native stack below
> Java frames.


To make it possible you need add java-unwinder to GDB or extension in case
of CDB ("Debugging Tools for Windows") (in short-term communicated with
JVMTI agent) and control JNI downcalls and upcalls for correct switching
between unwinders (dwarf-2 and java). Similar things was done for debugging
PHP with gdb.


Also gdb is unable to unwind native stack when the code
> is called from JVMTI breakpoint/single step handling routine.


that will be automatically solved with java-unwinder in GDB. The next
problem license dealing, that actually why I put several times  the
references to thread_db interface GDB is usually linked against that
libaries, so why they aren't GPL, but anyway java-unwinder will be GPL'ed,
but the rest of the code including JDB-logic library working in GDB address
space still can be Apache licensed or any other.


Moreover, gdb often provides native stack that is worse than the stack
> provided by the heuristic stack walker I've implemented for NCAI.
> (This stack walker is already integrated for using by the crash
> handler, so NCAI Frame functions are simply wrappers.)
> To tell the truth, the above statement is correct for IA-32; AMD64
> calling conventions prevent heuristic stack unwinding. AFAIU, GDB can
> use dwarf stack info for unwinding - so GDB could provide better stack
> on x86_64.
>
>
> 2007/10/14, Alexei Fedotov <alexei.fedotov@gmail.com>:
> > Hello Vasily,
> >
> > I like your constructive overview. I agree with you that traversing
> > managed frames is quite VM specific and proves to be a problem for
> > native debuggers.
> >
> > To my personal point of view, the complexity of using external thread
> > libraries is underestimated in your analysis since VM uses several
> > self-made synchronization primitives which do most of the job. For
> > example, not far from now gdb on IPF crashed in a simple test.
> >
> > Windows threading model may prove some unexpected challenges. For
> > example, Ivan Volosyuk spent a day trying to understand that our VM
> > thread local data should be prefixed with unused field to made
> > debugging possible. I faced a number of gdb crashes on Windows.
> >
> > As for your questions of alternatives, a year ago I investigated cross
> > platform alternatives for native frame enumeration. I considered gdb
> > and a debugger from Intel compiler. I talked to people who developed
> > the debuggers, and then decided to use our internal solution Ilya was
> > finalizing.
> >
> > Thank you for your attention.
> >
> > BTW, I believe we should not mix two things in our discussion. I like
> > if someone will come up with better solution: many places of DRLVM can
> > be improved. From the other side casting a vote
> >
> > BTW2, the file pthread_dbg.h you've provided a reference to doesn't
> > have a pure BSD license, see the third item.
> >
> > On 10/13/07, Vasily Levchenko <vasily.v.levchenko@gmail.com> wrote:
> > > Originally,  Java developers using JNI have to debug their application
> using
> > > two debugger for accessing java and native world(
> > >
> http://www-128.ibm.com/developerworks/java/library/j-jnidebug/index.html,
> > >
> http://developers.sun.com/learning/javaoneonline/2006/tools/TS-1011.pdf).
> > > This concept should be the foundation for "mixed" debugger. Native
> debugger
> > > (gdb, adb or other) to debug java world need information how to deal
> with
> > > java stack, other jdb functionality comes almost automatically. Let me
> start
> > > from the final stage of the good "mixed  mode debugger". Ideal mixed
> mode
> > > debugger should open core or dump file of java
> application  crush  restoring
> > > internal state like it do for other application, it should attach and
> start
> > > application in debugging mode. If look on debuggers' history we can
> see
> > > similar task that was done before, e.g. thread handling in debugger.
> How
> > > debugger operates with thread? How it get TSD and TLS from the thread
> on
> > > remote application or core file? A long-long time ago Solaris or Sun
> OS
> > > introduced thread debugging interface libc_db
> > > http://docs.sun.com/app/docs/doc/816-5173/6mbb8adsh?l=en&a=view and
> then it
> > > migrated to Linux world (
> > >
> http://sources.redhat.com/cgi-bin/cvsweb.cgi/libc/nptl_db/?cvsroot=glibc)
> > > and BSD (http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libpthread_dbg/)
> world
> > > (thread_db). This library inform debugger how to fetch thread
> information
> > > from core file or debugee process. Similar approach could be done with
> java
> > > world but before we need several baby steps.
> > >
> > > Short-term:
> > > 1. Using JVMTI possibilities provide information for debugger to
> operate
> > > with the java stack
> > > 2. Provide debugger operations throw JVMTI (break points, watch points
> and
> > > so on)
> > >
> > > Result construction could used as a source for a long solution
> described as
> > > ideal mixed-mode-debugger.
> > >
> > > BTW: here some links about ready to use short-term solutions:
> > > http://www.hp.com/products1/unix/java/pdfs/07_06Tools_NativeDebug.pdf
> > > http://docs.sun.com/source/819-3683/Java_debug.html
> > >
> > > On 10/12/07, Tim Ellison <t.p.ellison@gmail.com> wrote:
> > > >
> > > > It looks like there is some good discussion going on around this
> topic,
> > > > and possible alternatives being explored.  Rather than prematurely
> > > > forcing the issue with a vote, maybe we should stop the vote and
> > > > continue the discussion for a while until we reach a consensus, then
> > > > restart the vote.
> > > >
> > > > What do you think?
> > > >
> > > > Regards,
> > > > Tim
> > > >
> > > > Vasily Levchenko wrote:
> > > > > On 10/12/07, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> > > > >> Vasily,
> > > > >>
> > > > >> You wrote,
> > > > >>> As I understand my project is single user of NCAI
> > > > >> I believe this is not the only use case for NCAI. Keeping
> internal
> > > > >> layer of debugging interfaces is important for VM development
> itself.
> > > > >
> > > > >
> > > > > Why are you so sure that it will help you in internal VM
> development?
> > > > you
> > > > > can used it for building some kind viewer of state in JNI wrapper,
> but
> > > > not
> > > > > more. Your "viewer" can't get debugging services provided by
> kernel.
> > > > You're
> > > > > breaking the model that people build for a long time ago and
> develop new
> > > > one
> > > > > instead.  You haven't (not personally you but harmony community I
> guess)
> > > > > even try  re-use  the existing solutions.
> > > > >
> > > > >
> > > > > Let me assure you that I'm far from convincing you to support NCAI
> in
> > > > >> your stepchild tool if you don't want to.
> > > > >>
> > > > >> Thanks.
> > > > >>
> > > > >> On 10/12/07, Vasily Levchenko < vasily.v.levchenko@gmail.com>
> wrote:
> > > > >>> On 10/12/07, Alexei Fedotov <alexei.fedotov@gmail.com>
wrote:
> > > > >>>> Vasily,
> > > > >>>>
> > > > >>>> 1.
> > > > >>>> You use capitalized wordings like "Mixed Mode" debugging
or
> > > > >>>> "Integrated Debugger was transferred to me". This sound
like
> names of
> > > >
> > > > >>>> internal projects you are participating. It would be
great if
> you
> > > > >>>> start a new thread and disclose your activity to the
community
> (and
> > > > >>>> your humble servant). Or if they are top secret projects,
I
> have
> > > > never
> > > > >>>> seen these references on the dev list. :-) Thank you
in
> advance.
> > > > >>>
> > > > >>>
> > > > >>> Yes it's public available semi-product that available on
> > > > >>> http://whatif.intel.com, and actually I'm the owner of
> Integrated
> > > > >> debugger
> > > > >>> for Java/JNI environments. As I understand my project is
single
> user
> > > > of
> > > > >> NCAI
> > > > >>> and will available during Q4.
> > > > >>>
> > > > >>>
> > > > >>> 2.
> > > > >>>> Sorry for being unaware of projects like libthread_db.so.
I've
> tried
> > > > >>>> to google it and the first search item didn't describe
this
> project
> > > > >>>> well:
> > > > >>>>> 'Crashes in /lib/libthread_db.so.1' thread - MARC
> > > > >>>> Could you please support your words with references to
these
> > > > >>>> alternative projects and their licenses?
> > > > >>>
> > > > >>> libthread_db.so is usual part of system libraries e.g. for
GLIbc
> is
> > > > LGPL
> > > > >>> in BSD world  it's  BSDL and so on.
> > > > >>>
> > > > >>>
> > > > >>> 3.
> > > > >>>> In any case I believe we need to take the code now, and
than
> compare
> > > > >>>> it with alternative when it is ready. Technologies quickly
> change and
> > > > >>>> replace each other, and I don't think we should reject
a
> contribution
> > > > >>>> counting on its possible future obsolescence.
> > > > >>>
> > > > >>>
> > > > >>> I don't really undestand what harry for?
> > > > >>>
> > > > >>>
> > > > >>> Contrary, our mission is to make today's code a good base
for
> the new
> > > > >>>> development.
> > > > >>>
> > > > >>> If you really want good code for development, read NetBSD
source
> tree
> > > > >> and
> > > > >>> polish current code ;)
> > > > >>>
> > > > >>>
> > > > >>> Any code will become obsolete one day. I believe
> > > > >>>> interfaces are the easiest thing to be replaced. Other
things
> in this
> > > > >>>> implementation such as stack frame enumeration will live
much
> longer
> > > > >>>> because there are not too many sane maniacs which are
able to
> > > > >>>> understand all our calling conventions.
> > > > >>>>
> > > > >>>> Thanks!
> > > > >>>>
> > > > >>>>
> > > > >>>> On 10/12/07, Vasily Levchenko < vasily.v.levchenko@gmail.com>
> wrote:
> > > > >>>>> On 10/12/07, Alexei Fedotov <alexei.fedotov@gmail.com>
wrote:
> > > > >>>>>> Hello Vasily,
> > > > >>>>>>
> > > > >>>>>> Now I see your attitude towards this donation.
:-) When I
> initally
> > > > >>>>>> looked through interface specification, I was
not very
> supportive
> > > > >> and
> > > > >>>>>> replied with the same argument about gdb.
> > > > >>>>>>
> > > > >>>>>> Unfortunately, gdb is not BSD-licensed or Apache
compatible.
> So we
> > > > >>>>>> cannot link with it. It is important if we want
to debug
> > > > >> application
> > > > >>>>>> in a JVMTI-like way from the same process.
> > > > >>>>>
> > > > >>>>> It's really doesn't matter you can find the sample
of GDB
> agents
> > > > >> under
> > > > >>>> LGPL
> > > > >>>>> and BSD license, actually good example libthread_db.so
that
> provide
> > > > >>>> access
> > > > >>>>> to libpthread.so internals fo GDB usually it's the
same
> version as
> > > > >> the
> > > > >>>> rest
> > > > >>>>> of the system libraries.
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> If you like solutions with external debuggers, you
may come up
> with
> > > > >>>>>> one as well as Salikh did two years ago, and
this proved to
> be
> > > > >> useful.
> > > > >>>>>> This just does not help in two areas I've initially
outlined:
> > > > >>>>>> defensive programming and crash handling.
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> It were actually one of the lack of good crash handling
that
> we
> > > > >>>> discussed
> > > > >>>>> when the Integrated Debugger was transfered to me.
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> Having a level of internal
> > > > >>>>>> reflection from inside VM is very important to
express
> assertions
> > > > >>>>>> about correct VM state for a debug build.
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> Yes, it's true but it should be done in similar way
as
> thread_db
> > > > >>>> interface
> > > > >>>>> which could provide even core file analyzing, but
with
> in-process
> > > > >>>> debugger
> > > > >>>>> it is impossible or very hard to implement.
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> Finally I want to address your concern of a "pure
viewing"
> nature of
> > > >
> > > > >>>>>> NCAI. NCAI allows changes in runtime environment.
For
> example, you
> > > > >> can
> > > > >>>>>> change values of variables, or any memory location.
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> GDB or any other out-of-process debugger provides
the same
> > > > >> functionality
> > > > >>>>> without any side-effects.
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> Thanks.
> > > > >>>>>>
> > > > >>>>>> On 10/12/07, Vasily Levchenko <vasily.v.levchenko@gmail.com>
> > > > >> wrote:
> > > > >>>>>>> On 10/12/07, Alexei Fedotov < alexei.fedotov@gmail.com>
> wrote:
> > > > >>>>>>>> +1
> > > > >>>>>>>> NCAI adds very useful architectural level
for VM internal
> > > > >>>> debugging
> > > > >>>>>>>> and defensive programming. The best example
is a crash
> > > > >> handler,
> > > > >>>> but
> > > > >>>>>>>> other utilities such us self-detection
of deadlocks, is
> also
> > > > >> quite
> > > > >>>>>>>> interesting.
> > > > >>>>>>>
> > > > >>>>>>> NCAI could be interface to build some kind
of viewer, but no
> > > > >> real
> > > > >>>>>> debugger.
> > > > >>>>>>> 1. It couldn't be very useful  just because
it's single
> process
> > > > >>>>>> debugging
> > > > >>>>>>> model and if you set breakpoint incorrectly
you could lock
> whole
> > > > >>>> JVM.
> > > > >>>>>>> 2. building debugger from the very beginning
you try to
> invent
> > > > >> wheel
> > > > >>>>>> once
> > > > >>>>>>> again, you have to develop new mechanisms
that already
> exists in
> > > > >>>> native
> > > > >>>>>>> debuggers.
> > > > >>>>>>> 3. building infrastructure for Mixed  Mode
debugging using
> > > > >> native
> > > > >>>>>> debugger
> > > > >>>>>>> could be implemented in 1-2 month and it'll
be really
> powerful
> > > > >> and
> > > > >>>>>> useful
> > > > >>>>>>> solution especially JNI developers.
> > > > >>>>>>>
> > > > >>>>>>> HP(gdb) and Sun (dbx) already introduced
native
> debugger-centric
> > > > >>>>>> solutions
> > > > >>>>>>> for debugging in mixed environments. So NCAI
seams to be
> > > > >> potential
> > > > >>>> dead
> > > > >>>>>> code
> > > > >>>>>>> and I won't be surprised if year or two later
you'll be
> voting
> > > > >> for
> > > > >>>>>> removing
> > > > >>>>>>> this code.
> > > > >>>>>>>
> > > > >>>>>>> On 10/12/07, Vasily Levchenko <vasily.v.levchenko@gmail.com>
> > > > >> wrote:
> > > > >>>>>>>>> Hi Mikhail,
> > > > >>>>>>>>> Have you got any idea how to use
it? Except "Integrated
> > > > >> Debugger
> > > > >>>> for
> > > > >>>>>>>>> Java/JNI environment" (former MMD).
> > > > >>>>>>>>>
> > > > >>>>>>>>> On 10/12/07, Mikhail Loenko <
mloenko@gmail.com> wrote:
> > > > >>>>>>>>>> We now have all requisite paperwork
in place for
> > > > >>>>>>>>>> http://issues.apache.org/jira/browse/HARMONY-4689,
so
> > > > >> let's
> > > > >>>> vote
> > > > >>>>>> to
> > > > >>>>>>>>>> accept:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> [ ] +1 Accept this contribution
> > > > >>>>>>>>>> [ ] -1 Reject because...
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> As usual, 3 days unless someone
complains they need more
> > > > >> time.
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>> --
> > > > >>>>>>>>> --vvl
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> --
> > > > >>>>>>>> With best regards,
> > > > >>>>>>>> Alexei,
> > > > >>>>>>>> ESSD, Intel
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> --
> > > > >>>>>>> --vvl
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>> --
> > > > >>>>>> With best regards,
> > > > >>>>>> Alexei,
> > > > >>>>>> ESSD, Intel
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> --
> > > > >>>>> --vvl
> > > > >>>>>
> > > > >>>>
> > > > >>>> --
> > > > >>>> With best regards,
> > > > >>>> Alexei,
> > > > >>>> ESSD, Intel
> > > > >>>>
> > > > >>>
> > > > >>>
> > > > >>> --
> > > > >>> --vvl
> > > > >>>
> > > > >>
> > > > >> --
> > > > >> With best regards,
> > > > >> Alexei,
> > > > >> ESSD, Intel
> > > > >>
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > --vvl
> > >
> >
> >
> > --
> > With best regards,
> > Alexei,
> > ESSD, Intel
> >
>
>
> --
>
> Ilya
>



-- 
--vvl

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message