harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Fedotov" <alexei.fedo...@gmail.com>
Subject Re: Thoughts... (was: Re: [vote] HARMONY-4689 - JVMTI Extension: Native Code Access Interface implementation)
Date Sun, 14 Oct 2007 18:14:29 GMT
Hello Vasily,

I like your constructive overview. I agree with you that traversing
managed frames is quite VM specific and proves to be a problem for
native debuggers.

To my personal point of view, the complexity of using external thread
libraries is underestimated in your analysis since VM uses several
self-made synchronization primitives which do most of the job. For
example, not far from now gdb on IPF crashed in a simple test.

Windows threading model may prove some unexpected challenges. For
example, Ivan Volosyuk spent a day trying to understand that our VM
thread local data should be prefixed with unused field to made
debugging possible. I faced a number of gdb crashes on Windows.

As for your questions of alternatives, a year ago I investigated cross
platform alternatives for native frame enumeration. I considered gdb
and a debugger from Intel compiler. I talked to people who developed
the debuggers, and then decided to use our internal solution Ilya was
finalizing.

Thank you for your attention.

BTW, I believe we should not mix two things in our discussion. I like
if someone will come up with better solution: many places of DRLVM can
be improved. From the other side casting a vote

BTW2, the file pthread_dbg.h you've provided a reference to doesn't
have a pure BSD license, see the third item.

On 10/13/07, Vasily Levchenko <vasily.v.levchenko@gmail.com> wrote:
> Originally,  Java developers using JNI have to debug their application using
> two debugger for accessing java and native world(
> http://www-128.ibm.com/developerworks/java/library/j-jnidebug/index.html,
> http://developers.sun.com/learning/javaoneonline/2006/tools/TS-1011.pdf).
> This concept should be the foundation for "mixed" debugger. Native debugger
> (gdb, adb or other) to debug java world need information how to deal with
> java stack, other jdb functionality comes almost automatically. Let me start
> from the final stage of the good "mixed  mode debugger". Ideal mixed mode
> debugger should open core or dump file of java application  crush  restoring
> internal state like it do for other application, it should attach and start
> application in debugging mode. If look on debuggers' history we can see
> similar task that was done before, e.g. thread handling in debugger. How
> debugger operates with thread? How it get TSD and TLS from the thread on
> remote application or core file? A long-long time ago Solaris or Sun OS
> introduced thread debugging interface libc_db
> http://docs.sun.com/app/docs/doc/816-5173/6mbb8adsh?l=en&a=view and then it
> migrated to Linux world (
> http://sources.redhat.com/cgi-bin/cvsweb.cgi/libc/nptl_db/?cvsroot=glibc)
> and BSD (http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libpthread_dbg/) world
> (thread_db). This library inform debugger how to fetch thread information
> from core file or debugee process. Similar approach could be done with java
> world but before we need several baby steps.
>
> Short-term:
> 1. Using JVMTI possibilities provide information for debugger to operate
> with the java stack
> 2. Provide debugger operations throw JVMTI (break points, watch points and
> so on)
>
> Result construction could used as a source for a long solution described as
> ideal mixed-mode-debugger.
>
> BTW: here some links about ready to use short-term solutions:
> http://www.hp.com/products1/unix/java/pdfs/07_06Tools_NativeDebug.pdf
> http://docs.sun.com/source/819-3683/Java_debug.html
>
> On 10/12/07, Tim Ellison <t.p.ellison@gmail.com> wrote:
> >
> > It looks like there is some good discussion going on around this topic,
> > and possible alternatives being explored.  Rather than prematurely
> > forcing the issue with a vote, maybe we should stop the vote and
> > continue the discussion for a while until we reach a consensus, then
> > restart the vote.
> >
> > What do you think?
> >
> > Regards,
> > Tim
> >
> > Vasily Levchenko wrote:
> > > On 10/12/07, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> > >> Vasily,
> > >>
> > >> You wrote,
> > >>> As I understand my project is single user of NCAI
> > >> I believe this is not the only use case for NCAI. Keeping internal
> > >> layer of debugging interfaces is important for VM development itself.
> > >
> > >
> > > Why are you so sure that it will help you in internal VM development?
> > you
> > > can used it for building some kind viewer of state in JNI wrapper, but
> > not
> > > more. Your "viewer" can't get debugging services provided by kernel.
> > You're
> > > breaking the model that people build for a long time ago and develop new
> > one
> > > instead.  You haven't (not personally you but harmony community I guess)
> > > even try  re-use  the existing solutions.
> > >
> > >
> > > Let me assure you that I'm far from convincing you to support NCAI in
> > >> your stepchild tool if you don't want to.
> > >>
> > >> Thanks.
> > >>
> > >> On 10/12/07, Vasily Levchenko < vasily.v.levchenko@gmail.com> wrote:
> > >>> On 10/12/07, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> > >>>> Vasily,
> > >>>>
> > >>>> 1.
> > >>>> You use capitalized wordings like "Mixed Mode" debugging or
> > >>>> "Integrated Debugger was transferred to me". This sound like names
of
> >
> > >>>> internal projects you are participating. It would be great if you
> > >>>> start a new thread and disclose your activity to the community
(and
> > >>>> your humble servant). Or if they are top secret projects, I have
> > never
> > >>>> seen these references on the dev list. :-) Thank you in advance.
> > >>>
> > >>>
> > >>> Yes it's public available semi-product that available on
> > >>> http://whatif.intel.com, and actually I'm the owner of Integrated
> > >> debugger
> > >>> for Java/JNI environments. As I understand my project is single user
> > of
> > >> NCAI
> > >>> and will available during Q4.
> > >>>
> > >>>
> > >>> 2.
> > >>>> Sorry for being unaware of projects like libthread_db.so. I've
tried
> > >>>> to google it and the first search item didn't describe this project
> > >>>> well:
> > >>>>> 'Crashes in /lib/libthread_db.so.1' thread - MARC
> > >>>> Could you please support your words with references to these
> > >>>> alternative projects and their licenses?
> > >>>
> > >>> libthread_db.so is usual part of system libraries e.g. for GLIbc is
> > LGPL
> > >>> in BSD world  it's  BSDL and so on.
> > >>>
> > >>>
> > >>> 3.
> > >>>> In any case I believe we need to take the code now, and than compare
> > >>>> it with alternative when it is ready. Technologies quickly change
and
> > >>>> replace each other, and I don't think we should reject a contribution
> > >>>> counting on its possible future obsolescence.
> > >>>
> > >>>
> > >>> I don't really undestand what harry for?
> > >>>
> > >>>
> > >>> Contrary, our mission is to make today's code a good base for the new
> > >>>> development.
> > >>>
> > >>> If you really want good code for development, read NetBSD source tree
> > >> and
> > >>> polish current code ;)
> > >>>
> > >>>
> > >>> Any code will become obsolete one day. I believe
> > >>>> interfaces are the easiest thing to be replaced. Other things in
this
> > >>>> implementation such as stack frame enumeration will live much longer
> > >>>> because there are not too many sane maniacs which are able to
> > >>>> understand all our calling conventions.
> > >>>>
> > >>>> Thanks!
> > >>>>
> > >>>>
> > >>>> On 10/12/07, Vasily Levchenko < vasily.v.levchenko@gmail.com>
wrote:
> > >>>>> On 10/12/07, Alexei Fedotov <alexei.fedotov@gmail.com>
wrote:
> > >>>>>> Hello Vasily,
> > >>>>>>
> > >>>>>> Now I see your attitude towards this donation. :-) When
I initally
> > >>>>>> looked through interface specification, I was not very
supportive
> > >> and
> > >>>>>> replied with the same argument about gdb.
> > >>>>>>
> > >>>>>> Unfortunately, gdb is not BSD-licensed or Apache compatible.
So we
> > >>>>>> cannot link with it. It is important if we want to debug
> > >> application
> > >>>>>> in a JVMTI-like way from the same process.
> > >>>>>
> > >>>>> It's really doesn't matter you can find the sample of GDB agents
> > >> under
> > >>>> LGPL
> > >>>>> and BSD license, actually good example libthread_db.so that
provide
> > >>>> access
> > >>>>> to libpthread.so internals fo GDB usually it's the same version
as
> > >> the
> > >>>> rest
> > >>>>> of the system libraries.
> > >>>>>
> > >>>>>
> > >>>>> If you like solutions with external debuggers, you may come
up with
> > >>>>>> one as well as Salikh did two years ago, and this proved
to be
> > >> useful.
> > >>>>>> This just does not help in two areas I've initially outlined:
> > >>>>>> defensive programming and crash handling.
> > >>>>>
> > >>>>>
> > >>>>> It were actually one of the lack of good crash handling that
we
> > >>>> discussed
> > >>>>> when the Integrated Debugger was transfered to me.
> > >>>>>
> > >>>>>
> > >>>>> Having a level of internal
> > >>>>>> reflection from inside VM is very important to express
assertions
> > >>>>>> about correct VM state for a debug build.
> > >>>>>
> > >>>>>
> > >>>>> Yes, it's true but it should be done in similar way as thread_db
> > >>>> interface
> > >>>>> which could provide even core file analyzing, but with in-process
> > >>>> debugger
> > >>>>> it is impossible or very hard to implement.
> > >>>>>
> > >>>>>
> > >>>>> Finally I want to address your concern of a "pure viewing"
nature of
> >
> > >>>>>> NCAI. NCAI allows changes in runtime environment. For example,
you
> > >> can
> > >>>>>> change values of variables, or any memory location.
> > >>>>>
> > >>>>>
> > >>>>> GDB or any other out-of-process debugger provides the same
> > >> functionality
> > >>>>> without any side-effects.
> > >>>>>
> > >>>>>
> > >>>>> Thanks.
> > >>>>>>
> > >>>>>> On 10/12/07, Vasily Levchenko <vasily.v.levchenko@gmail.com>
> > >> wrote:
> > >>>>>>> On 10/12/07, Alexei Fedotov < alexei.fedotov@gmail.com>
wrote:
> > >>>>>>>> +1
> > >>>>>>>> NCAI adds very useful architectural level for VM
internal
> > >>>> debugging
> > >>>>>>>> and defensive programming. The best example is
a crash
> > >> handler,
> > >>>> but
> > >>>>>>>> other utilities such us self-detection of deadlocks,
is also
> > >> quite
> > >>>>>>>> interesting.
> > >>>>>>>
> > >>>>>>> NCAI could be interface to build some kind of viewer,
but no
> > >> real
> > >>>>>> debugger.
> > >>>>>>> 1. It couldn't be very useful  just because it's single
process
> > >>>>>> debugging
> > >>>>>>> model and if you set breakpoint incorrectly you could
lock whole
> > >>>> JVM.
> > >>>>>>> 2. building debugger from the very beginning you try
to invent
> > >> wheel
> > >>>>>> once
> > >>>>>>> again, you have to develop new mechanisms that already
exists in
> > >>>> native
> > >>>>>>> debuggers.
> > >>>>>>> 3. building infrastructure for Mixed  Mode debugging
using
> > >> native
> > >>>>>> debugger
> > >>>>>>> could be implemented in 1-2 month and it'll be really
powerful
> > >> and
> > >>>>>> useful
> > >>>>>>> solution especially JNI developers.
> > >>>>>>>
> > >>>>>>> HP(gdb) and Sun (dbx) already introduced native debugger-centric
> > >>>>>> solutions
> > >>>>>>> for debugging in mixed environments. So NCAI seams
to be
> > >> potential
> > >>>> dead
> > >>>>>> code
> > >>>>>>> and I won't be surprised if year or two later you'll
be voting
> > >> for
> > >>>>>> removing
> > >>>>>>> this code.
> > >>>>>>>
> > >>>>>>> On 10/12/07, Vasily Levchenko <vasily.v.levchenko@gmail.com>
> > >> wrote:
> > >>>>>>>>> Hi Mikhail,
> > >>>>>>>>> Have you got any idea how to use it? Except
"Integrated
> > >> Debugger
> > >>>> for
> > >>>>>>>>> Java/JNI environment" (former MMD).
> > >>>>>>>>>
> > >>>>>>>>> On 10/12/07, Mikhail Loenko < mloenko@gmail.com>
wrote:
> > >>>>>>>>>> We now have all requisite paperwork in
place for
> > >>>>>>>>>> http://issues.apache.org/jira/browse/HARMONY-4689,
so
> > >> let's
> > >>>> vote
> > >>>>>> to
> > >>>>>>>>>> accept:
> > >>>>>>>>>>
> > >>>>>>>>>> [ ] +1 Accept this contribution
> > >>>>>>>>>> [ ] -1 Reject because...
> > >>>>>>>>>>
> > >>>>>>>>>> As usual, 3 days unless someone complains
they need more
> > >> time.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> --
> > >>>>>>>>> --vvl
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> With best regards,
> > >>>>>>>> Alexei,
> > >>>>>>>> ESSD, Intel
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> --vvl
> > >>>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> With best regards,
> > >>>>>> Alexei,
> > >>>>>> ESSD, Intel
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> --vvl
> > >>>>>
> > >>>>
> > >>>> --
> > >>>> With best regards,
> > >>>> Alexei,
> > >>>> ESSD, Intel
> > >>>>
> > >>>
> > >>>
> > >>> --
> > >>> --vvl
> > >>>
> > >>
> > >> --
> > >> With best regards,
> > >> Alexei,
> > >> ESSD, Intel
> > >>
> > >
> > >
> > >
> >
>
>
>
> --
> --vvl
>


-- 
With best regards,
Alexei,
ESSD, Intel

Mime
View raw message