Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (herse.apache.org: domain of xiaofeng.li@gmail.com
 designates 66.249.82.225 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
        b=k9NUXQI5kfKoh5xZuI6dKinVQiTgOKxQfSGxy4posupedgLs9UA717JgnRVUlRvQd0I2DqOOUpFHmjkhR7lJISd7fqiXi90RMmGHU/UIacULm9C5LsXLaV7WkvjdwEfjQl1JxMfA7J1KlA4FLc67uLwPiBcEn4Uzu8ZLgZsqVHo=
Message-ID: <9623c9a50703200007m23ef04f0ka721d175ce567b3@mail.gmail.com>
Date: Tue, 20 Mar 2007 15:07:32 +0800
From: "Xiao-Feng Li" <xiaofeng.li@gmail.com>
To: dev@harmony.apache.org
Subject: Re: [drlvm][threading] using a thread verification tool to find
 deadlock and race conditions
In-Reply-To: <300B70C65DCD59468957941A60716EBC627C37@mssmsx411>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <9623c9a50703101940r61dd9c3fp757bb7e0a3959ab8@mail.gmail.com>
	 <300B70C65DCD59468957941A60716EBC627C37@mssmsx411>

Thanks, Ilia, for the detailed explanation.

I personally think it's a good idea to make DRLVM work with Thread Checker.

Thanks,
xiaofeng

On 3/20/07, Leviev, Ilia A <ilia.a.leviev@intel.com> wrote:
>
> Hi,
>
> "Xiao-Feng Li <xiaofeng.li@gmail.com> wrote:
> > We need understand what are those "full benefit" DRLVM can get from
> > Thread Checker.
>
> Full benefit - is ability to check threading problems with respect to
> not only standard APIs for Windows and POSIX but also with respect to
> User-Defined (non-standard) Synchronization constructs.
>
> > And, as a system software, I assume the threading
> > infrastructure will not change dramatically once it's stable.
>
> Imbedding of Thread Checker API will not change threading
> infrastructure, it will mark a non-standard synchronization constructs
> without impact on behavior/performance.
>
> Here is example of such marking.
>
> Harmony code contains such non-standard synchronization constructs. For
> example spin lock synchronization functions spin_lock() and
> spin_unlock() in GC_CC at trunk\working_vm\vm\gc_cc\src\gc_types.h file
> are not recognized by the Thread Checker. In order to track this
> synchronization I should mark it by imbedding Thread Checker API
> functions primitives into spin_lock()/spin_unlock() code.
>
> Here is an example of embedding Thread Checker API primitive
> "itt_notify_sync_releasing()" into spin_unlock() function :
>
>
> #ifdef ITCH  // build configuration switch
> # include <libittnotify.h>
> #else
> #define __itt_notify_sync_releasing(pAddr) //if ITCH unset, primitive do
> nothing
> #endif
>
> .......................
>
>
> void spin_unlock(volatile int* lock) {
>
>    assert(1 == *lock);
> // The releasing primitive of ITCH API
> //tell to ITCH tool that thread will release the lock
>    __itt_notify_sync_releasing(&lock);
>    *lock = 0;
> }
>
> >So it would be good if you can summarize the current status of DRLVM
> when
> > examined with Thread Checker.
>
> Here is Thread Checker results that have checked Harmony which run Hello
> World:
>
>
>
>  _____________________________________________________
> |Component                      |Number of occurrences|
> |_______________________________|_____________________|
> |jitrino                        | 25 -data-races      |
> |_______________________________|_____________________|
> |vmcore/object                  | 15 -data-races      |
> |_______________________________|_____________________|
> |tm                             | 3 -data-races       |
> |_______________________________|_____________________|
> |vmcore util                    | 8 -data-races       |
> |_______________________________|_____________________|
>
>
> For getting more special result contributors can run Thread Checker on
> intrusting DRLVM component while VM execute some tests intended for the
> component or other application.
>
> >
> > I guess the current situation as you described is, DRLVM need to be
> > instrumented to really use Thread Checker's assistance. Before that,
> > most of the reported race conditions are false-positive, hence the
> > report is not very useful.
>
> Non standard sync constructs as described in the above example result in
> false alarms if it wasn't marked. Other results are useful (see JIRA
> H-3425).
>
>
> > We need break the cyclic dependence with
> > reasonable estimation on the potential efforts of the instrumentation
> > and the potential benefits.
>
> During examining DRLVM by Thread Checker I have find place that should
> be instrumented ones. The please is spin lock synchronization constructs
> from GC_CC. So cases were instrumentation will be needed are uncommon.
>
> However if such pleases are instrumented, we will can to test if these
> synchronization constructs can provide thread safe behavior.
>
> >
> > My personal previous experience with Thread Checker was with parallel
> > application development, or application parallelization in C/C++. I am
> > not sure how Thread Checker can help a threading infrastructure
> > development, esp. when there exist dynamically generated code
> > sequence, and Java app has its own synchronization logic.
> >
>
> The purpose of Thread Checker is finding of deadlocks and race
> conditions.
> It helps to make the implementation more thread safe.
>
>
> > That being said, I think if the efforts are not huge and the
> > instrumentation doesn't impact much the code base, it would be
> > worthwhile to have a try, since we want Harmony to be rock solid in
> > threading part, any helping efforts should be appreciated, as long as
> > it does help.
> >
>
> Instrumentation cases will be rarely and will not impact much the code
> base.
> And it will increase a range of DRLVM synchronization mechanisms that
> may be examined by the Thread Checker.
>
> Thanks,
> Ilya Leviev
>
>
>
> On 3/11/07, Xiao-Feng Li <xiaofeng.li@gmail.com> wrote:
> On 3/9/07, Leviev, Ilia A <ilia.a.leviev@intel.com> wrote:
> >
> > Hi,
> > I have started using http://intel.com/thread_checker_download.zip to
> > find deadlock and race conditions in the VM.
> >
> > First I have checked Harmony which run Hello World
> > Application and tool have detected about 30 race conditions. But on
> some
> > DRLVM kernel tests it shows several hundred problems.
> >
> > There some problem of tracking Harmony code by the Thread Checker.
> > The reason for this is that the Intel Thread Checker supports
> > interpretation only of standard APIs for Windows and POSIX threading.
> > Any specially built synchronization constructs that are not part of
> > Windows
> > API or POSIX API are not normally tracked by the Thread Checker.
> > However, the Thread Checker includes the User-Level Synchronization
> API
> > that to help gather information related to user-defined
> synchronization
> > constructs.
> > Thus if we use our own synchronization we should inform Thread Checker
> > runtime about it. Otherwise the tool will generate incorrect
> diagnostics
> > and will be useless for us.
>
> Not necessarily useless, it still can be a good assistant tool. My
> previous experience with Thread Checker did help me discover some real
> hard bugs.
>
> > In other words, to gain full benefit from Thread Checker, the VM
> > developers will need to embed thread checker calls in the
> > synchronization code.  Should we put this on the harmony development
> > schedule for Q2?
>
> We need understand what are those "full benefit" DRLVM can get from
> Thread Checker. And, as a system software, I assume the threading
> infrastructure will not change dramatically once it's stable. So it
> would be good if you can summarize the current status of DRLVM when
> examined with Thread Checker.
>
> I guess the current situation as you described is, DRLVM need to be
> instrumented to really use Thread Checker's assistance. Before that,
> most of the reported race conditions are false-positive, hence the
> report is not very useful. We need break the cyclic dependence with
> reasonable estimation on the potential efforts of the instrumentation
> and the potential benefits.
>
> My personal previous experience with Thread Checker was with parallel
> application development, or application parallelization in C/C++. I am
> not sure how Thread Checker can help a threading infrastructure
> development, esp. when there exist dynamically generated code
> sequence, and Java app has its own synchronization logic.
>
> That being said, I think if the efforts are not huge and the
> instrumentation doesn't impact much the code base, it would be
> worthwhile to have a try, since we want Harmony to be rock solid in
> threading part, any helping efforts should be appreciated, as long as
> it does help.
>
> Thanks,
> xiaofeng
>
> >
> > Thanks,
> > Ilya Leviev
> > Intel Enterprise Solutions Software Division
> >
>