Return-Path: 
 <dev-return-25689-apmail-harmony-dev-archive=harmony.apache.org@harmony.apache.org>
Delivered-To: apmail-harmony-dev-archive@www.apache.org
Received: (qmail 41220 invoked from network); 5 Apr 2007 06:16:18 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2)
  by minotaur.apache.org with SMTP; 5 Apr 2007 06:16:18 -0000
Received: (qmail 75551 invoked by uid 500); 5 Apr 2007 06:16:24 -0000
Delivered-To: apmail-harmony-dev-archive@harmony.apache.org
Received: (qmail 75512 invoked by uid 500); 5 Apr 2007 06:16:23 -0000
Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:dev-help@harmony.apache.org>
List-Unsubscribe: <mailto:dev-unsubscribe@harmony.apache.org>
List-Post: <mailto:dev@harmony.apache.org>
List-Id: <dev.harmony.apache.org>
Reply-To: dev@harmony.apache.org
Delivered-To: mailing list dev@harmony.apache.org
Received: (qmail 75479 invoked by uid 99); 5 Apr 2007 06:16:23 -0000
Received: from herse.apache.org (HELO herse.apache.org) (140.211.11.133)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Apr 2007 23:16:23 -0700
X-ASF-Spam-Status: No, hits=-0.0 required=10.0
	tests=SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (herse.apache.org: domain of alexey.a.petrenko@gmail.com
 designates 66.249.82.233 as permitted sender)
Received: from [66.249.82.233] (HELO wx-out-0506.google.com) (66.249.82.233)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Apr 2007 23:16:15 -0700
Received: by wx-out-0506.google.com with SMTP id i26so505847wxd
        for <dev@harmony.apache.org>; Wed, 04 Apr 2007 23:15:54 -0700 (PDT)
DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed;
        d=gmail.com; s=beta;
        h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
        b=c9gb5YkZ6EJt1Hh9Lx1guG07eTsA6SeVZNv7yXGhOYtfXEHU+BfWWMAjGyHPj2MKa+Svs7wjTpXTjhPI5NfCUR0+JL6nlOQxHsSf5AvYMJwOGn0qw0VZuF341whK/QMteXXc6Y+hQ4as5HB/eX9zPGlXgxWUnCIUaBvmigkaTTc=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
        b=qnLmIhLp/fDyho1HNr8KCrj0xBtNLC1AXh3qqpusGyIUhreixlxcZN6QCoRyGAKRkknAniio1k+BMVSuCg94i7g3RCOfzudRdjdU8Q36pplQ7NpM/XRseY5s8eMbZXqLlCDIdVRs5Vfoup1fTLZoSRUPSOk5nJhfS+54G9XacrY=
Received: by 10.114.194.1 with SMTP id r1mr595092waf.1175753754111;
        Wed, 04 Apr 2007 23:15:54 -0700 (PDT)
Received: by 10.114.120.5 with HTTP; Wed, 4 Apr 2007 23:15:54 -0700 (PDT)
Message-ID: <c3755b3a0704042315m716ffd04udb04a270b10841e0@mail.gmail.com>
Date: Thu, 5 Apr 2007 10:15:54 +0400
From: "Alexey Petrenko" <alexey.a.petrenko@gmail.com>
To: dev@harmony.apache.org
Subject: Re: [general] Discussion: how to keep up stability and fast progress
 all together?
In-Reply-To: <f944d83d0704042246w3dfd93f6tc2e4506dcd9520c1@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <6e47b64f0703300131j291450c8if09efc59914c9076@mail.gmail.com>
	 <460D1B1F.5000006@gmail.com>
	 <6e47b64f0704020210j3b5ac8c9rb81d5871381dd4f@mail.gmail.com>
	 <7273946b0704022036l4057dc02gbeed4697e4970e9d@mail.gmail.com>
	 <6e47b64f0704030133u560ba37ck89f99106f5924db2@mail.gmail.com>
	 <7273946b0704040348p47046a72w3327a009b257d18c@mail.gmail.com>
	 <906dd82e0704042035h422b7623t7a3ff365cb5fae8e@mail.gmail.com>
	 <f944d83d0704042246w3dfd93f6tc2e4506dcd9520c1@mail.gmail.com>
X-Virus-Checked: Checked by ClamAV on apache.org

2007/4/5, Alexey Varlamov <alexey.v.varlamov@gmail.com>:
> 2007/4/5, Mikhail Loenko <mloenko@gmail.com>:
> > 2007/4/4, Vladimir Ivanov <ivavladimir@gmail.com>:
> > > On 4/3/07, Stepan Mishura <stepan.mishura@gmail.com> wrote:
> > > > On 4/3/07, Vladimir Ivanov wrote:
> > > > <SNIP>
> > > > >
> > > > > > I'd like to propose the next approach that may help us to know about
> > > > > > instabilities: develop (or take existing one, for example, Eclipse
> > > > > > hello world) a scenario for testing stability and configure CC to run
> > > > > > it at all times. The stability scenario must be the only one scenario
> > > > > > for CC; it must be short (no longer then an hour), test JRE in stress
> > > > > > conditions and cover most of functionality. If the scenario fails then
> > > > > > all newly committed updates are subject for investigation and fix (or
> > > > > > rollback).
> > > > >
> > > > > Actually, I prefer something without GUI or at least without using
> > > > > special 'GUI testing" tools. It should improve quality of this testing
> > > > > (than less tools than more predictable results :)) Current "Eclipse
> > > > > hello world" scenario based on the AutoIT for Win and X11GuiTest for
> > > > > Linux platform. Also we have this scenario based on API calls which
> > > > > should emulate GUI scenario. From these 2 approaches I prefer second
> > > > > to minimize 'false alarms'. Or may be some other scenarios (non-GUI)?
> > > > >
> > > >
> > >
> > > > Did I understand you correctly that there may be 'false alarms' caused
> > > > by using external 'GUI testing' tools? If yes which kind of 'false
> > > > alarms' are there?
> > >
> > > Seems, in some cases the input from the X11GuiTest can be lost in the system :(
> > > Also timeouts between different symbols may depends on the system load etc
> >
> > How often does EHWA fail? Given that this scenario exercises VM very well
> > and the scenario itself is pretty fast we may run it twice and report
> > a problem if
> > it fails both attempts.
>
> As Vladimir already mentioned, drlvm build provides automated EHWA
> test driven via Eclipse APIs, it is reliable and very fast - why not
> use it instead of GUI robots, indeed?
But it would be nice to have some non-Eclipse GUI testing since
Eclipse does not use awt and swing.

SY, Alexey