Return-Path: Delivered-To: apmail-harmony-dev-archive@www.apache.org Received: (qmail 46339 invoked from network); 6 Jun 2007 09:49:56 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 6 Jun 2007 09:49:56 -0000 Received: (qmail 70172 invoked by uid 500); 6 Jun 2007 09:49:58 -0000 Delivered-To: apmail-harmony-dev-archive@harmony.apache.org Received: (qmail 70145 invoked by uid 500); 6 Jun 2007 09:49:58 -0000 Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@harmony.apache.org Delivered-To: mailing list dev@harmony.apache.org Received: (qmail 70136 invoked by uid 99); 6 Jun 2007 09:49:58 -0000 Received: from herse.apache.org (HELO herse.apache.org) (140.211.11.133) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Jun 2007 02:49:58 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (herse.apache.org: domain of oleg.oleinik@gmail.com designates 66.249.92.170 as permitted sender) Received: from [66.249.92.170] (HELO ug-out-1314.google.com) (66.249.92.170) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Jun 2007 02:49:54 -0700 Received: by ug-out-1314.google.com with SMTP id s2so393280uge for ; Wed, 06 Jun 2007 02:49:32 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=kczLl6v+TuzABEo44vjq1KkmkLQFzyf3YQh0ITOmeWTU7xwGsaS/5T95G80puAjSF2XVzooTeYaAZGEWoiF75GHu7qHPYraEZeY/6bpXM16dxhqhaH9gSqgy0lc+lPY+5keNF9lI19AOlur/VeVbQnoy91dU5YB4XZWO2yTjb8I= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=cNPlHqJrFvXV8hRaXx4IIGS1qwW58Zm1kKd34FfqVJ4whjugWIt/a9PNt8XjoaLdvr1TZrwE27fQ58wdflEsOtJivRAD7cjNFLZRZJKX+1HJ/PXdjxuMpDAvZPTVVqCYdZaIl+kWoT5SOLwbz5Cod7jHci6QpRpIrKZBFcUxzWs= Received: by 10.82.126.5 with SMTP id y5mr592652buc.1181123372426; Wed, 06 Jun 2007 02:49:32 -0700 (PDT) Received: by 10.82.141.10 with HTTP; Wed, 6 Jun 2007 02:49:32 -0700 (PDT) Message-ID: Date: Wed, 6 Jun 2007 16:49:32 +0700 From: "Oleg Oleinik" To: dev@harmony.apache.org Subject: Re: [general] Harmony M2 schedule In-Reply-To: <0vqwsyhvaqd.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <906dd82e0706040123n6f821119y8a4f289ee0255aab@mail.gmail.com> <4dd1f3f00706041917s20b38cd4k627e2402e2856b60@mail.gmail.com> <906dd82e0706042055i1bdbe501pab06ac39917b2f0c@mail.gmail.com> <0vqk5ujf0rb.fsf@gmail.com> <2c9597b90706050036t433413c8kf0caf82050b0e5@mail.gmail.com> <0vqfy56fpk5.fsf@gmail.com> <51d555c70706050853j227a187bt559c82749349fad3@mail.gmail.com> <906dd82e0706052154o77311488h631d3aa2253c373@mail.gmail.com> <469bff730706052334m631082fj56193d3f855692ad@mail.gmail.com> <0vqwsyhvaqd.fsf@gmail.com> X-Virus-Checked: Checked by ClamAV on apache.org > > Thanks for capturing the requirements. IMO the requirements still lack > > good tests to ensure stability and reliability. I propose to include > > the following recently accepted tests: > > > > 1. VM validation test suite - http://issues.apache.org/jira/browse/HARMONY-3206 > > 2. Functional test suite - http://issues.apache.org/jira/browse/HARMONY-3528 > > 3. Stress tests - http://issues.apache.org/jira/browse/HARMONY-3536 > > 4. Reliability test - https://issues.apache.org/jira/browse/HARMONY-2918 > > > > Olegs seems proposed a reasonable target for the tests- 98% pass rate > > of all valid tests from these test suites. > > what is the % today? where can I find these numbers to make sure 98% is real? Here are some approximate numbers I have (without details with some probability of mistake) to estimate how far we are from 98% pass rates: Reliability test suite: valid 83, pass 59 (0,5 hour per test), fail 24, rate: 71% Functional: valid 1328, pass 1254, fail 74, rate 94% Stress: valid 376, pass 245, fail 131, rate 65% VM validation: valid 4346, pass 4263, fail 83, rate 98% TPTP profiler tests: valid 174, pass 174, rate 100% Geronimo unit tests: valid 616, pass 603, fail 13, rate 97,8% Eclipse unit tests (wrong test suites patched): valid 36642, pass 35598, fail 1044, rate 97% Harmony class library tests: total ~24000, fail ~300, rate 98,8%