Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Received-SPF: error (athena.apache.org: local policy)
Date: Tue, 23 Jul 2013 11:21:01 +0530
From: Prasanna Santhanam <tsp@apache.org>
To: CloudStack Dev <dev@cloudstack.apache.org>,
	Girish Shilamkar <girish@clogeny.com>
Subject: Re: Regression failures
Message-ID: <20130723055101.GB3744@cloud-2.local>
Mail-Followup-To: CloudStack Dev <dev@cloudstack.apache.org>,
	Girish Shilamkar <girish@clogeny.com>
References: <20130610023334.GA677@cloud-2.local>
 <20130717110426.GA16683@cloud-2.local>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20130717110426.GA16683@cloud-2.local>
User-Agent: Mutt/1.5.21 (2010-09-15)

We've been working regularly on fixing the regression tests trying to
get it into good shape for 4.2. You will see bugs being filed from
regular test runs on citrix internal environments.

The following filter can be used in JIRA if you would like to help in
fixing the bugs and/or review fixed bugs:

"""
project = CloudStack and (status = Open or status = Reopened or status = "Ready To Review")  and summary ~ Automation order by key desc
"""

Please assign the bug to yourself if you begin effort on its fix so
effort is not duplicated.

Thanks,


On Wed, Jul 17, 2013 at 04:34:26PM +0530, Prasanna Santhanam wrote:
> Reviving this effort starting today:
> 
> I'm going to switch the test infra to run regression tests only for
> the next couple of weeks. So far the BVTs have been running and we
> have good success rate on them. It's time to start fixing the
> regression tests as well since that constitutes the bulk of cloudstack
> features. 
> 
> The pace has been quite slow because these run only once a week and we
> end up waiting a whole week to see the fixes we make.
> 
> People who would like to contribute to this effort can join on the IRC
> meeting today for any issues/pointers they will need to fix the
> failures. The test runs are available on jenkins.bacd.org and the report
> is available along with logs from here:
> 
> http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-regression-matrix/
> 
> Thanks,
> 
> On Mon, Jun 10, 2013 at 08:03:34AM +0530, Prasanna Santhanam wrote:
> > The regression failure rate has been quite high and the failures are
> > mostly related to test scripts. Let's please focus with high priority
> > on fixing as many as we can by the next run ie Wednesday. I would like
> > to see the failure rates reduce by the 4.2 release cut on 6/28
> > 
> > http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-regression-matrix/23/testReport/?
> > 
> > Our BVTs are more or less well sorted now. Rayees and Talluri will be
> > focussing if there are any remaining failures and reporting bugs and
> > applying patches where they can.
> > 
> > Girish and I will be working on the regression failures. There are too
> > many of them so I won't be filing bugs. Instead, please watch the
> > jenkins report and pick the fixes you intend to make. If you intend to
> > work on a fix, please file a bug report and work on the patch so we
> > don't have overlaps.
> > 
> > After that we need to focus on reducing the lag in the runs which are
> > at about 7hr now. The more tests we add, the slower this gets. We
> > might think about consolidating all our infrastructure to accomadate
> > runs on all hypervisors.
> > 
> > Thanks,
> > 
> > -- 
> > Prasanna.,
> > 
> > ------------------------
> > Powered by BigRock.com
> 
> -- 
> Prasanna.,
> 
> ------------------------
> Powered by BigRock.com

-- 
Prasanna.,

------------------------
Powered by BigRock.com