Mailing-List: contact dev-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@accumulo.apache.org
Message-ID: <570829D4.1040908@gmail.com>
Date: Fri, 08 Apr 2016 17:59:48 -0400
From: Josh Elser <josh.elser@gmail.com>
User-Agent: Postbox 3.0.11 (Macintosh/20140602)
MIME-Version: 1.0
To: dev@accumulo.apache.org
Subject: Re: Master Thesis on False Positives in Test Failures
References: 
 <CAN53w=Ecm-YiEoRZwcdYHVnPc1=DeC1T2+sy9T4Hi1w=9vihhA@mail.gmail.com>
In-Reply-To: 
 <CAN53w=Ecm-YiEoRZwcdYHVnPc1=DeC1T2+sy9T4Hi1w=9vihhA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Kevin,

Many of those test bugs and fixes were probably my doing.

Most of them were just flakiness in general, but, if you can provide an 
explicit list, I can try to confirm whether or not that was exactly the 
case.

- Josh

Kevin van den Bekerom wrote:
> Dear Developers of the Apache Accumulo project,
>
>
>
> My name is Kevin van den Bekerom and I am currently doing my Master's
> research on the topic of false alarms in test code. I would like to ask the
> input of the Accumulo development team categorizing test code bugs.
>
>
>
> My research is based on a recent paper by Arash et al. (
> http://salt.ece.ubc.ca/publications/docs/icsme15.pdf). They conducted an
> empirical study, categorizing "test code bugs" in Apache software projects,
> e.g. semantic, flaky, environmental, etc. A "test code bug" is a failing
> test, where the System Under Test is correct, but the test code is
> incorrect. To identify test code bugs they looked at issues in JIRA, and
> checked if the fixing commit was only in the test code. Only fixed issues
> were counted and categorised.
>
>
>
> My goal is to replicate their results using a different approach, i.e. ask
> developers that were involved in the issue and/or fix how they would
> categorize it.  For the Accumulo project they counted 187 test code bugs.
> Insight into false positives can therefore be very relevant for your
> project. Note that they only sampled a number of identified test code bugs
> for individual inspection (30 for the Accumulo project).
>
>
> I would like to ask the Accumulo team’s participation in categorizing the
> various test code bugs. I will provide a list of JIRA IDs which are
> identified as test code bugs and an initial list of categories to aid in
> the categorization process. In my belief, the developers that worked on the
> issue are the one's that are most capable of categorizing the issue. Please
> let me know if this project looks interesting to you and you are willing to
> help me out.
>
>
>
> As a next step I will look for common patterns in identified test code bugs
> and my aim is to extent static source code analysis techniques to be also
> suited to find test code bugs. I am of course very happy to share my
> findings with the team.
>
>
>
> Hope to hear from you!
>
>
>
> With kind regards,
>