accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Master Thesis on False Positives in Test Failures
Date Fri, 08 Apr 2016 21:59:48 GMT
Hi Kevin,

Many of those test bugs and fixes were probably my doing.

Most of them were just flakiness in general, but, if you can provide an 
explicit list, I can try to confirm whether or not that was exactly the 
case.

- Josh

Kevin van den Bekerom wrote:
> Dear Developers of the Apache Accumulo project,
>
>
>
> My name is Kevin van den Bekerom and I am currently doing my Master's
> research on the topic of false alarms in test code. I would like to ask the
> input of the Accumulo development team categorizing test code bugs.
>
>
>
> My research is based on a recent paper by Arash et al. (
> http://salt.ece.ubc.ca/publications/docs/icsme15.pdf). They conducted an
> empirical study, categorizing "test code bugs" in Apache software projects,
> e.g. semantic, flaky, environmental, etc. A "test code bug" is a failing
> test, where the System Under Test is correct, but the test code is
> incorrect. To identify test code bugs they looked at issues in JIRA, and
> checked if the fixing commit was only in the test code. Only fixed issues
> were counted and categorised.
>
>
>
> My goal is to replicate their results using a different approach, i.e. ask
> developers that were involved in the issue and/or fix how they would
> categorize it.  For the Accumulo project they counted 187 test code bugs.
> Insight into false positives can therefore be very relevant for your
> project. Note that they only sampled a number of identified test code bugs
> for individual inspection (30 for the Accumulo project).
>
>
> I would like to ask the Accumulo team’s participation in categorizing the
> various test code bugs. I will provide a list of JIRA IDs which are
> identified as test code bugs and an initial list of categories to aid in
> the categorization process. In my belief, the developers that worked on the
> issue are the one's that are most capable of categorizing the issue. Please
> let me know if this project looks interesting to you and you are willing to
> help me out.
>
>
>
> As a next step I will look for common patterns in identified test code bugs
> and my aim is to extent static source code analysis techniques to be also
> suited to find test code bugs. I am of course very happy to share my
> findings with the team.
>
>
>
> Hope to hear from you!
>
>
>
> With kind regards,
>

Mime
View raw message