incubator-ooo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From J├╝rgen Schmidt <>
Subject Re: Proposal for AOO test tool
Date Fri, 17 Feb 2012 08:09:35 GMT
On 2/17/12 7:42 AM, Dennis E. Hamilton wrote:
> Ji Yan,
> I don't think the situation is that bad:
> Progression through an instance of a test can be handled by comments and managing the
state of the bugzilla issue.
> That may be too crude, and it doesn't deal with the need to automatically populate a
set of open tests for a new release.
> Some better tooling would be useful that didn't have one bugzilla issue per test case.

of course, some tolling can be of big help. And especially when the 
format is well defined. A wiki is fine but it requires also high 
discipline to keep it well formatted and aligned. Ok Mediawiki would 
allow to use tags to aggregate all test cases on one page and 
potentially organize them for releases as well with different tags.

I think we should give it a chance and we will see how it fits into our 
workflow in the future.

I am sure we will discuss some kind of strategy how we want provide 
releases in the future, how often, based on which criteria etc. At least 
for me is true that I would like to deliver high quality software that 
is enterprise ready from day one of availability. And I think this can 
be aligned with a lot of fun and collaboration in a community as well.

> However, a hierarchical management structure is unlikely here.  It is a bit contrary
to the Apache Way.  No design should depend on the existence of "formal" testers.
we all know that and I think the example of Ji Yan can be seen as an 
example how a organize release process can look like. Somebody who wants 
to help testing, can choose a test, can execute it and report the 
results. Another volunteer who feels responsible to provide an overview 
on the current status, generate some weekly report where we are. 
Everybody can and especially the release manager can use this generated 
information to get the necessary information where we are.

Our project is very huge and and everything that we can simplify by some 
tooling is good for my point of view.

> However, having a wiki-based map and template for the tests that are needed is a valuable
contribution.  Individuals can indicate in wiki tabulations what tests they are carrying out,
so that there is not unnecessary duplication (and if result reports are not forthcoming, someone
can communicate an offer to take them on instead).
> So everyone can see the open test items, the closed test items, the test results, and
the ones that have been offered to be done and are not yet reported.
> It should not be difficult to create some sort of simple summary report if that is called

why not simply using existing tools who can do that out of the box.

> This can all be done on a wiki page too, since links to attachments are supported and
it is possible to process the markdown for tables in scripts.
yes it can but it has to be done

> Creation of new test sequences against subsequent release candidates or milestone builds
should not be difficult to clone onto a new wiki page.
again yes, but it has to be done

I would definitely support the idea to test such a tool


>   - Dennis
> PS: Some philosophy for work in this kind of project:  It should be possible to accomplish
the work with minimal tooling, so anyone can participate and also support the analysis and
any coordination.  There can be additional automation.  However, a way for folks to operate
with the minimal tooling should remain possible.  It is how there can be sustained expansion
of participation and tolerance of turnover.  The on-ramp for entry-level participation has
to be minimal.  It should be possible to understand what the testing process is, and to contribute,
without having to also master specialized technology.  That's my thinking based on my observations
of Apache OpenOffice so far.  In particular, all structures that are created to facilitate
coordinated work must be designed to work with all-volunteer resources, whether or not some
or many of those willing and able to participate are being compensated for their efforts.
> -----Original Message-----
> From: Ji Yan []
> Sent: Thursday, February 16, 2012 21:28
> To:
> Subject: Re: Proposal for AOO test tool
>      Thanks for all of your response. I like your brilliant idea, but from
> QA point of view, I'm afraid it couldn't help our work.
>      A formal tester should follow test case to do test, once test is done,
> tester fills result in a document, we called it as execution record. QA
> lead will generate test report regularly based on the execution records,
> and QA manager or PM and others will know the test status. From this
> procedure we can tell test case is base element of test, and we need to
> generate test report based on execution record.
>      If we use bugzilla to record test case, where could we put result into?
> If we put test case to wiki page, where could we put result into? another
> wiki page? while, how could we generate test report? I don't think bugzilla
> and wiki page are right approach.
>      So, a powerful test management tool will help us on managing  test
> case, test execution and work procedure. So I strongly recommend using test
> tool to manage our QA work.
>       Here I bring another question, if I want to install the tool into web
> server, how to do that? Who will help me on this?
> 2012/2/14 Andre Fischer<>
>> On 13.02.2012 22:17, Rob Weir wrote:
>>> On Mon, Feb 13, 2012 at 6:55 AM, Ji Yan<>   wrote:
>>>> Hi all,
>>>>   Recently, I'm thinking about how testing work should be done and what
>>>> the
>>>> procedure should followed under Apache OO structure. Before OO goes into
>>>> ASF, testing work was controlled by QUASTe and manual test cases stored
>>>> in
>>>> TCM but both tools were disconnected once Oracle donated OO to Apache.
>>>> Now,
>>>> it's time for us to think about how can we move on for testing.
>>>>   While within AOO 3.4, we store the manual scripts in wiki page, it's
>>>> good
>>>> place at this time, but should not be permanent. As it's hard to tell
>>>> test
>>>> status and collect testing data, also it has no connection with
>>>> automation
>>>> test tool.
>>> I wonder if Bugzilla would be better than the wiki?
>> Hm, to me the wiki seems to be a better place.  I think of the manual test
>> cases as some form of documentation (about how and what to test). The wiki
>> provides better support for organizing and searching.  But, not being a QA
>> engineer, I can easily be mistaken.
>> -Andre
>>> We could create a "product" in BZ for all test cases, with
>>> "components" under that for different test areas, like "performance
>>> test", "smoke test", "detailed test", etc.
>>> One BZ issue per test case.
>>> For each test pass, we simply reset each test case/issue back to "New
>>> state".  We then test each issue.  If the test case passes, then we
>>> mark the BZ issue as closed.  If the test case fails, then we already
>>> have a BZ issue for the developers.
>>> Pro: Makes it very easy to make new test cases from existing BZ
>>> issues, or to make BZ issues from testcases.
>>> Con: Reporting not so good.   Does not handle doing multiple test
>>> passes in parallel.  For example, if we wanted to test AOO 4.0 in
>>> parallel with a maintenance AOO 3.4.1 release.
>>>    After review some tools, I find the "Test Link"[1], maybe the proper
>>>> tool
>>>> for us to manage testing work. If anyone has any suggestion on other
>>>> tools,
>>>> please let me know. The target is to customize and deploy it to OO
>>>> website. I'll move forward with this tool with no objection
>>>> [1] http://testlink.sourceforge.**net/docs/testLink.php<>
>>>> --
>>> I tried their demo site.  It was very slow.  Does anyone have
>>> experience with Test Link?
>>> -Rob
>>>> Thanks&   Best Regards, Yan Ji

View raw message