incubator-ooo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dennis E. Hamilton" <dennis.hamil...@acm.org>
Subject RE: Proposal for AOO test tool
Date Fri, 17 Feb 2012 06:42:32 GMT
Ji Yan,

I don't think the situation is that bad:

Progression through an instance of a test can be handled by comments and managing the state
of the bugzilla issue.

That may be too crude, and it doesn't deal with the need to automatically populate a set of
open tests for a new release.  

Some better tooling would be useful that didn't have one bugzilla issue per test case.

However, a hierarchical management structure is unlikely here.  It is a bit contrary to the
Apache Way.  No design should depend on the existence of "formal" testers.

However, having a wiki-based map and template for the tests that are needed is a valuable
contribution.  Individuals can indicate in wiki tabulations what tests they are carrying out,
so that there is not unnecessary duplication (and if result reports are not forthcoming, someone
can communicate an offer to take them on instead).

So everyone can see the open test items, the closed test items, the test results, and the
ones that have been offered to be done and are not yet reported.  

It should not be difficult to create some sort of simple summary report if that is called
for.  

This can all be done on a wiki page too, since links to attachments are supported and it is
possible to process the markdown for tables in scripts.

Creation of new test sequences against subsequent release candidates or milestone builds should
not be difficult to clone onto a new wiki page.

 - Dennis

PS: Some philosophy for work in this kind of project:  It should be possible to accomplish
the work with minimal tooling, so anyone can participate and also support the analysis and
any coordination.  There can be additional automation.  However, a way for folks to operate
with the minimal tooling should remain possible.  It is how there can be sustained expansion
of participation and tolerance of turnover.  The on-ramp for entry-level participation has
to be minimal.  It should be possible to understand what the testing process is, and to contribute,
without having to also master specialized technology.  That's my thinking based on my observations
of Apache OpenOffice so far.  In particular, all structures that are created to facilitate
coordinated work must be designed to work with all-volunteer resources, whether or not some
or many of those willing and able to participate are being compensated for their efforts.

-----Original Message-----
From: Ji Yan [mailto:yanji.yj@gmail.com] 
Sent: Thursday, February 16, 2012 21:28
To: ooo-dev@incubator.apache.org
Subject: Re: Proposal for AOO test tool

    Thanks for all of your response. I like your brilliant idea, but from
QA point of view, I'm afraid it couldn't help our work.
    A formal tester should follow test case to do test, once test is done,
tester fills result in a document, we called it as execution record. QA
lead will generate test report regularly based on the execution records,
and QA manager or PM and others will know the test status. From this
procedure we can tell test case is base element of test, and we need to
generate test report based on execution record.
    If we use bugzilla to record test case, where could we put result into?
If we put test case to wiki page, where could we put result into? another
wiki page? while, how could we generate test report? I don't think bugzilla
and wiki page are right approach.
    So, a powerful test management tool will help us on managing  test
case, test execution and work procedure. So I strongly recommend using test
tool to manage our QA work.

     Here I bring another question, if I want to install the tool into web
server, how to do that? Who will help me on this?


2012/2/14 Andre Fischer <af@a-w-f.de>

> On 13.02.2012 22:17, Rob Weir wrote:
>
>> On Mon, Feb 13, 2012 at 6:55 AM, Ji Yan<yanji.yj@gmail.com>  wrote:
>>
>>> Hi all,
>>>
>>>  Recently, I'm thinking about how testing work should be done and what
>>> the
>>> procedure should followed under Apache OO structure. Before OO goes into
>>> ASF, testing work was controlled by QUASTe and manual test cases stored
>>> in
>>> TCM but both tools were disconnected once Oracle donated OO to Apache.
>>> Now,
>>> it's time for us to think about how can we move on for testing.
>>>  While within AOO 3.4, we store the manual scripts in wiki page, it's
>>> good
>>> place at this time, but should not be permanent. As it's hard to tell
>>> test
>>> status and collect testing data, also it has no connection with
>>> automation
>>> test tool.
>>>
>>
>> I wonder if Bugzilla would be better than the wiki?
>>
>
> Hm, to me the wiki seems to be a better place.  I think of the manual test
> cases as some form of documentation (about how and what to test). The wiki
> provides better support for organizing and searching.  But, not being a QA
> engineer, I can easily be mistaken.
>
> -Andre
>
>
>
>> We could create a "product" in BZ for all test cases, with
>> "components" under that for different test areas, like "performance
>> test", "smoke test", "detailed test", etc.
>>
>> One BZ issue per test case.
>>
>> For each test pass, we simply reset each test case/issue back to "New
>> state".  We then test each issue.  If the test case passes, then we
>> mark the BZ issue as closed.  If the test case fails, then we already
>> have a BZ issue for the developers.
>>
>> Pro: Makes it very easy to make new test cases from existing BZ
>> issues, or to make BZ issues from testcases.
>>
>> Con: Reporting not so good.   Does not handle doing multiple test
>> passes in parallel.  For example, if we wanted to test AOO 4.0 in
>> parallel with a maintenance AOO 3.4.1 release.
>>
>>
>>   After review some tools, I find the "Test Link"[1], maybe the proper
>>> tool
>>> for us to manage testing work. If anyone has any suggestion on other
>>> tools,
>>> please let me know. The target is to customize and deploy it to OO
>>> website. I'll move forward with this tool with no objection
>>>
>>> [1] http://testlink.sourceforge.**net/docs/testLink.php<http://testlink.sourceforge.net/docs/testLink.php>
>>> --
>>>
>>
>> I tried their demo site.  It was very slow.  Does anyone have
>> experience with Test Link?
>>
>> -Rob
>>
>>
>>> Thanks&  Best Regards, Yan Ji
>>>
>>


-- 

Thanks & Best Regards, Yan Ji


Mime
View raw message