hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nigel Daley <nda...@yahoo-inc.com>
Subject Re: Feature Designs and Test Plans
Date Wed, 18 Feb 2009 06:58:03 GMT
[resurrecting an old thread now that I've had time to put together a  
proposal]

Sounds like folks are agreeing we need design docs and test plans for  
features, with common sense prevailing on when to apply this rule to  
smaller features.

So, let me propose some design doc and test plan templates.

Design Doc Template:
Problem Definition
what problem does the feature solve?
why is it needed?
Use Cases
sequence of simple steps on how users interacts with the feature
Interaction with Current System
does this feature build on an existing feature? if so, how?
how does the user interact with the new feature from outside the system
how does the new feature interact within the system
Requirements
focus on what functionality must exist and minimize discussion of how  
that functionality will be implemented
list items out of scope
Design
give a high level description of the implementation
algorithms and/or data-structures involved, this may be in the form of  
pseudo-code
failure analysis
Future Work
what additional work is related or could be done, but is not included  
in this design

Test Plan Template:
Feature(s) Tested: enumerate the feature(s)
which Jira issue(s)?
what is the feature?
what is the externally visible view of the feature?
Risk Scenarios: enumerate the bad things that could happen in the  
system that either:
could be caused by the feature
could have an effect on the feature
Test Cases: enumerate all tests in tables
using table headings Id, Type of Test, Description, Expected Behavior,  
Is Automated?
enumerate positive and negative test case types
include functional, system, scale, and performance test case types
indicate which tests are automated in JUnit suite

Past Design Doc Examples
http://issues.apache.org/jira/secure/attachment/12348296/DFSUpgradeProposal3.html

Past Test Plan Examples
https://issues.apache.org/jira/secure/attachment/12373559/PermissionsTestPlan2.pdf
https://issues.apache.org/jira/secure/attachment/12363605/BlockCrcFeatureTestPlan.pdf
https://issues.apache.org/jira/secure/attachment/12351299/TestPlan-HdfsUpgrade.html

Some examples requiring the above:
HADOOP-1700 (file append)
HADOOP-702 (DFS upgrades)
HADOOP-3938 (disk quotas)
HADOOP-372 (input format per dir)
HADOOP-153 (skip bad records)


Thoughts?  Is this something we can all commit to do?

Cheers,
Nige


> Hadoop Committers,
>
> The Hadoop project has grown to become a critical platform for many  
> companies.  As we approach a 1.0 release (perhaps within the next 6  
> months) I think it is time we adopt some more rigor around designing  
> and testing new features.
>
> As committers, we are already given a couple duties that relate to  
> quality.  Fromhttp://wiki.apache.org/hadoop/HowToCommit :
>
>> If a committer reviews a patch they've not authored, and believe it  
>> to be of sufficient quality, then they can commit the patch,  
>> otherwise the patch should be cancelled with a clear explanation  
>> for why it was rejected.
>>
>> ...
>>
>> Patches should be rejected which do not adhere to the guidelines inhttp://wiki.apache.org/hadoop/HowToContribute

>>  and to thehttp://wiki.apache.org/hadoop/CodeReviewChecklist.  
>> Committers should always be polite to contributors and try to  
>> instruct and encourage them to contribute better patches.
>>
> These have been a good starting point, but I'd like to propose more  
> rigor.
>
> I propose that before we commit issues marked as "New Feature", they  
> must have:
>  1. a design doc attachment
>  2. a test plan attachment
> (Templates to be provided for both)
>
> I believe these 2 items will enhance the communication, design, and  
> quality of our new features.
>
> Are committers open to adopting these further guidelines?
>
> Thanks,
> Nige

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message