Return-Path: <shemnon@apache.org>
Mailing-List: contact dev-help@ant.apache.org; run by ezmlm
Delivered-To: mailing list dev@ant.apache.org
Received: (qmail 62233 invoked by uid 500); 19 Feb 2003 05:11:36 -0000
Received: (qmail 62230 invoked from network); 19 Feb 2003 05:11:36 -0000
Received: from icarus.apache.org (208.185.179.13)
  by daedalus.apache.org with SMTP; 19 Feb 2003 05:11:36 -0000
Received: (qmail 40505 invoked by uid 1108); 19 Feb 2003 05:11:35 -0000
Date: 19 Feb 2003 05:11:34 -0000
Message-ID: <20030219051134.40504.qmail@icarus.apache.org>
From: shemnon@apache.org
To: ant-cvs@apache.org
Subject: cvs commit: ant/src/testcases/org/apache/tools/ant/taskdefs
 ParallelTest.java
X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N

shemnon     2003/02/18 21:11:34

  Modified:    docs/manual/CoreTasks parallel.html
               src/etc/testcases/taskdefs parallel.xml
               src/testcases/org/apache/tools/ant/taskdefs
                        ParallelTest.java
  Log:
  Improve reliability if ant parallel task tests
  docuemnt new attrutes to parallel task
  
  Revision  Changes    Path
  1.9       +70 -0     ant/docs/manual/CoreTasks/parallel.html
  
  Index: parallel.html
  ===================================================================
  RCS file: /home/cvs/ant/docs/manual/CoreTasks/parallel.html,v
  retrieving revision 1.8
  retrieving revision 1.9
  diff -u -r1.8 -r1.9
  --- parallel.html	4 Sep 2002 11:05:16 -0000	1.8
  +++ parallel.html	19 Feb 2003 05:11:34 -0000	1.9
  @@ -11,6 +11,31 @@
   <h3>Description</h3>
   <p>Parallel is a container task - it can contain other Ant tasks. Each nested
   task within the parallel task will be executed in its own thread. </p>
  +<h3>Parameters</h3>
  +<table border="1" cellpadding="2" cellspacing="0">
  +  <tr>
  +    <td valign="top"><b>Attribute</b></td>
  +    <td valign="top"><b>Description</b></td>
  +    <td align="center" valign="top"><b>Required</b></td>
  +  </tr>
  +  <tr>
  +    <td valign="top">threadCount</td>
  +    <td valign="top">Maximum numbers of thread to use.</td>
  +    <td align="center" valign="top">No</td>
  +  </tr>
  +  <tr>
  +    <td valign="top">threadsPerProcessor</td>
  +    <td valign="top">Maximum number of threads to use per available processor
  +(Requires JDK 1.4)</td>
  +    <td align="center" valign="top">No, defers to threadCount</td>
  +  </tr>
  +  <tr>
  +    <td valign="top">pollInterval</td>
  +    <td valign="top">Maximum number of milliseconds to wait for before checking 
  +when waiting for available threads.</td>
  +    <td align="center" valign="top">No, default is 1000</td>
  +  </tr>
  +</table>
   
   <p>Parallel tasks have a number of uses in an Ant build file including:</p>
   <ul>
  @@ -41,6 +66,26 @@
   sequential</a> task to define sequences of tasks to be executed on each thread
   within the parallel block</p>
   
  +<p>The threadCount attribute can be used to place a maximum number of available
  +threads for the execution.  When not present all child tasks will be executed at
  +once.  When present then the maximum number of concurrently executing tasks will
  +not exceed the number of threads specified.  Furthermore, each task will be
  +started in the order they are given.  But no guarantee is made as to the speed 
  +of execution or the order of completion of the tasks, only that each will be
  +started before the next.<p>
  +
  +<p>If you are using J2RE 1.4 or later you can also use the threadsPerProcessor
  +and the number of available threads will be the stated multiple of the number of
  +processors (there is no affinity to a particular processor however).  This will
  +override the value in threadCount.  If threadsPerProcessor is specified using
  +any version prior to 1.4 then the value in threadCount will be used as is.</p>
  +
  +<p>When using threadCount and threadsPerProcessor care should be taken to insure
  +that the build does not deadlock.  This can be caused by tasks such as waitFor
  +takeing up all available threads before the tasks that would unlock the waitfor
  +would occur.  This is not a repalcement for Java Language level thread
  +semantics and is best used for "embarasingly parallel" tasks.</p>
  +
   <h3>Examples</h3>
   <pre>
   &lt;parallel&gt;
  @@ -75,6 +120,31 @@
   noted above, you need to be careful that the two tasks are independent, both in 
   terms of their dependencies and in terms of their potential interactions in
   Ant's external environment.</p>
  +
  +<pre>
  +&lt;parallel threadCount='4'&gt;
  +  &lt;ant target='TargetThatConsumesLotsOfCPUTimeAndMemory'&gt;
  +    &lt;param name='file' value='one.txt'/&gt;
  +  &lt;/ant&gt;
  +  &lt;ant target='TargetThatConsumesLotsOfCPUTimeAndMemory'&gt;
  +    &lt;param name='file' value='two.txt'/&gt;
  +  &lt;/ant&gt;
  +  &lt;ant target='TargetThatConsumesLotsOfCPUTimeAndMemory'&gt;
  +    &lt;param name='file' value='three.txt'/&gt;
  +  &lt;/ant&gt;
  +  &lt;!-- repeated about 40 times --&gt;
  +&lt;/parallel&gt;
  +</pre>
  +
  +<p>This example represents a typical need for use of the threadCount and 
  +threadsPerProcessor attributes.  Spinning up all 40 of those tasks could cripple
  +the JVM for memory and the CPU for available time.  By limiting the number of
  +concurrent executions you can get the task done in about the same assuming
  +infinite memory time without needing infinite memory.  This is also a good
  +candidiate for use of threadCount (and possibly threadsPerProcessor) because
  +each task (in this hypothetical case) is independent and has no dependencies on
  +the other tasks.</p>
  +
   <hr>
   <p align="center">Copyright &copy; 2001-2002 Apache Software Foundation. All rights
   Reserved.</p>
  
  
  
  1.4       +53 -59    ant/src/etc/testcases/taskdefs/parallel.xml
  
  Index: parallel.xml
  ===================================================================
  RCS file: /home/cvs/ant/src/etc/testcases/taskdefs/parallel.xml,v
  retrieving revision 1.3
  retrieving revision 1.4
  diff -u -r1.3 -r1.4
  --- parallel.xml	17 Feb 2003 13:27:38 -0000	1.3
  +++ parallel.xml	19 Feb 2003 05:11:34 -0000	1.4
  @@ -22,103 +22,97 @@
     </target>
   
     <target name="testThreadCount">
  -    <parallel threadCount='1' pollInterval="30">
  -      <!-- expected start 1, end 1, start 2, end 2, start 3, end 3 -->
  +    <echo>|1/</echo>
  +    <parallel threadCount='1' pollInterval="60">
         <sequential>
  -        <echo message="+1"/>
  -        <sleep seconds="1"/>
  -        <echo message="-1"/>
  +        <echo message="+"/>
  +        <sleep milliseconds="30"/>
  +        <echo message="-"/>
         </sequential>
         <sequential>
  -        <echo message="+2"/>
  -        <sleep seconds="2"/>
  -        <echo message="-2"/>
  +        <echo message="+"/>
  +        <sleep milliseconds="60"/>
  +        <echo message="-"/>
         </sequential>
         <sequential>
  -        <echo message="+3"/>
  -        <sleep seconds="3"/>
  -        <echo message="-3"/>
  +        <echo message="+"/>
  +        <sleep milliseconds="90"/>
  +        <echo message="-"/>
         </sequential>
       </parallel>
  +    <echo>|2/</echo>
       <parallel threadCount='2' pollInterval="30">
  -      <!-- expected start 1, start 2, end 1, start 3, end 2, end 3 -->
         <sequential>
  -        <echo message="+1"/>
  -        <sleep seconds="1"/>
  -        <echo message="-1"/>
  +        <echo message="+"/>
  +        <sleep milliseconds="30"/>
  +        <echo message="-"/>
         </sequential>
         <sequential>
  -        <sleep milliseconds="200"/>
  -        <echo message="+2"/>
  -        <sleep seconds="2"/>
  -        <echo message="-2"/>
  +        <echo message="+"/>
  +        <sleep milliseconds="60"/>
  +        <echo message="-"/>
         </sequential>
         <sequential>
  -        <sleep milliseconds="300"/>
  -        <echo message="+3"/>
  -        <sleep seconds="3"/>
  -        <echo message="-3"/>
  +        <echo message="+"/>
  +        <sleep milliseconds="90"/>
  +        <echo message="-"/>
         </sequential>
       </parallel>
  +    <echo>|3/</echo>
       <parallel threadCount='3' pollInterval="30">
  -      <!-- expected start 1, start 2, start 3, end 1, end 2, end 3 -->
         <sequential>
  -        <echo message="+1"/>
  -        <sleep seconds="1"/>
  -        <echo message="-1"/>
  +        <echo message="+"/>
  +        <sleep milliseconds="30"/>
  +        <echo message="-"/>
         </sequential>
         <sequential>
  -        <sleep milliseconds="200"/>
  -        <echo message="+2"/>
  -        <sleep seconds="2"/>
  -        <echo message="-2"/>
  +        <echo message="+"/>
  +        <sleep milliseconds="60"/>
  +        <echo message="-"/>
         </sequential>
         <sequential>
  -        <sleep milliseconds="300"/>
  -        <echo message="+3"/>
  -        <sleep seconds="3"/>
  -        <echo message="-3"/>
  +        <echo message="+"/>
  +        <sleep milliseconds="90"/>
  +        <echo message="-"/>
         </sequential>
       </parallel>
  +    <echo>|4/</echo>
       <parallel threadCount='4' pollInterval="30">
  -      <!-- expected start 1, start 2, start 3, end 1, end 2, end 3 -->
         <sequential>
  -        <echo message="+1"/>
  -        <sleep seconds="1"/>
  -        <echo message="-1"/>
  +        <echo message="+"/>
  +        <sleep milliseconds="30"/>
  +        <echo message="-"/>
         </sequential>
         <sequential>
  -        <sleep milliseconds="200"/>
  -        <echo message="+2"/>
  -        <sleep seconds="2"/>
  -        <echo message="-2"/>
  +        <echo message="+"/>
  +        <sleep milliseconds="60"/>
  +        <echo message="-"/>
         </sequential>
         <sequential>
  -        <sleep milliseconds="300"/>
  -        <echo message="+3"/>
  -        <sleep seconds="3"/>
  -        <echo message="-3"/>
  +        <echo message="+"/>
  +        <sleep milliseconds="90"/>
  +        <echo message="-"/>
         </sequential>
       </parallel>
  -    <parallel threadsPerProcessor='1' pollInterval="30">
  -      <!-- expected result varies, depends on setup -->
  -      <!-- this is a smoke test for threadsPerProcessor -->
  +    <echo>|4/</echo>
  +    <parallel threadsPerProcessor='1' threadcount='4' pollInterval="30">
         <sequential>
  -        <!--echo message="+1"/-->
  -        <sleep seconds="1"/>
  -        <!--echo message="-1"/-->
  +        <echo message="+"/>
  +        <sleep milliseconds="30"/>
  +        <echo message="-"/>
         </sequential>
         <sequential>
  -        <!--echo message="+2"/-->
  -        <sleep seconds="2"/>
  -        <!--echo message="-2"/-->
  +        <echo message="+"/>
  +        <sleep milliseconds="60"/>
  +        <echo message="-"/>
         </sequential>
         <sequential>
  -        <!--echo message="+3"/-->
  -        <sleep seconds="3"/>
  -        <!--echo message="-3"/-->
  +        <echo message="+"/>
  +        <sleep milliseconds="90"/>
  +        <echo message="-"/>
         </sequential>
       </parallel>
  +    <echo>|</echo>
       
     </target>
   
  
  
  
  1.6       +50 -4     ant/src/testcases/org/apache/tools/ant/taskdefs/ParallelTest.java
  
  Index: ParallelTest.java
  ===================================================================
  RCS file: /home/cvs/ant/src/testcases/org/apache/tools/ant/taskdefs/ParallelTest.java,v
  retrieving revision 1.5
  retrieving revision 1.6
  diff -u -r1.5 -r1.6
  --- ParallelTest.java	11 Feb 2003 11:57:27 -0000	1.5
  +++ ParallelTest.java	19 Feb 2003 05:11:34 -0000	1.6
  @@ -53,10 +53,13 @@
    */
   package org.apache.tools.ant.taskdefs;
   import java.io.PrintStream;
  +import junit.framework.AssertionFailedError;
  +import org.apache.tools.ant.BuildException;
   
   import org.apache.tools.ant.BuildFileTest;
   import org.apache.tools.ant.DemuxOutputStream;
   import org.apache.tools.ant.Project;
  +import org.apache.tools.ant.Task;
   
   /**
    * Test of the parallel TaskContainer
  @@ -104,17 +107,59 @@
       }
   
       /** tests basic operation of the parallel task */
  -    public void testTreadCount() {
  +    public void testThreadCount() {
           // should get no output at all
           Project project = getProject();
           project.setUserProperty("test.direct", DIRECT_MESSAGE);
           project.setUserProperty("test.delayed", DELAYED_MESSAGE);
           expectOutputAndError("testThreadCount", "", "");
           String log = getLog();
  -        assertEquals("parallel tasks did't block on threads properly", log,
  -            "+1-1+2-2+3-3+1+2-1+3-2-3+1+2+3-1-2-3+1+2+3-1-2-3");
  -
  +        int pos = 0;
  +        while (pos > -1) {
  +            pos = countThreads(log, pos);
  +        }
  +    }
  +    
  +    /**
  +     * the test result string should match the regex 
  +     * <code>^(\|\d+\/(+-)*)+\|$</code> for someting like 
  +     * <code>|3/++--+-|5/+++++-----|</code>
  +     *
  +     *@returns -1 no more tests
  +     *          # start pos of next test
  +     *@throws AssertionFailedException when a constraint is invalid
  +     */
  +    static int countThreads(String s, int start) {
  +        int firstPipe = s.indexOf('|', start);
  +        int beginSlash = s.indexOf('/', firstPipe);
  +        int lastPipe = s.indexOf('|', beginSlash);
  +        if ((firstPipe == -1) || (beginSlash == -1) || (lastPipe == -1)) {
  +            return -1;
  +        }
  +        
  +        int max = Integer.parseInt(s.substring(firstPipe + 1, beginSlash));
  +        int current = 0;
  +        int pos = beginSlash + 1;
  +        while (pos < lastPipe) {
  +            switch (s.charAt(pos++)) {
  +                case '+': 
  +                    current++;
  +                    break;
  +                case '-':
  +                    current--;
  +                    break;
  +                default:
  +                    throw new AssertionFailedError("Only expect '+-' in result count, found " 
  +                        + s.charAt(--pos) + " at position " + pos);
  +            }
  +            if (current > max) {
  +                throw new AssertionFailedError("Number of executing threads exceeded number allowed: " 
  +                    + current + " > " + max);
  +            }
  +        }
  +        return lastPipe;
       }
  +    
   
       /** tests the failure of a task within a parallel construction */
       public void testFail() {
  @@ -142,5 +187,6 @@
               System.setErr(err);
           }
       }
  +    
   }