Mailing-List: contact dev-help@forrest.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@forrest.apache.org
Received-SPF: neutral (asf.osuosl.org: local policy)
Message-ID: <4387FBB6.2070301@apache.org>
Date: Sat, 26 Nov 2005 06:07:50 +0000
From: Ross Gardler <rgardler@apache.org>
User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206)
MIME-Version: 1.0
To: dev@forrest.apache.org
Subject: Re: deployed plugins with wrong version number
References: <20051112023828.GA7202@igg.indexgeo.com.au>
 <20051112040156.GA7576@igg.indexgeo.com.au>
 <20051112065348.GA8479@igg.indexgeo.com.au> <4375D0A0.7040502@apache.org>
 <20051113084052.GA9667@igg.indexgeo.com.au> <437710F3.2030509@apache.org>
 <20051114102321.GA12951@igg.indexgeo.com.au> <4378735F.8070108@apache.org>
 <20051125064839.GA5260@igg.indexgeo.com.au> <4386EC77.5040507@apache.org>
 <20051126011510.GB6703@igg.indexgeo.com.au>
In-Reply-To: <20051126011510.GB6703@igg.indexgeo.com.au>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

David Crossley wrote:
> Ross Gardler wrote:
> 
>>David Crossley wrote:
>>
>>>Ross Gardler wrote:
>>>
>>>>David Crossley wrote:
>>>>
>>>>>Ross Gardler wrote:
>>>>>
>>>>>>David Crossley wrote:
>>>>>>
>>>>>>
>>>>>>>I wonder if it is because there in no "unversioned"
>>>>>>>plugin in the 0.7 directory. At 0.7 release time
>>>>>>>perhaps we needed to copy the unversioned ones there too.
>>>>>>
>>>>>>Yes, that will be it, I'll look into this ASAP. Thanks.
>>>>>
>>>>>I tried this by:
>>>>>cd forrest-site/plugins/0.7/
>>>>>svn cp org.apache.forrest.plugin.input.projectInfo-0.1.zip 
>>>>>org.apache.forrest.plugin.input.projectInfo.zip
>>>>>svn commit
>>>>>... then waited for the hourly cronjob to 'svn update' our site.
>>>>>
>>>>>Then tried again with forrest_07 and same problem that was
>>>>>reported above.
>>>>>
>>>>>I noticed that it is trying to retrieve:
>>>>>http://forrest.apache.org/plugins//org.apache.forrest.plugin.input.projectInfo.zip
>>>>>... perhaps needs /0.7/ in that empty space.
>>>>
>>>>Thanks David. I'll have to dig into the build files to see what is 
>>>>happening. Probaboly won't have the time until the end of the week at 
>>>>the earliest.
>>>
>>>I did this today and it now works for me with forrest_07_branch.
>>>
>>>---------------
>>>[main]$ svn diff var/pluginlist2fetch.xsl 
>>>Index: var/pluginlist2fetch.xsl
>>>===================================================================
>>>--- var/pluginlist2fetch.xsl    (revision 329512)
>>>+++ var/pluginlist2fetch.xsl    (working copy)
>>>@@ -46,7 +46,7 @@
>>>         <target name="fetch-unversioned-plugin" 
>>>         unless="versioned-plugin.present">
>>>            <echo>Versioned plugin unavailable, trying to get versionless 
>>>            plugin...</echo>
>>>            <get verbose="true" usetimestamp="true" ignoreerrors="true">
>>>-               <xsl:attribute name="src"><xsl:value-of 
>>>select="plugin[@name=$plugin-name]/@url" />/<xsl:value-of 
>>>select="$plugin-name" />.zip</xsl:attribute>
>>>+               <xsl:attribute name="src"><xsl:value-of 
>>>select="plugin[@name=$plugin-name]/@url" />/<xsl:value-of 
>>>select="$forrest-version" />/<xsl:value-of select="$plugin-name" 
>>>/>.zip</xsl:attribute>
>>>---------------
>>>
>>>So our plugin deployment process needs to put a copy
>>>of the plugin into the specific version directory, e.g.
>>>in plugins/0.8-dev/ and we need to manually add them to
>>>the plugins/0.7/ directory.
>>
>>Thanks for looking at this David, as you guessed I have not had the time 
>>yet. Something does not ring true for me with this patch.
> 
> 
> It is only part of the solution. The rest is the changes
> to the deployment process that we arrived at below.
> See above, i also temporarily placed an unversioned copy
> of the projectInfo plugin into plugins/0.7/ but it doesn't
> yet have copies of the rest.
> 
> 
>>I don't have 
>>the time to check the main plugin build.xml right now, but from memory:
>>
>>What happens when a plugin is deployed is that a versioned copy is put 
>>into the a specific Forrest directory (i.e. 0.8) and an unversioned copy 
>>is put into the root distirbution directory (I *think* that I prevented 
>>a versioned copy with a "-dev" version number going into the versioned 
>>Forrest directory).
>>
>>This means that a versioned copy is in the correct versioned forrest 
>>directory (identified in the plugins own build.xml). Remember that a 
>>plugin is intended to work with a minimum version of Forrest. The 
>>unversioned copy of the plugin was plaved in the root unversioned 
>>directory in order to minimise the number of copies that we need on the 
>>server.
> 
> 
> So are we saying below at [A] that we won't worry about
> duplication? I am fine with that.

Yes

>>The assumption was that people using development versions of 
>>plugins will be using development versions of Forrest.
>>
>>This is not a good assumption.
> 
> 
> [A]
> 
> 
>>>Perhaps we then don't need the unversioned plugins at
>>>the plugins/ directory, rather only have them in the
>>>specific sub-directory.
>>
>>Yes, and I think the version resolving gode needs to be refined as well. 
>>Currently, if a plugin works in 0.7 and 0.8-dev I think we need a copy 
>>of the plugin in both subdirectories as the plugin resolving code does 
>>not check other directories. What we need is for the loading code to 
>>look first in the correct Forrest version directory, then in the 
>>previous version and so on until it finds the right version of the plugin.
>>
>>We can hack this solutoin together in the ANT scripts fairly easily 
>>(time allowing ;-)

Done. Very easy once my son gets me up in the middles of the night (poor 
blighter has a horrible cold)

WARNING
I have only tested this on one plugin (I'm going back to bed now, Zeph 
is sound asleep again):

Added:
 
forrest/site/plugins/0.8-dev/org.apache.forrest.plugin.input.projectInfo.zip 
   (with props)

> I made an Issue to remind us.
> http://issues.apache.org/jira/browse/FOR-742
> 
> On a related issue, what happens at release time?
> I suppose that we just do an 'svn copy' of 0.8-dev
> directory to 0.8 directory and create a 0.9-dev dir.

That sounds right.

> The other steps about incrementing the release number
> in plugins/build.xml is already noted in etc/RELEASE_PROCESS.txt

There is also this code in the plugins/build.xml file:

<if>
       <contains string="${forrest.version}" substring="0.7"/>
       <then>
         <property name="docsVersion" value="0_70"/>
       </then>
       <else>
         <if>
           <contains string="${forrest.version}" substring="0.8"/>
           <then>
             <property name="docsVersion" value="0_80"/>
           </then>
           <else>
             <!-- FIXME: the docs version number should be derived 
programmatically from the forrest.version in use
                  (see http://issues.apache.org/jira/browse/FOR-535) -->
             <fail message="Trying to deploy plugin docs to an unkown 
version of Forrest. Please update the plugins build file."/>
           </else>
         </if>
       </else>
      </if>

This is necessary because the docs directories are not the same as the 
version number.

(I have not updated the release process - need sleep)

> Also i notice that there is already a f.a.o/plugins/0.8/
> Perhaps that was an accident by specifying 0.8
> rather than 0.8-dev in a plugins.xml entry.

The forrest.version is no longer taken from plugins.xml, it is taken 
from the plugin build.xml file (need to generate plugins.xml from 
build.xml, there's an issue for that somewhere).

The two plugins that have been deployed into 0.8 now have the correct 
value for forrest.version property.

The plugins/build.xml, which provides a fallback value for the property
also had 0.8 in it, I fixed that too.

> Do we need to remove that and redeploy them?

projectInfo was already redeployed so no problem with that one. The 
voice one should be redeployed, but my local version has some 
modifications that are unfinished so I can't redeploy now, perhaps 
someone can do it after an "svn up".

>>For the future I want to create a new plugin resolving mechanism that 
>>loads plugins at runtime rather than at startup. We need to modify the 
>>Locationmap somthing like this:
>>
>><match pattern="plugin.xmap.*">
>>  <location src="{project:tmp}/{1}/>
>>  <install src="http://f.a.o/plugins/0.8/{1}/>
>>  <install src="http://f.a.o/plugins/0.7/{1}/>
>>  <location src="{forrest:plugin.fallback}/error.pluginNotFound.xmap"/>
>></match>
>>
>>The we can dynamically install features at run time rather than at 
>>startup. We would even be able to add plugins at runtime.
>>
>>Still, I digress. I've added these comments to 
>>http://issues.apache.org/jira/browse/FOR-741 (good to see JIRA forcing 
>>me to add a component)
>>
>>
>>>>In the meantime we need to advise users to add a version number as you 
>>>>discovered.
>>>
>>>I am surprised that we are not getting more pleas for
>>>help from users of the 0.7 projectInfo plugin.
>>
>>It will only affect new users. Those who have the plugin installed 
>>already will not be trying to retrieve it, therefore no problem emerges..
> 
> 
> However if anyone does a "build clean" then they
> are going to hit this trouble, e.g. people with their
> own Java customisations or someone doing 'build clean'
> when they don't really need to. Stylesheet and sitemap
> changes don't need to.

This problem will be solved by "use plugins inplace" [1] and/or 
"override plugin installation directory" [2]. I've linked both to "allow 
dynamic loading of plugin" [3]

[1] http://issues.apache.org/jira/browse/FOR-388
[2] http://issues.apache.org/jira/browse/FOR-462
[3] http://issues.apache.org/jira/browse/FOR-741

Seems like the issue tracker has everything covered, now if only the 
tracker could actually implement the solutions ;-)

(mind you with Tim's willingness to grab issues from the tracker we are 
going that way - we need more "Tims")

Ross