xmlgraphics-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Bowditch <bowditch_ch...@hotmail.com>
Subject Re: FOP Release Automation
Date Fri, 25 Jul 2014 09:26:57 GMT
Hi Vincent,

On 24/07/2014 17:45, Vincent Hennebert wrote:
> [Moving to general@ as other sub-projects can benefit from this too.]
>
> I thought I would have a go at it and believe I’ve found a solution.
> There was a problem of conflicting syntaxes between Markdown’s way of
> specifying custom header IDs, and Dotiac::DTL’s syntax for comments.

That's great news. Well done Vincent! Now we just need someone to add 
all the variables we need and write a script to change the variables at 
release time.

>
> For example:
>
>     ## Project Status {#status}
>
> The ‘{#status}’ is Mardown’s way of giving a custom ID to the generated
> H2 element. But ‘{#’ is also a way to introduce a comment in
> a Dotiac::DTL template, which then complains that there is no closing
> ‘#}’.
>
> Fortunately, both can be made to agree by inserting a space between ‘{’
> and ‘#’:
>
>     ## Project Status { #status}
>
> The Dotiac:DTL comment has now gone, but Markdown still recognises it as
> a custom ID. (A lot of them are actually unnecessary since they are
> identical to the title, and would be automatically generated by the
> Markdown HeaderId extension. They come from the conversion of Forrest
> sources, at some point we may want to scan through and remove them.)
>
> As you may have noticed I’ve just committed a batch change of header IDs
> that introduces that space. Now we can enable pre-processing of the
> Markdown sources and enjoy the use of variables (and much more if you are
> daring).
>
> Fortunately there’s hardly any Perl involved. We just need to set some
> parameters in the path.pm and define variables there. See
> http://svn.apache.org/r1613178 for more details. I’ve added just two
> variables as a proof of concept but now we can proceed scanning the
> sources and adding more variables.
>
> I don’t know whether there’s a risk that our variables conflict with the
> CMS’ own variables. If we name them properly (e.g., prefixing them with
> fop_ or batik_ or xgc_), then this should not happen.
>
> Vincent

Thanks,

Chris

>
>
> On 15/07/14 11:35, Robert Meyer wrote:
>> Hi All,
>>
>> This is an update into my investigations on automating the release 
>> process for
>> FOP. As we're nearing release it looks as though version 2.0 will 
>> remain a
>> manual process for the time being. That's not to say that it will 
>> forever
>> remain like that but at present unless a breakthrough occurs or I 
>> receive some
>> feedback from the infrastructure team, I don't currently have the 
>> necessary
>> knowledge on the Apache infrastructure (or Perl know how) to achieve the
>> desired result despite my efforts.
>>
>> During the time since my last message I found a solution using a 
>> markdown
>> extension. This appeared to fulfil all of our requirements and after 
>> writing
>> and testing one, it seemed to simply be a case of installing it. Due 
>> to the
>> nature of Apache's websites this was not something I could do myself 
>> as we
>> don't have control over the CMS. After raising a ticket with the
>> infrastructure team about doing this, I was pointed to another 
>> project called
>> Thrift. Their site appeared to provide tag replacement using preexisting
>> functionality found in the perl modules of the Apache CMS.
>>
>> After reading the documentation and experimenting I've reached 
>> somewhat of an
>> impasse due to a number of reasons. Firstly the documentation on 
>> customizing
>> these patterns is limited and covers only basic patterns. Second, my own
>> experience with Perl is lacking and as such makes it hard to debug and
>> understand some of the more complicated required modules and sections 
>> of the
>> CMS. Finally during my testing the errors I was getting were extremely
>> unhelpful and provide next to no clues as to where the problem lay in 
>> my own
>> code. Instead they point to the Perl CMS libraries highlighting missing
>> expected characters and at a guess incompatibilities between the 
>> markdown
>> we're using and what's expected by the pattern's own subroutine.
>>
>> I have tried to follow and utilize the code found in the Thrift 
>> project with
>> little luck. I have posted on the infrastructure mailing list for 
>> help but as
>> of yet have not had any responses. I am guessing this is not a 
>> commonly used
>> feature and as such knowledge on the subject may be in short supply. 
>> As such
>> and without possibility of using the markdown extension we're left 
>> with the
>> manual process for the time being. I will keep an eye out on the
>> infrastructure page and prod them occasionally to see if I can move 
>> things along.
>>
>> Apologies for the long e-mail but just wanted to keep you all updated.
>>
>> Robert Meyer
>>
>>  > Date: Mon, 2 Jun 2014 14:44:58 +0100
>>  > From: bowditch_chris@hotmail.com
>>  > To: fop-dev@xmlgraphics.apache.org
>>  > Subject: Re: FOP Release Automation
>>  >
>>  > Hi All,
>>  >
>>  > I certainly use the web interface when making small tweaks to the 
>> docs.
>>  > As you know users occasionally report small mistakes that require 
>> minor
>>  > tweaks. I'd like to streamline the updating of the website for 
>> release
>>  > purposes but I don't want to disable/prevent the current web
>>  > interface which works well when all you want to do is make a minor
>>  > adjustment in response to a user e-mail.
>>  >
>>  > Rob is away this week, but he will continue the investigation into
>>  > scripting the website updates when he returns.
>>  >
>>  > Thanks for the ideas so far, a few promising leads.
>>  >
>>  > Thanks,
>>  >
>>  > Chris
>>  >
>>  > On 30/05/2014 17:23, Clay Leeds wrote:
>>  > > Agreed, ‘some’ people wouldn’t be happy with that. ;-)
>>  > >
>>  > > I wonder if the CMS Web interface could be extended to allow for 
>> a few
>>  > > keywords like FOP_VERSION, FOP_REVISION, FOP_BRANCH, etc.
>>  > >
>>  > > The CMS tool's WYSIWYG interface indicates it uses the Wysiwym
>>  > > MarkDown Editor, which is extensible:
>>  > >
>>  > > https://web.archive.org/web/20110121060932/http://wmd-editor.com/
>>  > >
>>  > > (site’s down & hasn’t been updated since 2011)...
>>  > >
>>  > > or
>>  > >
>>  > > https://code.google.com/p/wmd/
>>  > >
>>  > > We might still need to do some ANT hanky panky, but at least if we
>>  > > could leverage WMD’s extensibility it would help us get where we’re
>>  > > trying to go?
>>  > >
>>  > > Clay
>>  > >
>>  > > On May 30, 2014, at 7:19 AM, Robert Meyer <rmeyer@hotmail.co.uk
>>  > > <mailto:rmeyer@hotmail.co.uk>> wrote:
>>  > >> Hi,
>>  > >>
>>  > >> I like the simplicity of your idea, but the web interface is 
>> not so
>>  > >> easy to dismiss unfortunately.
>>  > >>
>>  > >> If you do have a copy with those tags in, if any changes are 
>> made on
>>  > >> the web interface then that copy would become out of date.
>>  > >>
>>  > >> We could always shutdown the web interface, but I don't think too
>>  > >> many people would be happy with that ;-)
>>  > >>
>>  > >> Regards,
>>  > >>
>>  > >> Robert
>>  > >>
>>  > >> 
>> ------------------------------------------------------------------------
>>  > >> From: simonsteiner1984@gmail.com 
>> <mailto:simonsteiner1984@gmail.com>
>>  > >> To: fop-dev@xmlgraphics.apache.org
>>  > >> <mailto:fop-dev@xmlgraphics.apache.org>
>>  > >> Subject: RE: FOP Release Automation
>>  > >> Date: Fri, 30 May 2014 14:48:15 +0100
>>  > >>
>>  > >> Hi,
>>  > >>
>>  > >> Simple way is to store docs inside fop repo:
>>  > >>
>>  > >> Fop/docs/index.markdown
>>  > >>
>>  > >> Inside markdown file you reference ant properties eg:
>>  > >> ${version}
>>  > >>
>>  > >> Then you call which does ant expandproperties and calls 
>> markdown to
>>  > >> html tool:
>>  > >> ant docs
>>  > >>
>>  > >> Then you call which does a zip, scp and unzip of html files to web
>>  > >> server:
>>  > >> ant upload-docs
>>  > >>
>>  > >> This method doesn’t support web interface of editing files but I
>>  > >> don’t see how this is really needed.
>>  > >> If I submit a patch to fop it should also contain doc changes 
>> rather
>>  > >> than having separate repo and patch for that.
>>  > >>
>>  > >> Thanks
>>  > >>
>>  > >> *From:*Robert Meyer [mailto:rmeyer@hotmail.co.uk]
>>  > >> *Sent:*30 May 2014 14:05
>>  > >> *To:*fop-dev@xmlgraphics.apache.org
>>  > >> <mailto:fop-dev@xmlgraphics.apache.org>
>>  > >> *Subject:*RE: FOP Release Automation
>>  > >>
>>  > >> Hi,
>>  > >>
>>  > >> After investigating your suggestions Clay I have found that 
>> svn-hooks
>>  > >> can't be used for the purpose we require unfortunately as it 
>> may lead
>>  > >> to problems with how SVN operates and also may have some 
>> unexpected
>>  > >> results with files being committed. This is stated in the
>>  > >> documentation under "Creating Repository Hooks" highlighted in the
>>  > >> warning red box further down:
>>  > >>
>>  > >>
>> http://www-inst.eecs.berkeley.edu/~cs61b/fa09/docs/svn-book-html-chunk/svn.reposadmin.create.html

>>
>>
>>  > >>
>> <http://www-inst.eecs.berkeley.edu/%7Ecs61b/fa09/docs/svn-book-html-chunk/svn.reposadmin.create.html>

>>
>>  > >>
>>  > >> Pascal, I agree that the process is fairly straightforward, but I
>>  > >> have been asked to automate this further so am just looking into
>>  > >> ideas presently.
>>  > >>
>>  > >> I think a possible way forward then would be to use your 
>> suggestion
>>  > >> Pascal of placing the versioned docs for the site inside the FOP
>>  > >> repository for their associated version. These can then be 
>> referenced
>>  > >> using the svn-externals from the main site repository.
>>  > >>
>>  > >> In addition to this, the main site files (which would need to be
>>  > >> updated) could contain tags for the last three versions which 
>> would
>>  > >> be replaced using Clay's markdown pre-processor suggestion. The
>>  > >> pre-processor would replace the tags with values stored in a
>>  > >> properties file in the main site repository.
>>  > >>
>>  > >> To create a release, the user would need to update the 
>> svn-external
>>  > >> references to account for the new version and update the 
>> properties
>>  > >> file for tag replacement. When the properties file is pushed it 
>> will
>>  > >> be read by the custom markdown pre-processor and display the new
>>  > >> version when rendered.
>>  > >>
>>  > >> Those two stages could be done using a single script to simplify
>>  > >> things further, but the main complication is getting the markdown
>>  > >> pre-processor working. From looking at this page:
>>  > >>
>>  > >> http://www.apache.org/dev/cmsref.html#markdown
>>  > >>
>>  > >> I am guessing we use PyPy (Python Markdown) which supports
>>  > >> extensions, so I will look at this shortly to try out a small 
>> example
>>  > >> and investigate the feasibility of doing this. There is also the
>>  > >> matter of updating the versioned documents for each FOP when a new
>>  > >> version is released, but maybe this could be done with the
>>  > >> pre-processor as well.
>>  > >>
>>  > >> Anyway, let me know what you think.
>>  > >>
>>  > >> Regards,
>>  > >>
>>  > >> Robert
>>  > >
>>  >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@xmlgraphics.apache.org
> For additional commands, e-mail: general-help@xmlgraphics.apache.org
>
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: general-help@xmlgraphics.apache.org


Mime
View raw message