xmlgraphics-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vincent Hennebert <vhenneb...@gmail.com>
Subject Re: FOP Release Automation
Date Thu, 24 Jul 2014 16:45:04 GMT
[Moving to general@ as other sub-projects can benefit from this too.]

I thought I would have a go at it and believe I’ve found a solution.
There was a problem of conflicting syntaxes between Markdown’s way of
specifying custom header IDs, and Dotiac::DTL’s syntax for comments.

For example:

     ## Project Status {#status}

The ‘{#status}’ is Mardown’s way of giving a custom ID to the generated
H2 element. But ‘{#’ is also a way to introduce a comment in
a Dotiac::DTL template, which then complains that there is no closing
‘#}’.

Fortunately, both can be made to agree by inserting a space between ‘{’
and ‘#’:

     ## Project Status { #status}

The Dotiac:DTL comment has now gone, but Markdown still recognises it as
a custom ID. (A lot of them are actually unnecessary since they are
identical to the title, and would be automatically generated by the
Markdown HeaderId extension. They come from the conversion of Forrest
sources, at some point we may want to scan through and remove them.)

As you may have noticed I’ve just committed a batch change of header IDs
that introduces that space. Now we can enable pre-processing of the
Markdown sources and enjoy the use of variables (and much more if you are
daring).

Fortunately there’s hardly any Perl involved. We just need to set some
parameters in the path.pm and define variables there. See
http://svn.apache.org/r1613178 for more details. I’ve added just two
variables as a proof of concept but now we can proceed scanning the
sources and adding more variables.

I don’t know whether there’s a risk that our variables conflict with the
CMS’ own variables. If we name them properly (e.g., prefixing them with
fop_ or batik_ or xgc_), then this should not happen.

Vincent


On 15/07/14 11:35, Robert Meyer wrote:
> Hi All,
>
> This is an update into my investigations on automating the release process for
> FOP. As we're nearing release it looks as though version 2.0 will remain a
> manual process for the time being. That's not to say that it will forever
> remain like that but at present unless a breakthrough occurs or I receive some
> feedback from the infrastructure team, I don't currently have the necessary
> knowledge on the Apache infrastructure (or Perl know how) to achieve the
> desired result despite my efforts.
>
> During the time since my last message I found a solution using a markdown
> extension. This appeared to fulfil all of our requirements and after writing
> and testing one, it seemed to simply be a case of installing it. Due to the
> nature of Apache's websites this was not something I could do myself as we
> don't have control over the CMS. After raising a ticket with the
> infrastructure team about doing this, I was pointed to another project called
> Thrift. Their site appeared to provide tag replacement using preexisting
> functionality found in the perl modules of the Apache CMS.
>
> After reading the documentation and experimenting I've reached somewhat of an
> impasse due to a number of reasons. Firstly the documentation on customizing
> these patterns is limited and covers only basic patterns. Second, my own
> experience with Perl is lacking and as such makes it hard to debug and
> understand some of the more complicated required modules and sections of the
> CMS. Finally during my testing the errors I was getting were extremely
> unhelpful and provide next to no clues as to where the problem lay in my own
> code. Instead they point to the Perl CMS libraries highlighting missing
> expected characters and at a guess incompatibilities between the markdown
> we're using and what's expected by the pattern's own subroutine.
>
> I have tried to follow and utilize the code found in the Thrift project with
> little luck. I have posted on the infrastructure mailing list for help but as
> of yet have not had any responses. I am guessing this is not a commonly used
> feature and as such knowledge on the subject may be in short supply. As such
> and without possibility of using the markdown extension we're left with the
> manual process for the time being. I will keep an eye out on the
> infrastructure page and prod them occasionally to see if I can move things along.
>
> Apologies for the long e-mail but just wanted to keep you all updated.
>
> Robert Meyer
>
>  > Date: Mon, 2 Jun 2014 14:44:58 +0100
>  > From: bowditch_chris@hotmail.com
>  > To: fop-dev@xmlgraphics.apache.org
>  > Subject: Re: FOP Release Automation
>  >
>  > Hi All,
>  >
>  > I certainly use the web interface when making small tweaks to the docs.
>  > As you know users occasionally report small mistakes that require minor
>  > tweaks. I'd like to streamline the updating of the website for release
>  > purposes but I don't want to disable/prevent the current web
>  > interface which works well when all you want to do is make a minor
>  > adjustment in response to a user e-mail.
>  >
>  > Rob is away this week, but he will continue the investigation into
>  > scripting the website updates when he returns.
>  >
>  > Thanks for the ideas so far, a few promising leads.
>  >
>  > Thanks,
>  >
>  > Chris
>  >
>  > On 30/05/2014 17:23, Clay Leeds wrote:
>  > > Agreed, ‘some’ people wouldn’t be happy with that. ;-)
>  > >
>  > > I wonder if the CMS Web interface could be extended to allow for a few
>  > > keywords like FOP_VERSION, FOP_REVISION, FOP_BRANCH, etc.
>  > >
>  > > The CMS tool's WYSIWYG interface indicates it uses the Wysiwym
>  > > MarkDown Editor, which is extensible:
>  > >
>  > > https://web.archive.org/web/20110121060932/http://wmd-editor.com/
>  > >
>  > > (site’s down & hasn’t been updated since 2011)...
>  > >
>  > > or
>  > >
>  > > https://code.google.com/p/wmd/
>  > >
>  > > We might still need to do some ANT hanky panky, but at least if we
>  > > could leverage WMD’s extensibility it would help us get where we’re
>  > > trying to go?
>  > >
>  > > Clay
>  > >
>  > > On May 30, 2014, at 7:19 AM, Robert Meyer <rmeyer@hotmail.co.uk
>  > > <mailto:rmeyer@hotmail.co.uk>> wrote:
>  > >> Hi,
>  > >>
>  > >> I like the simplicity of your idea, but the web interface is not so
>  > >> easy to dismiss unfortunately.
>  > >>
>  > >> If you do have a copy with those tags in, if any changes are made on
>  > >> the web interface then that copy would become out of date.
>  > >>
>  > >> We could always shutdown the web interface, but I don't think too
>  > >> many people would be happy with that ;-)
>  > >>
>  > >> Regards,
>  > >>
>  > >> Robert
>  > >>
>  > >> ------------------------------------------------------------------------
>  > >> From: simonsteiner1984@gmail.com <mailto:simonsteiner1984@gmail.com>
>  > >> To: fop-dev@xmlgraphics.apache.org
>  > >> <mailto:fop-dev@xmlgraphics.apache.org>
>  > >> Subject: RE: FOP Release Automation
>  > >> Date: Fri, 30 May 2014 14:48:15 +0100
>  > >>
>  > >> Hi,
>  > >>
>  > >> Simple way is to store docs inside fop repo:
>  > >>
>  > >> Fop/docs/index.markdown
>  > >>
>  > >> Inside markdown file you reference ant properties eg:
>  > >> ${version}
>  > >>
>  > >> Then you call which does ant expandproperties and calls markdown to
>  > >> html tool:
>  > >> ant docs
>  > >>
>  > >> Then you call which does a zip, scp and unzip of html files to web
>  > >> server:
>  > >> ant upload-docs
>  > >>
>  > >> This method doesn’t support web interface of editing files but I
>  > >> don’t see how this is really needed.
>  > >> If I submit a patch to fop it should also contain doc changes rather
>  > >> than having separate repo and patch for that.
>  > >>
>  > >> Thanks
>  > >>
>  > >> *From:*Robert Meyer [mailto:rmeyer@hotmail.co.uk]
>  > >> *Sent:*30 May 2014 14:05
>  > >> *To:*fop-dev@xmlgraphics.apache.org
>  > >> <mailto:fop-dev@xmlgraphics.apache.org>
>  > >> *Subject:*RE: FOP Release Automation
>  > >>
>  > >> Hi,
>  > >>
>  > >> After investigating your suggestions Clay I have found that svn-hooks
>  > >> can't be used for the purpose we require unfortunately as it may lead
>  > >> to problems with how SVN operates and also may have some unexpected
>  > >> results with files being committed. This is stated in the
>  > >> documentation under "Creating Repository Hooks" highlighted in the
>  > >> warning red box further down:
>  > >>
>  > >>
> http://www-inst.eecs.berkeley.edu/~cs61b/fa09/docs/svn-book-html-chunk/svn.reposadmin.create.html
>
>  > >>
> <http://www-inst.eecs.berkeley.edu/%7Ecs61b/fa09/docs/svn-book-html-chunk/svn.reposadmin.create.html>
>  > >>
>  > >> Pascal, I agree that the process is fairly straightforward, but I
>  > >> have been asked to automate this further so am just looking into
>  > >> ideas presently.
>  > >>
>  > >> I think a possible way forward then would be to use your suggestion
>  > >> Pascal of placing the versioned docs for the site inside the FOP
>  > >> repository for their associated version. These can then be referenced
>  > >> using the svn-externals from the main site repository.
>  > >>
>  > >> In addition to this, the main site files (which would need to be
>  > >> updated) could contain tags for the last three versions which would
>  > >> be replaced using Clay's markdown pre-processor suggestion. The
>  > >> pre-processor would replace the tags with values stored in a
>  > >> properties file in the main site repository.
>  > >>
>  > >> To create a release, the user would need to update the svn-external
>  > >> references to account for the new version and update the properties
>  > >> file for tag replacement. When the properties file is pushed it will
>  > >> be read by the custom markdown pre-processor and display the new
>  > >> version when rendered.
>  > >>
>  > >> Those two stages could be done using a single script to simplify
>  > >> things further, but the main complication is getting the markdown
>  > >> pre-processor working. From looking at this page:
>  > >>
>  > >> http://www.apache.org/dev/cmsref.html#markdown
>  > >>
>  > >> I am guessing we use PyPy (Python Markdown) which supports
>  > >> extensions, so I will look at this shortly to try out a small example
>  > >> and investigate the feasibility of doing this. There is also the
>  > >> matter of updating the versioned documents for each FOP when a new
>  > >> version is released, but maybe this could be done with the
>  > >> pre-processor as well.
>  > >>
>  > >> Anyway, let me know what you think.
>  > >>
>  > >> Regards,
>  > >>
>  > >> Robert
>  > >
>  >

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: general-help@xmlgraphics.apache.org


Mime
View raw message