maven-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christofer Dutz <christofer.d...@c-ware.de>
Subject Re: [IMPORTANT CHANGE] building and publishing our website from Jenkins and Git instead of CMS
Date Fri, 11 May 2018 07:34:03 GMT
Hi guys,

just stumbled over this thread ... sorry for not noticing it earlier.

As I did the same for several other projects here (Apache PLC4X probably being the most advanced
site), I thought I might be able to help.

When updating to GIT, why not also switch to GitPubSub?

Also I just finished finetuning plc4x's Jenkinsfile ... perhaps it's worth having a look at.
The important parts are:
- Build the site on any node able to do that (the "git-websites" labeled nodes shouldn't be
used for the heavy testing ... we only have 1 or 2)
- Jenkins-Stash the generated site
- Unstash the site on a "git-websites" labeled node
- push the changes to the "asf-site" branch of the current projects repo
- check what's committed to the "asf-site" branch and reduce the changes to an absolute minimum
(I removed most timestamp-stuff as the pub-sub-systems do tend to choke on large changes)

Some remarks from several months of working with this setup:
- As mentioned above: gitpubsub tends to choke on large changes. Whenever it choked no changes
were published at all. Usually I had to beg Infra to re-sync the repos, a few weeks ago they
added a "resync" to the self-service portal at https://selfserve.apache.org/ (The "synchronize
git repositories" Button). Don't know if I begged so often, that they added the button to
get rid of my begging ... but in the end now things work nicely. I did reduce the amount of
choking by inspecting the commits done to the asf-site branch and refactored things to omit
unnecessary timestamps and invisible things. 

And if you run into problems ... I'm glad to help ...

Chris

PLC4X's Jenkinsfile: https://github.com/apache/incubator-plc4x/blob/master/Jenkinsfile
PLC4X's Documentation on Website generation: http://plc4x.apache.org/developers/website.html




Am 11.05.18, 01:45 schrieb "Stephen Connolly" <stephen.alan.connolly@gmail.com>:

    On Thu 10 May 2018 at 23:51, Hervé BOUTEMY <herve.boutemy@free.fr> wrote:
    
    > Le jeudi 10 mai 2018, 10:52:01 CEST Stephen Connolly a écrit :
    > >
    > https://builds.apache.org/view/M-R/view/Maven/job/maven-box/job/maven-site/j
    > > ob/master/1/console looks good. @Hervé can you delete
    > > https://builds.apache.org/view/M-R/view/Maven/job/maven-site/ if you are
    > > happy with it?
    > great job, even with multi-branch: I'll copy the Jenkinsfile to Doxia site
    >
    > How can we consult the local site build result? I don't see on master:
    > would
    > something appear in a branch?
    
    
    Ok I am having a hard time parsing that request. Could you try to rephrase?
    
    
    >
    >
    > >
    > > On 10 May 2018 at 09:37, Stephen Connolly <
    > stephen.alan.connolly@gmail.com>
    > >
    > > wrote:
    > > > On 10 May 2018 at 07:25, Hervé BOUTEMY <herve.boutemy@free.fr> wrote:
    > > >> Le mercredi 9 mai 2018, 19:40:54 CEST Karl Heinz Marbaise a écrit
:
    > > >> > Hi,
    > > >> >
    > > >> >
    > > >> > This means if you do a commit in the gitbox repository[1] the
    > job[2] on
    > > >> > our build server will start and deploy directly to production
    > site[3]
    > > >> > without the need to public via cms[4].
    > > >>
    > > >> yes, you get it
    > > >>
    > > >> > Great work...
    > > >>
    > > >> thank you
    > > >>
    > > >> > BTW: @Hervé can you explain what is meant to write the Jenkinsfile
    > to
    > > >> > make a mvn site-deploy ?
    > > >>
    > > >> currently, the Jenkins job that builds and deploy the website is
    > manually
    > > >> configured
    > > >> If there was a Jenkinsfile in the git repo, the handcrafted Jenkins
    > job
    > > >> could
    > > >> be replaced by the equivalent maven-box generated one
    > > >> https://builds.apache.org/job/maven-box/
    > > >>
    > > >> My personal knowledge of Jenkinsfile and how to test it before
    > committing
    > > >> is
    > > >> null, then we're staying for the moment with the handcrafted job
    > > >>
    > > >> Initially, I thought that only the Jenkinsfile would bring us the
    > > >> immediate
    > > >> rebuild on Git commit: but it seems the Git pubsub feature is
    > available
    > > >> also
    > > >> on handcrafted job.
    > > >
    > > > Tsk Tsk did you think that little of me that I would implement pub-sub
    > for
    > > > multibranch and leave regular in the cold just to try and drive
    > > > multibranch
    > > > adoption :-(
    > I don't know Jenkins sufficiently to know if Git pubsub is really a
    > separate
    > feature from Apache Hosted Git plugin: glad to see it works for any type
    > of
    > job
    >
    >
    > > >
    > > >> Then the only benefit we'll have is to remove the handmade
    > > >> job: nice to have, but nothing critical...
    > > >>
    > > >> Regards,
    > > >>
    > > >> Hervé
    > > >>
    > > >> > Kind regards
    > > >> > Karl Heinz Marbaise
    > > >> >
    > > >> >
    > > >> >
    > > >> > [1]: https://gitbox.apache.org/repos/asf/maven-site.git
    > > >> > [2]: https://builds.apache.org/view/M-R/view/Maven/job/maven-site/
    > > >> > [3]: https://maven.apache.org/
    > > >> > [4]: https://cms.apache.org/maven/
    > > >> >
    > > >> > On 30/04/18 22:49, Hervé BOUTEMY wrote:
    > > >> > > sites sources migration to Git at GitBox done: now you can
edit
    > Maven
    > > >>
    > > >> and
    > > >>
    > > >> > > Doxia sites directly from GitHub through the edit link
    > > >>
    > > >> > > The site build and publish Jenkins jobs are still the previous
    > ones:
    > > >> if
    > > >>
    > > >> > > someone creates a Jenkinsfile, we'll have immediate build
in
    > > >> > > maven-box
    > > >> > > job.
    > > >> > >
    > > >> > > Next step will be the move of components svnpubsub location
from
    > > >> > > infra
    > > >> > > location [1] to ASF [2] this will require a change in
    > > >> > > distributionManagement/site/url in a new parent POM and every
    > child
    > > >>
    > > >> POMs:
    > > >> > > if you intend to do a release, please tell and we'll check
what
    > to do
    > > >> > >
    > > >> > > Regards,
    > > >> > >
    > > >> > > Hervé
    > > >> > >
    > > >> > > [1]
    > > >> > > https://svn.apache.org/repos/infra/websites/production/maven
    > > >>
    > > >> /components/
    > > >>
    > > >> > > +
    > > >> > > https://svn.apache.org/repos/infra/websites/production/maven
    > > >>
    > > >> -doxia/compon
    > > >>
    > > >> > > ents/
    > > >> > >
    > > >> > > [2] https://svn.apache.org/repos/asf/maven/website/components/
    > > >> > > +
    > https://svn.apache.org/repos/asf/maven/doxia/website/components/
    > > >> > >
    > > >> > > Le dimanche 15 avril 2018, 15:25:47 CEST Hervé BOUTEMY a
écrit :
    > > >> > >> Jira issue for source migration to Git opened:
    > > >> > >> https://issues.apache.org/jira/browse/INFRA-16361
    > > >> > >>
    > > >> > >> Regards,
    > > >> > >>
    > > >> > >> Hervé
    > > >> > >>
    > > >> > >> Le dimanche 15 avril 2018, 01:23:06 CEST Hervé BOUTEMY
a écrit :
    > > >> > >>> here it is, now Maven site and Doxia sub-site are
published by
    > > >>
    > > >> Jenkins
    > > >>
    > > >> > >>> CMS is not used any more
    > > >> > >>> We'll just need to move components reference documentation
out
    > of
    > > >>
    > > >> the
    > > >>
    > > >> > >>> CMS
    > > >> > >>> svn space: I'll do it later with infra
    > > >> > >>>
    > > >> > >>> now, I'll work on GitBox migration
    > > >> > >>>
    > > >> > >>> If someone can write a Jenkinsfile, this would be
great: the job
    > > >>
    > > >> has to
    > > >>
    > > >> > >>> be
    > > >> > >>> tied to websites1 node, and the command is simply
"mvn
    > site-deploy"
    > > >> > >>>
    > > >> > >>> Regards,
    > > >> > >>>
    > > >> > >>> Hervé
    > > >> > >>>
    > > >> > >>> Le samedi 14 avril 2018, 22:07:04 CEST Hervé BOUTEMY
a écrit :
    > > >> > >>>> FYI, switch is happening currently, I'm on HipChat
with infra
    > > >> > >>>>
    > > >> > >>>> then CMS does not have any effect now: this will
be Jenkins job
    > > >> > >>>> (or
    > > >> > >>>> manual
    > > >> > >>>> build)
    > > >> > >>>>
    > > >> > >>>> please don't try to publish anything for now:
I'll tell once
    > > >>
    > > >> everything
    > > >>
    > > >> > >>>> is
    > > >> > >>>> stabilized
    > > >> > >>>>
    > > >> > >>>> Regards,
    > > >> > >>>>
    > > >> > >>>> Hervé
    > > >> > >>>>
    > > >> > >>>> Le vendredi 13 avril 2018, 18:15:21 CEST Karl
Heinz Marbaise a
    > > >>
    > > >> écrit :
    > > >> > >>>>> Hi Hervé,
    > > >> > >>>>>
    > > >> > >>>>> On 12/04/18 08:17, Hervé BOUTEMY wrote:
    > > >> > >>>>>> Hi,
    > > >> > >>>>>>
    > > >> > >>>>>> I'm working with infra for a good time
on Git migration for
    > site
    > > >> > >>>>>> source,
    > > >> > >>>>>> which will de-facto mean that we drop
Apache CMS (that only
    > > >>
    > > >> supports
    > > >>
    > > >> > >>>>>> svn
    > > >> > >>>>>> for source): see INFRA-16088 [1]
    > > >> > >>>>>>
    > > >> > >>>>>> Today, we got a first visible result:
Jenkins was able to
    > build
    > > >>
    > > >> and
    > > >>
    > > >> > >>>>>> publish
    > > >> > >>>>>> html output of the Maven main website
to svn [2]
    > > >> > >>>>>>
    > > >> > >>>>>> We're now ready to switch to the new
Jenkins build, then drop
    > > >>
    > > >> CMS:
    > > >> > >>>>>> any
    > > >> > >>>>>> objection to do it?
    > > >> > >>>>>>
    > > >> > >>>>>> The plan will be then to switch source
to GitBox, to benefit
    > > >> > >>>>>> from
    > > >> > >>>>>> immediate
    > > >> > >>>>>> commit detection (instead of svn hourly
polling): at this
    > step,
    > > >> > >>>>>> you'll
    > > >> > >>>>>> be
    > > >> > >>>>>> able to do easy site source updates through
GitHub, with
    > > >>
    > > >> immediate
    > > >>
    > > >> > >>>>>> publish. I don't see the value of adding
a staging step, but
    > if
    > > >> > >>>>>> someone
    > > >> > >>>>>> wants staging, it will be a matter of
Jenkinsfile tweaking.
    > > >> > >>>>>>
    > > >> > >>>>>> In the temporary time with Jenkins polling
on source from
    > svn ,
    > > >> > >>>>>> either
    > > >> > >>>>>> you'll wait for less than 1 hour or you'll
manually launch
    > the
    > > >> > >>>>>> Jenkins
    > > >> > >>>>>> job
    > > >> > >>>>>>
    > > >> > >>>>>> any question?
    > > >> > >>>>>
    > > >> > >>>>> Jenkins will build the Site (as done before
    > > >> > >>>>> ci.apache.org/builders/maven-site-staging)
and deploys it to
    > the
    > > >>
    > > >> site
    > > >>
    > > >> > >>>>> (maven.staging.apache.org?)...but it's gets
the source of the
    > > >>
    > > >> site
    > > >>
    > > >> > >>>>> from
    > > >> > >>>>> svn at the moment ?.
    > > >> > >>>>>
    > > >> > >>>>> If I wan't to publish it to live I have to
go via
    > > >> > >>>>> cms.apache.org/maven/publish as before (for
now)...
    > > >> > >>>>>
    > > >> > >>>>>
    > > >> > >>>>>
    > > >> > >>>>> Kind regards
    > > >> > >>>>> Karl Heinz Marbaise
    > > >> > >>>>
    > > >> > >>>> ------------------------------------------------------------
    > > >>
    > > >> ---------
    > > >>
    > > >> > >>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
    > > >> > >>>> For additional commands, e-mail: dev-help@maven.apache.org
    > > >> > >>>
    > > >> > >>> ------------------------------------------------------------
    > > >>
    > > >> ---------
    > > >>
    > > >> > >>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
    > > >> > >>> For additional commands, e-mail: dev-help@maven.apache.org
    > > >> > >>
    > > >> > >> ------------------------------------------------------------
    > > >>
    > > >> ---------
    > > >>
    > > >> > >> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
    > > >> > >> For additional commands, e-mail: dev-help@maven.apache.org
    > > >> > >
    > > >> > >
    > ---------------------------------------------------------------------
    > > >> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
    > > >> > > For additional commands, e-mail: dev-help@maven.apache.org
    > > >> >
    > > >> > Mit freundlichem Gruß
    > > >> > Karl-Heinz Marbaise
    > > >>
    > > >> ---------------------------------------------------------------------
    > > >> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
    > > >> For additional commands, e-mail: dev-help@maven.apache.org
    >
    >
    >
    > ---------------------------------------------------------------------
    > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
    > For additional commands, e-mail: dev-help@maven.apache.org
    >
    > --
    Sent from my phone
    

Mime
View raw message