sling-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bertrand Delacretaz (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SLING-7262) Generate reusable lists of Git repositories for machine consumption
Date Wed, 31 Jan 2018 10:23:00 GMT

    [ https://issues.apache.org/jira/browse/SLING-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346554#comment-16346554
] 

Bertrand Delacretaz commented on SLING-7262:
--------------------------------------------

Trying to summarize the various initiatives going on, as well as our requirements, please
correct/complete as needed.

*Inputs*
* Either the GitHub API to list all Sling repositories as currently used at https://github.com/apache/sling-aggregator
* Or the Gitbox OPML at https://gitbox.apache.org/repos/asf?a=opml as currently used by https://github.com/oliverlietz/apache-sling-aggregator

*Desired Outputs*
* 1) A manifest for the Google Repo tool, which requires just the repository names and their
paths.
* 2) A list suitable for updating http://sling.apache.org/repolist.html (or a similar canonical
Web page)
* 3) A Maven reactor pom that allows for building everything or sets of modules, probably
based on Maven profiles

For 1) I haven't checked if the Repo manifest format officially allows us to add extra information,
currently we have an additional "groups" attribute, if that's supported we might add "tags",
"badges" and "descriptions" as needed.

For 2) the repositories need to be grouped by functional areas (already partially done) and
should also have tags or badges for contrib/core/legacy and similar classifications. It would
be nice to have a brief description of each repository as well in that list, maybe the first
few lines of their README (excluding "belongs to Apache Sling" of course)

*Environment*
The generation of these files needs to run as a Jenkins build so that they can be regenerated
automatically.

Commiting the output of Jenkins jobs to Git is easy if the target is the {{asf-site}} branch,
for that we just need to tie the jobs to nodes having the {{git-websites}} label. Committing
the output to the {{asf-site}} branch of the https://github.com/apache/sling-site repository
seems to be a good solution to me and other scripts can pick it up there at a stable URL,
even if the raw result is not visible on the website.

*Here's what we currently have "in production"*
* https://github.com/apache/sling-aggregator generates a Repo manifest, needs to be updated
manually by running a script and pushing the result when new Git repos are created
* http://sling.apache.org/repolist.html is generated from that Repo manifest. It's missing
tags/badges and descriptions
* Our Jenkins jobs are generated from https://github.com/apache/sling-tooling-jenkins and
optionally use information from a {{.sling-module.xml}} file described at https://cwiki.apache.org/confluence/display/SLING/Sling+module+descriptor
* I don't think we have an official Maven aggregator pom at this point
* [~olli] has been working on https://github.com/oliverlietz/apache-sling-aggregator with
useful things in various branches of that repository, which generate a Repo manifest, a Maven
pom and a GitHub page.

See https://github.com/apache/sling-org-apache-sling-launchpad-contrib-testing for an example
of a sling:contrib badge that can easily be parsed from the README and is displayed there,
as opposed to {{.sling-module.xml}} which has no impact on the README page on GitHub:

{code}
curl -s https://raw.githubusercontent.com/apache/sling-org-apache-sling-launchpad-contrib-testing/master/README.md
| grep sling:badge
![sling:badge:contrib](https://img.shields.io/badge/sling-contrib-blue.svg)
{code}

We should consolidate all that and ideally generate all the outputs from a single master list
that's automatically updated at regular intervals.

> Generate reusable lists of Git repositories for machine consumption
> -------------------------------------------------------------------
>
>                 Key: SLING-7262
>                 URL: https://issues.apache.org/jira/browse/SLING-7262
>             Project: Sling
>          Issue Type: Task
>          Components: Build and Source Control
>            Reporter: Robert Munteanu
>            Priority: Major
>
> We currently have two 'clients' that query the github repos and the .sling-module.xml
descriptors:
> * the jenkins job creation DSL script
> * the repo manifest creation script
> We've also discussed in SLING-7161 the possibility of listing all repositories on the
website.
> To minimize the effort of talking to github in each location and reduce the number of
calls to Github ( for rate limiting purposes ) we should generate a JSON/XML file that contains
all information about the git repos. This file is in turns read by various "clients" that
need the information.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message