maven-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Plamen Totev <plamen.iv.to...@gmail.com>
Subject Re: [IMPORTANT] Re: Git migration next steps
Date Mon, 25 Dec 2017 08:58:36 GMT
Hi,

What I can see is that the tags actually introduce new branches that
contain commits with the same content as the one in the master (trunk)
branch. But this is not caused by the script that splits of the
repository - the problem exists in the maven-plugins Git repository as
well. Maybe it's not quite visible because there are a lot of commits,
but it's there.

How can it be fixed? I think it would be easier to fix the individual
plugin repositories (after the split) as they are smaller and easier
to work with. The plan is simple - as the branches created by the tags
does not contain new commits(with a single exception but I'll explain
that in a minute) and all of them are present in the master branch  we
could just change the tag to point to the same commits but in the
master branch. That will cause all extra commits and branches to be
garbage collected as they are accessible only from the tags. For our
purposes commits are equal if they point to the same root tree (the
same files with the same content). But unfortunately there is a catch.
What I said about the branches created by the tags is not strictly
true - they do have some extra commits. When you release in SVN you
have the following sequence of commits:

* [maven-release-plugin] copy for tag maven-war-plugin-3.2.0 (this is
the tagged commit)
* [maven-release-plugin] prepare release maven-war-plugin-3.2.0

After the migration to Git the master (trunk) branch contain only the
'prepare release' commit but not the 'copy for tag' commit. So if we
apply my plan then the 'copy for tag' commit will be lost. It's just a
copy (no changes, it contains the same files as the previous commit)
so I don't think it's a problem but would be nice if somebody
confirms.

So the proposed plan have the following caveats:

* the 'copy for tag' commits will be lost. No changes will be lost as
those are only copy commits
* the tags will point to the previous 'prepare release' commits
* the tags SHA will be changed because they point to a different commit

If that is OK I can write a script that will apply those changes.

What causes the branches and all those duplicating commits? Well the
master branch tracks
https://svn.apache.org/repos/asf/maven/components/trunk/maven-plugins
but the tags for the plugins are actually tracking the plugin
directory (for example:
https://svn.apache.org/repos/asf/maven/components/trunk/maven-plugins/maven-war-plugin).
So for maven-war-plugins you have two commits in the maven-plugins Git
repository - one for
https://svn.apache.org/repos/asf/maven/components/trunk/maven-plugins/maven-war-plugin
and one for https://svn.apache.org/repos/asf/maven/components/trunk/
no matter that actually those commits are the same - same date,
author, message, files, etc.

Regards,
Plamen Totev

On Sun, Dec 24, 2017 at 11:54 AM,  <herve.boutemy@free.fr> wrote:
> I'd suggest to try the process to a personal personal repo on GitHub to see if you're
able to get a better result before involving manual work from INFRA (on more than 60 repos...)
>
> (it's sad to see nobody try to explain what's happenning or improve the documented commands,
just get to a conclusion: re-do everything and pray)
>
> Regards,
>
> Hervé
>
> ----- Mail original -----
> De: "Karl Heinz Marbaise" <khmarbaise@gmx.de>
> À: "Maven Developers List" <dev@maven.apache.org>, "Robert Scholte" <rfscholte@apache.org>
> Envoyé: Dimanche 24 Décembre 2017 10:47:43
> Objet: Re: [IMPORTANT] Re: Git migration next steps
>
> Hi,
>
> On 24/12/17 10:40, Robert Scholte wrote:
>> How about a hard reset or dropping the repo and doing it all over again?
>
> If I correctly seen that ..there had no commit yet on the new git repos..
>
> So I think it would be the easiest way to do as Robert suggest ...to
> redo migration for those repos..
>
> Kind regards
> Karl Heinz
>>
>> On Wed, 20 Dec 2017 10:42:36 +0100, Hervé BOUTEMY
>> <herve.boutemy@free.fr> wrote:
>>
>>> INFRA-15679 fixed by infra team
>>> then I re-run migrate-plugins.sh script to split the svn2git mirror to
>>> per-
>>> plugin git repo
>>> and I pushed "master2" branches for m-javadoc-p, m-site-p and m-pdf-p,
>>> which
>>> were the 3 plugins which suffered from missing commits
>>>
>>> on m-site-p and m-pdf-p, I'll cherry pick the unique commit that was
>>> missed:
>>> not a big deal
>>>
>>> on m-javadoc-p, the situation is more coplex, since there was a release
>>>
>>> I also noticed that I forgot to push tags when importing: I started to
>>> do "git
>>> push --tags", but the result does not look as I expected: it creates a
>>> lot of
>>> parallel branches
>>>
>>> I'll need help from git experts: what is happening?
>>>
>>> I stopped the tags push half the way, we'll need to decide what to do...
>>> (I knew I was not a git expert and there was a risk for something
>>> weird like
>>> what's currently happening...)
>>>
>>> Any hint?
>>>
>>> Regards,
>>>
>>> Hervé
>>>
>>> Le samedi 16 décembre 2017, 16:28:48 CET Hervé BOUTEMY a écrit :
>>>> ok, I was confused by the different takes at m-javadoc-p 3.0.0
>>>>
>>>> yes, svn2git mirror is stuck [1] at r1815675
>>>>
>>>> I just opened an INFRA Jira issue
>>>> https://issues.apache.org/jira/browse/INFRA-15679
>>>>
>>>> once the svn2git mirror will be updated, we'll have to re-run the split
>>>> scripts and cherry pick the missing commits
>>>>
>>>> Regards,
>>>>
>>>> Hervé
>>>>
>>>> [1] https://github.com/apache/maven-plugins/commits/trunk
>>>>
>>>> Le samedi 16 décembre 2017, 13:01:05 CET Robert Scholte a écrit :
>>>> > I was triggered by some failing unit tests, which should have been
>>>> solved
>>>> > in maven-javadoc-plugin-3.0.0
>>>> >
>>>> > My last commit according to GIT was november 18th
>>>> > My last commit according to SVN was december 3rd
>>>> >
>>>> > comparing svnlog with gitlog most of these commits are lost:
>>>> >
>>>> > moved to git
>>>> > ----
>>>> > [maven-release-plugin] prepare for next development iteration
>>>> > ----
>>>> > [maven-release-plugin] prepare release maven-javadoc-plugin-3.0.0
>>>> > ----
>>>> > [MJAVADOC-498] "module not found" when Java 9 module-info present
>>>> > Support aggrated javadoc
>>>> > ----
>>>> > Skip several unittests for Java9
>>>> > ----
>>>> > JDK-8032205 was closed as not an issue, so not solved in Java9.
>>>> > Need to review the conclusion
>>>> > ----
>>>> > Upgrade mockito to remove warning about illegal reflective access
>>>> > ----
>>>> > Improve TestJavadocReportTest#testTestJavadoc
>>>> > J8 warns and continues with missing dependency, J9 fails.
>>>> > In fact test was wrong: dependency should have been on classpath
>>>> > ----
>>>> > unittest should prefer JAVA_HOME when executing from cmdline
>>>> > When running with Java9+ no need to switch from jre to jdk directory
>>>> > (jep220)
>>>> > ----
>>>> > MJAVADOC-502 Update DEFAULT_JAVA_API_LINKS
>>>> > ----
>>>> > session is required parameter, so cannot be null. Fix related
>>>> unittests
>>>> > ----
>>>> > Add project/artifact key to set of sourcePaths to recognize reactor
>>>> > projects versus dependencies
>>>> > ----
>>>> > Group sets of sourcepaths per project, in prepare of usage of
>>>> > module-source-path.
>>>> > ----
>>>> > Switch from List to Collection to make it easier to use Sets when
>>>> > preferred
>>>> > ----
>>>> > [maven-release-plugin] prepare for next development iteration
>>>> > ----
>>>> > [maven-release-plugin] prepare release maven-javadoc-plugin-3.0.0
>>>> > ----
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > On Sat, 16 Dec 2017 12:53:23 +0100, Hervé BOUTEMY
>>>> <herve.boutemy@free.fr>
>>>> >
>>>> > wrote:
>>>> > > looking at commits@ content https://lists.apache.org/list.html?
>>>> > > commits@maven.apache.org with subject containing
>>>> "maven/plugins/trunk"
>>>> > >
>>>> > > and plugins svn2git mirror
>>>> > > https://github.com/apache/maven-plugins/commits/
>>>> > > trunk
>>>> > >
>>>> > > only 1 commit is missing: my latest commit on maven-site-plugin
>>>> > > (the last commit for Git migration is not useful)
>>>> > >
>>>> > >
>>>> > > Same on shared showed no missing commit.
>>>> > >
>>>> > >
>>>> > > what latest commit of maven-javadoc-plugin are you looking for?
>>>> > >
>>>> > > Regards,
>>>> > >
>>>> > > Hervé
>>>> > >
>>>> > > Le samedi 16 décembre 2017, 11:56:31 CET Robert Scholte a écrit
:
>>>> > >> For everybody just a warning I faced today:
>>>> > >> If you switch to the git repos, please make sure all commits
are
>>>> > >> migrated.
>>>> > >> I noticed the latest commits of the maven-javadoc-plugin got
lost.
>>>> > >>
>>>> > >> thanks,
>>>> > >> Robert
>>>> > >>
>>>> > >> On Sat, 09 Dec 2017 17:06:09 +0100, Stephen Connolly
>>>> > >>
>>>> > >> <stephen.alan.connolly@gmail.com> wrote:
>>>> > >> > I see we have a large number of repos now on gitbox ;-)
>>>> > >> >
>>>> > >> > On Thu 7 Dec 2017 at 07:00, Hervé BOUTEMY <herve.boutemy@free.fr>
>>>> > >>
>>>> > >> wrote:
>>>> > >> >> ok, I didn't update my repo clone: now the run-its
profile is
>>>> > >>
>>>> > >> activated
>>>> > >>
>>>> > >> >> then the plan should just confirm "it works!" :)
>>>> > >> >>
>>>> > >> >> and find which jobs are special, like maven-dist-tool
(which
>>>> has to
>>>> > >>
>>>> > >> be
>>>> > >>
>>>> > >> >> scheduled daily instead of code change, and one platform
only)
>>>> > >> >>
>>>> > >> >> Regards,
>>>> > >> >>
>>>> > >> >> Hervé
>>>> > >> >>
>>>> > >> >> Le mercredi 6 décembre 2017, 23:58:45 CET Stephen
Connolly a
>>>> écrit :
>>>> > >> >> > On Wed 6 Dec 2017 at 22:38, Hervé BOUTEMY
>>>> <herve.boutemy@free.fr>
>>>> > >> >>
>>>> > >> >> wrote:
>>>> > >> >> > > Now that we have 2 ASF Organization Jenkins
jobs (one for
>>>> gitbox
>>>> > >>
>>>> > >> [1]
>>>> > >>
>>>> > >> >> and
>>>> > >> >>
>>>> > >> >> > > one
>>>> > >> >> > > for git-wip: thank you Stephen) and that
it looks quite
>>>> > >>
>>>> > >> successful,
>>>> > >>
>>>> > >> >> let's
>>>> > >> >>
>>>> > >> >> > > plan
>>>> > >> >> > > the next steps.
>>>> > >> >> > >
>>>> > >> >> > > Here is what I see:
>>>> > >> >> > > 1. removal of hand-defined Jenkins jobs
that are now
>>>> duplicates
>>>> > >> >> > >
>>>> > >> >> > > 2. preparation of the 60 new empty git repos
for shared &
>>>> > >> >> > > plugins
>>>> > >> >> > >
>>>> > >> >> > > 3. migration of the 1 shared component and
1 plugin using
>>>> > >> >>
>>>> > >> >> migrate-*.sh
>>>> > >> >>
>>>> > >> >> > > scripts
>>>> > >> >> > > [3] to test and eventually rework the Jenkinsfile
(I
>>>> suppose it
>>>> > >>
>>>> > >> will
>>>> > >>
>>>> > >> >> > > require
>>>> > >> >> > > some little change, to run add "run-its"
profile)
>>>> > >> >> >
>>>> > >> >> > As far as I recall, I added -P+run-its already
>>>> > >> >> >
>>>> > >> >> > For the plugin, I'd like to do the job for maven-site-plugin,
>>>> > >>
>>>> > >> since we
>>>> > >>
>>>> > >> >> > > expect
>>>> > >> >> > > to release it soon.
>>>> > >> >> > > For the shared component, I don't know if
there is a best
>>>> > >>
>>>> > >> candidate
>>>> > >>
>>>> > >> >> > > 4. once previous step is ok, do the full
migration: if
>>>> there are
>>>> > >> >> > > volunteers
>>>> > >> >> > > for helping, that would be great, since
populating 60 git
>>>> repos
>>>> > >> >>
>>>> > >> >> won't
>>>> > >> >> be
>>>> > >> >>
>>>> > >> >> > > really fun...
>>>> > >> >> > >
>>>> > >> >> > > And as part of 60 empty git repos creation,
I propose to
>>>> migrate
>>>> > >>
>>>> > >> the
>>>> > >>
>>>> > >> >> > > "Google
>>>> > >> >> > > repo manifest" maven-aggregator [4] to ASF:
my personal
>>>> use has
>>>> > >>
>>>> > >> been
>>>> > >>
>>>> > >> >> quite
>>>> > >> >>
>>>> > >> >> > > successful, I hope it's the same for others.
Perhaps
>>>> there are
>>>> > >> >>
>>>> > >> >> better
>>>> > >> >>
>>>> > >> >> > > ideas
>>>> > >> >> > > for its name: maven-aggregator
>>>> > >> >> > >
>>>> > >> >> > > Any other idea? any objection?
>>>> > >> >> > >
>>>> > >> >> > > Regards,
>>>> > >> >> > >
>>>> > >> >> > > Hervé
>>>> > >> >> > >
>>>> > >> >> > > [1]
>>>> https://builds.apache.org/view/M-R/view/Maven/job/maven-box/
>>>> > >> >> > >
>>>> > >> >> > > [2]
>>>> https://builds.apache.org/view/M-R/view/Maven/job/maven-wip/
>>>> > >> >> > >
>>>> > >> >> > > [3]
>>>> > >>
>>>> > >> https://svn.apache.org/viewvc/maven/sandbox/trunk/scripts/git/
>>>> > >>
>>>> > >> >> > > [4] https://github.com/hboutemy/maven-aggregator
>>>> > >> >>
>>>> > >> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Mime
View raw message