Mailing-List: contact infrastructure-dev-help@apache.org; run by ezmlm
Precedence: bulk
Reply-To: infrastructure-dev@apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Sender: Brett Porter <brett@porterclan.net>
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1081)
Subject: Re: Full support for Git at Apache, a dangerously indigo plan.
From: Brett Porter <brett@apache.org>
In-Reply-To: <AANLkTikjF01nji-+5j9dxfVzsdKt8EWMReCsgyz_Zys_@mail.gmail.com>
Date: Wed, 3 Nov 2010 17:51:13 -0400
Content-Transfer-Encoding: quoted-printable
Message-Id: <351257C0-A506-45C4-AE1B-6236BEC8DDE2@apache.org>
References: <AANLkTikjF01nji-+5j9dxfVzsdKt8EWMReCsgyz_Zys_@mail.gmail.com>
To: infrastructure-dev@apache.org

Probably a trivial thing, but how important are aggregated statistical =
info about where individual committers do their work, the "1,000,000 =
commits" type of tracking, svnsearch.org, etc. for us or PR? Do we need =
to do the svn commit mirroring to keep that, or perhaps have something =
else optionally available?

On 03/11/2010, at 10:08 AM, Paul Querna wrote:

> Please provide feedback and more tasks.  This is a brainstorm.  Please =
help out.
>=20
> If you want to help with a task, go do it.  The velocity of this
> project is entirely dependent on volunteers.
>=20
> Tomorrow I will take all the reasonable tasks we have, and put them
> each into a INFRA jira ticket.
>=20
>=20
> Stage -1: Please avoid bike sheds.
>=20
>  Goal: Indigo is my color today. <http://indigo.bikeshed.com/>  We
> will be providing full r/w git support for projects.  It isn't a
> trivial project, both for communities and for infrastructure.
>=20
>  Task: Recruit people to help with the following tasks.
>=20
>=20
>=20
> Stage 0: Basic development and design
>=20
>  Goal: Resolve any remaining questions about design and
> implementation of full Git support at Apache.
>=20
>  Task: Send this email. Status: Done!
>=20
>  Task: Evaluate Gitolite <https://github.com/sitaramc/gitolite>  We
> can definitely at least learn things from Gitolite even if we do not
> end up using the code.
>=20
>  Task: Decide on push transport.  Currently leaning towards offering
> only HTTPS based pushing, as then it is very easy to re-use our
> existing LDAP infrastructure.  We can do SSH based with a single git
> user + ssh public keys, but it is more work.
>=20
>  Task: Start putting tools, scripts and documentation in SVN.
> =
<https://svn.apache.org/repos/infra/infrastructure/trunk/projects/git/>
>=20
>=20
>=20
> Stage 1: Testing Repository
>=20
>  Goal: Build experience and document how the ASF is hosting
> repositories, and ensure correct permissions, hooks, backups,
> replication/mirror, etc are all working.
>=20
>  Task: Find server to do testing repository.  My preference is to use
> eris / harmonia as a pair in the long run, but for stage 1 we can just
> use a pair of small VMs.
>=20
>  Task: HTTPS Server setup.  Configure Apache + LDAP +
> git-http-backend
> =
<http://www.kernel.org/pub/software/scm/git/docs/git-http-backend.html>
>=20
>  Task: Figure out details of Git Slave.  Just like SVN, we will have
> git mirrored in both EU and US.  A push to the EU server should
> transparently push to the US server, and replicate back as needed.
>=20
>  Task: pre-receive hook: Authorization.  Based on groups in LDAP,
> restrict write access to members of the correct project. We will not
> have path based authorization under, only whole project.
>=20
>  Task: pre-receive hook: Repository temporarily disabled.  If
> /etc/nocommit is present, do not allow any commits.  This allows for
> server and repository maintenance.
>=20
>  Task: pre-receive hook: Do not allow history rewriting / git push =
--force.
>=20
>  Task: post-update hook: Logging of commit.  Log both the Apache
> committer, and the 'git' committer. Figure out how to make this
> information useful.
>=20
>  Task: post-receive hook: E-Mail notification.  We have a traditional
> dependence on 'good' commit emails.  We need a very good mailer, as
> SvnMailer handles many cases like large commits, weird charactersets,
> etc, that most git based mailers don't even approach.
>=20
>  Task: post-receive hook: CIA Notification.  <http://cia.vc/>
>=20
>  Task: post-receive hook: SvnPubSub Notification.  I think we should
> add support to SvnPubSub for git repositories. Maybe rename SvnPubSub
> -> VcPubSub too.
>=20
>  Task: post-receive hook: Slave Mirroring.  Every commit will be
> mirrored to a second ASF machine, just like we do with Subversion.
> This should just be a git push --mirror to a remote repository, but it
> needs testing.
>=20
>  Task: post-receive hook: Github mirroring.  After each push, every
> repository will be mirrored to github.
>=20
>  Task: Find web based view of Git repositories.  Gitweb or cgit or
> viewvc.  Evaluate and configure one of them.
>=20
>  Task: Determine policies for merge commits.  See what Postgres has
> done: <http://lwn.net/Articles/409635/>
>=20
>  Task: Document committer workflow.  Needs a /dev/ page describing
> all of the basic workflows that will be used by ASF committers.  This
> includes community expectations on how often to push, merge, and
> moving patches around.
>=20
>  Task: Document infrastructure workflows.  How to import a project
> from SVN.  How to import an external git repository. How to backup a
> repository. How to verify a repository.  How to setup a new TLP.
>=20
>  Task: Infrastructure Git run book.  Add run book for all services
> used in the git project.
> =
<https://svn.apache.org/repos/infra/infrastructure/trunk/docs/services/>
>=20
>  Task: Consider SVN-Mirroring of projects.  Once the primary VC is in
> Git, it is possible to provide a replicated copy in Subversion, so
> that existing users and build tools can continue using their checkouts
> over SVN.  We need to determine how feasible this is to actually
> implement.
>=20
>=20
>=20
> Stage 2: "beta" Testing by a project.
>=20
>  Goal: Test the new service on a volunteer project, ensuring both the
> technical and community issues are successfully addressed.
>=20
>  Task: Testing full history conversion.  We will be preserving 100%
> of the history of a project.
>=20
>  Task: Evaluate after a few months.  Whats working, whats not.
> Iterate on problems and documentation.
>=20
>  Task: Ensure widespread Infrastructure Team knowledge of git.  As
> the goal is to have Git be a core service, equivalent to Subversion,
> we must have multiple members of the team who are experienced with our
> setup and git in general. "Team" includes adding new root@ members, so
> step up :-)
>=20
>=20
>=20
> Stage 3: Wide availability
>=20
>  Goal: Projects can choose Subversion or Git as their primary Version
> Control, and conversion between each is done via standard INFRA
> tickets.

--
Brett Porter
brett@apache.org
http://brettporter.wordpress.com/