Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Mon, 30 May 2011 02:01:48 +0000 (UTC)
From: "Todd Lipcon (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: 
 <1905605821.52599.1306720908543.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: <6014584.371831295027265635.JavaMail.jira@thor>
Subject: [jira] [Commented] (HADOOP-7106) Re-organize hadoop subversion
 layout
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-7106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040971#comment-13040971 ] 

Todd Lipcon commented on HADOOP-7106:
-------------------------------------

This afternoon I performed the following tests:

- I set up a local mirror from last month's SVN dump, and then used the ASF's git mirror scripts to create a local git mirror as well.
- Modified Nigel's HADOOP-7106.sh script with following changes:
-- parameterized svn location (the version of svn on my rhel box didn't support the --depth argument)
-- parameterized SVN root, so I could point it at my local mirror
-- fixed the svn externals links to point at hadoop/common/trunk/common instead of hadoop/trunk/common
- Ran Nigel's script and verified that trunk and branch-0.22 had the correct layout
- Committed it to my local svn mirror
- ran the "update-mirror.sh" git mirror script. This took 20 minutes or so as it pulled in all of the history from the new branches. It might take longer upstream.

At this point, the git mirror showed a single commit in trunk that moved all of the files inside common/ and added hdfs/* and mapreduce/* as new files (rather than detecting any kind of merge). This is what I expected

I tried a few things at this point, but ran into some limitations of git: namely, that git won't detect renames that happen as part of a merge commit. So, I took the following angle of attack in a local repo:
- fetch trunk from hadoop-hdfs.git (this is the last commit before 7106 is committed -- since 7106 removed the svn directory, the hdfs.git repo basically got frozen at this point)
- add a new commit where I mv everything inside an hdfs/ directory
- do the same thing for mapreduce
- do the same thing for common (with the commit right before HADOOP-7106
- create a new commit for "trunk" which has the above three branches as parents, and the same log message:

{noformat}
commit 34f047ed9e435be5932d53165477064144f5961c
Merge: 898037a 2ec2d49 0619a1a
Author: Todd Lipcon <todd@cloudera.com>
Date:   Sun May 29 18:35:31 2011 -0700

    HADOOP-7106. Re-organize layout
    
    git-svn-id: file:///data/1/todd/asf-load/hadoop/common/trunk@1098499 13f79535-47bb-0310-9956-ffa450edef68
{noformat}

- verified that commands like "git log -M --follow hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java" properly follow the history through the merge
- force push this new 'trunk' back into hadoop-common.git:refs/remotes/trunk

Next I made some svn commits in the merged repo and verified that the 'update-mirror.sh' script pulled them in on top of the merge with no problems.

A script to perform the above sequence is attached here as HADOOP-7106-git.sh.

In order to do this on ASF, I'll need to have access to the box that does the git mirroring. I guess I need to talk to Infra people to get that.

> Re-organize hadoop subversion layout
> ------------------------------------
>
>                 Key: HADOOP-7106
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7106
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: build
>            Reporter: Nigel Daley
>            Assignee: Nigel Daley
>            Priority: Blocker
>             Fix For: 0.22.0
>
>         Attachments: HADOOP-7106-auth.patch, HADOOP-7106-auth.patch, HADOOP-7106-auth.patch, HADOOP-7106-git.sh, HADOOP-7106.sh, HADOOP-7106.sh, HADOOP-7106.sh, HADOOP-7106.sh, HADOOP-7106.sh, HADOOP-7106.sh
>
>
> As discussed on general@ at http://tinyurl.com/4q6lhxm

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira