cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daan Hoogland <daan.hoogl...@gmail.com>
Subject Re: [DISCUSS][PROPOSAL] git workflow
Date Thu, 24 Jul 2014 08:45:38 GMT
Any advice on our procedure from this, Leo?

On Thu, Jul 24, 2014 at 10:39 AM, Leo Simons <LSimons@schubergphilis.com> wrote:
> On Jul 24, 2014, at 8:39 AM, Rajani Karuturi <Rajani.Karuturi@citrix.com> wrote:
>> here is what i propose:
>>
>> 1. rename 'master' to 'develop’
>> 2. branch a new 'master' from '4.4’
>> 3. branch 'RELEASE-4.5' from the develop
>> 4. merge 'RELEASE-4.5' to master once the release voting is done.
>
> I like this conceptually; I’m not sure if its feasible because step 4 requires resolving
the cherry-pick mess.
>
> Ignoring awsapi (which has a lot of churn) and ignoring formatting changes, here are
sizes of very minimalized diffs between branches
>
>               branch two    diff size   big diff chunks
>   branch one
>   4.3         4.4           11MB        hyperv,netscaler,server,engine
>   4.4         4.4-forward    2MB        tests,marvin
>   4.4-forward master         6MB        xenapi,xen,xenserver,storage
>   4.4         master         8MB        xenapi,tests,marvin,xen,xenserver
>   (See below how I got the numbers.)
>
> Starting git-flow from 4.4 or 4.4-forward and then merging things in from master means
processing hundreds of thousands of lines of diff. Of those lines, many many thousands of
lines will probably not auto-merge due to the cherry-pick approach. A rebase between any of
these branches is not feasible; git cannot un-pick what happened so it cannot offer much assistance.
>
> Looking at all the diff stats, breaking xenapi and awsapi out into their own git repositories
(and release cycles) seems like a really good thing to do; I expect it will help a lot with
future merges.
>
> You can try step 4. for yourself now:
>
>   git checkout 4.4
>   git merge master
>
> Good luck ;-)
>
>> RELEASE-4.6 branch should be off develop as all the feature branches would be merged
there before freeze for 4.6 and would be merged to master when the release is voted.
>
> Small note, git-flow tends to use release/ prefixes for release branches. You'd label
‘em
>
>   master
>   develop
>   feature/nuage-vsp-support
>   feature/frob-the-foo
>   support/license-header-update
>   release/4.5
>   release/4.6
>
>> The other question I have is in the step:4. how are we going to manage fixes to 4.5
post branch cut?  ( assuming no features as the freeze is done)
>
> Yes that’d be typical with git-flow. Once you create the release branch, all bugfixes
that are to make it into that release go onto the release branch _first_ (and _only_ fixes
go there), before being merged into develop.
>
> With git-flow you don’t really have to call it a “freeze”. Development of new features
continues (on their feature branches) uninterrupted even if a release is being cut, and merging
of completed features still is exactly the same (to develop) regardless of whether there’s
active release branches.
>
> On many projects using git-flow there is no rush to try and get features merged before
a release cut-off date, because cutting another release is easy and cheap that you can just
do one ‘whenever’ to get your feature out there.
>
> For cloudstack that’s not quite the case due to the somewhat heavyweight test/release
process. What I imagine is that if a feature misses a deadline, and the community decides
it wants to include that feature anyway, is rebasing it onto the release branch and merging
it
>
>   ...decide a finished feature goes into 4.5 after all...
>   git branch feature/frob-the-foo-4.5 feature/frob-the-foo
>   git checkout feature/frob-the-foo-4.5
>   git rebase -i release/4.5
>   git push origin feature/frob-the-foo-4.5
>   git checkout release/4.5
>   git pull --rebase
>   git merge feature/frob-the-foo-4.5
>   git checkout develop
>   git pull --rebase
>   git merge release/4.5
>   git push
>   git branch -d feature/frob-the-foo-4.5
>   git branch -d feature/frob-the-foo
>
> There’s little to no chance cloudstack could ever safely decide to do this right now,
but a few months down the road, this might become safe enough, especially since the two explicit
merge commits allow figuring out what happened, and possibly rolling back if the feature destabilizes
the release.
>
> Your workflow would be that most tests and most QA work runs on the release branch, while
developers switch between release branch and develop branch depending on what they are doing.
A developer that is picking up a bug report associated with that release will basically
>
>   ...get the bug report...
>   git stash save what-I-was-doing
>   git checkout release/4.5
>   git pull --rebase
>   ...code code code...
>   ...test...
>   git commit && git push
>   git checkout develop
>   git merge release/4.5
>   git push
>   git stash pop what-I-was-doing
>   ...code code code...
>
> For bigger amounts of change/experimentation to work on a nasty kind of bug, you might
have your own local temporary branch (I almost always do this myself):
>
>   ...get the bug report...
>   git stash save what-I-was-doing
>   git checkout release/4.5
>   git pull --ff-only
>   git branch bugfix/CLOUDSTACK-42 release/4.5
>   git checkout bugfix/CLOUDSTACK-42
>   ...code code code...
>   git commit
>   ...code code code...
>   git commit
>   ...test...
>   git pull
>   git rebase -i release/4.5
>   git commit
>   git checkout release/4.5
>   git merge bugfix/CLOUDSTACK-42
>   git commit
>   git checkout develop
>   git merge release/4.5
>   git push origin develop release/4.5
>   git branch -d bugfix/CLOUDSTACK-12345
>   git stash pop what-I-was-doing
>   ...code code code...
>
> It’s a bit of adjustment, but once you get used to it, this lots-of-branching way of
working is soooo much nicer than most alternatives :-)
>
>
> cheers,
>
>
> Leo
>
> PS: diff stats...
>
> $ git clean -f -x -d
>
> $ git remote -v
> lsimons git@github.com:lsimons/cloudstack.git (fetch)
> lsimons git@github.com:lsimons/cloudstack.git (push)
> origin  git://git.apache.org/cloudstack.git (fetch)
> origin  git://git.apache.org/cloudstack.git (push)
> sbpgh git@github.com:schubergphilis/cloudstack.git (fetch)
> sbpgh git@github.com:schubergphilis/cloudstack.git (push)
>
> # snapshot important branches
> git branch 4.3-201407231043 4.3
> git branch 4.4-201407231043 4.4
> git branch 4.4-forward-201407231043 4.4-forward
> git branch master-201407231043 master
>
> # create branches which reformat the important branches
> git branch 4.3-201407231043-format 4.3-201407231043
> git branch 4.4-201407231043-format 4.4-201407231043
> git branch 4.4-forward-201407231043-format 4.4-forward-201407231043
> git branch master-201407231043-format master-201407231043
>
> # patcher....this is pseudocode, the pom.xml patch doesn't work / apply cleanly
> #   across branches
> function patcher() {
> patch -p1 <<END
> diff --git a/pom.xml b/pom.xml
> index 3d1e747..ad792ac 100644
> --- a/pom.xml
> +++ b/pom.xml
> @@ -463,6 +463,11 @@
>      <pluginManagement>
>        <plugins>
>          <plugin>
> +          <groupId>com.googlecode.maven-java-formatter-plugin</groupId>
> +          <artifactId>maven-java-formatter-plugin</artifactId>
> +          <version>0.4</version>
> +        </plugin>
> +        <plugin>
>            <artifactId>maven-clean-plugin</artifactId>
>            <configuration>
>              <excludeDefaultDirectories>true</excludeDefaultDirectories>
> END
> }
>
> # trick to reformatting all java code on a branch
> function formatter() {
>   git clean -f -x -d
>   git checkout $1
>   git clean -f -x -d
>   mvn com.googlecode.maven-java-formatter-plugin:maven-java-formatter-plugin:0.4:format
>   git add .
>   git commit -m "format"
> }
>
> formatter 4.3-201407231043-format
> formatter 4.4-201407231043-format
> formatter 4.4-forward-201407231043-format
> formatter master-201407231043-format
>
> # trick to get minimalistic diff between two branches
> function minimaldiff() {
>   git diff \
>    \
>     -U0 -M -C -w --ignore-blank-lines --find-copies-harder \
>     --diff-algorithm=minimal \
>     $1..$2 | egrep -v '^[+-] *#'
> }
>
> minimaldiff 4.3-201407231043-format 4.4-201407231043-format | > ~/4.3-to-4.4.txt
> 4.3-to-4.4.txt is a 31MB text file
>
> $ git diff --dirstat=lines,2,cumulative 4.3-201407231043-format 4.4-201407231043-format
>   37.9% awsapi/src/com/amazon/ec2/client/
>   76.5% awsapi/src/com/amazon/ec2/
>    3.5% awsapi/src/com/amazon/s3/client/
>    7.7% awsapi/src/com/amazon/s3/
>   84.2% awsapi/src/com/amazon/
>    2.5% awsapi/src/com/cloud/
>   86.7% awsapi/src/com/
>   86.7% awsapi/
>    4.3% plugins/
>
> 85% of the diff is in awsapi, so lets filter that out...
>
> git filter-branch \
>   --index-filter 'git rm -r --cached --ignore-unmatch awsapi' \
>   4.3-201407231043-format \
>   4.4-201407231043-format \
>   4.4-forward-201407231043-format \
>   master-201407231043-format
>
> minimaldiff 4.3-201407231043-format 4.4-201407231043-format > ~/4.3-to-4.4-no-aws.txt
> 4.3-to-4.4-no-aws.txt is a 11MB text file
>
> $ git diff --dirstat=lines,4 4.3-201407231043-format 4.4-201407231043-format
>    5.4% api/src/org/apache/cloudstack/api/command/
>    4.0% deps/XenServerJava/src/com/xensource/xenapi/
>    6.4% engine/
>    4.1% plugins/hypervisors/hyperv/DotNet/ServerResource/
>   10.3% plugins/hypervisors/
>    4.3% plugins/network-elements/netscaler/src/com/cloud/
>   10.0% plugins/network-elements/
>    8.1% server/src/com/cloud/
>    5.1% services/secondary-storage/
>    4.9% ui/scripts/
>    4.7% ui/
>    4.1% vmware-base/src/com/cloud/hypervisor/vmware/mo/
>
> minimaldiff 4.4-201407231043-format 4.4-forward-201407231043-format > ~/4.4-to-4.4.1-no-aws.txt
> ~/4.4-to-4.4.1-no-aws.txt is a 2MB text file
>
> $ git diff --dirstat=lines,2 4.4-201407231043-format 4.4-forward-201407231043-format
>    2.8% plugins/hypervisors/hyperv/DotNet/ServerResource/WmiWrappers/
>    2.4% plugins/hypervisors/
>   47.6% test/integration/component/
>    8.1% test/integration/smoke/
>    2.8% tools/marvin/marvin/config/
>    9.4% tools/marvin/marvin/integration/lib/
>   11.7% tools/marvin/marvin/lib/
>    6.9% tools/marvin/marvin/
>    2.2% ui/scripts/
>
> minimaldiff 4.4-201407231043-format master-201407231043-format > ~/4.4.0-to-4.5-no-aws.txt
> ~/4.4.0-to-4.5-no-aws.txt is a 7.7MB text file
>
> $ git diff --dirstat=lines,4 4.4-201407231043-format master-201407231043-format
>   26.5% deps/XenServerJava/src/com/xensource/xenapi/
>    9.8% plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/
>    9.7% plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resourc
>    4.9% plugins/hypervisors/
>    7.6% plugins/
>   13.1% test/integration/component/
>    8.8% tools/marvin/marvin/
>
> minimaldiff 4.4-forward-201407231043-format master-201407231043-format > ~/4.4.1-to-4.5-no-aws.txt
> ~/4.4.1-to-4.5-no-aws.txt is a 5.6MB text file
>
> $ git diff --dirstat=lines,4 4.4-forward-201407231043-format master-201407231043-format
>   35.4% deps/XenServerJava/src/com/xensource/xenapi/
>   13.1% plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/
>   13.0% plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resourc
>    4.6% plugins/hypervisors/
>    4.2% plugins/network-elements/
>    5.1% plugins/storage/volume/
>    4.1% scripts/
>    4.1% server/
>



-- 
Daan

Mime
View raw message