cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lsimons <...@git.apache.org>
Subject [GitHub] cloudstack pull request: CLOUDSTACK-7143: Refactoring of the syste...
Date Thu, 04 Sep 2014 13:39:31 GMT
GitHub user lsimons opened a pull request:

    https://github.com/apache/cloudstack/pull/16

    CLOUDSTACK-7143: Refactoring of the systemvm build process

    E-mail thread:
      http://mail-archives.apache.org/mod_mbox/cloudstack-dev/201407.mbox/%3C7A6CF878-7A28-4D4A-BCD2-0C264F8C90B7%40schubergphilis.com%3E
    
    This started out as wanting the systemvm build to take
    systemvm/patches/debian/{debian,vpn} from the local machine/branch,
    rather than downloading from the apache git master [1]. In working out
    how on earth to get veewee to do that cleanly (hint: you can’t, hence
    resorting to shar usage) I got quite frustrated with the image rebuild
    times.
    
    It so happens that veewee has a --skip-to-postinstall instruction which
    is _quite_ useful while debugging these scripts. To get that working
    requires the post install steps to be retryable/convergent. Of course,
    our existing scripts weren’t set up for that. So I had to add a bunch
    of tests whether changes had applied already. Which implied a pretty
    significant refactor.
    
    Summarizing this kind of thing is always hard...it’s many little
    things...the interesting stuff is at the end/bottom, in particular
    the two main improvements
    
      https://github.com/schubergphilis/cloudstack/commit/142d087f6a97f6ac70a858a35d2fe8b638c58cbb
    	When working on the systemvm in isolation, or using vagrant or
    	similar tools, it can be useful to inject a custom SSH key before
    	merging a management server systemvm.iso into it. This option
    	allows that. It should _not_ have effect on management-server-
    	managed vms which always get their SSH keys	injected.
    
      https://github.com/schubergphilis/cloudstack/commit/e2240eaed18000d4d94dbf6a6e40612db1aeda34
    	The current build downloads its script from master by fetching a
    	cloudstack tarball. Besides being an unneeded load on the apache
    	git server, this is a problem when working on a branch and
    	wanting to inject a different set of scripts. It also makes it
    	pretty likely that the injected copy of the script will not match
    	what a production release wants, so there is very little chance of
    	not needing to overwrite the scripts.
    
    	Ideally we would just rsync over some files. However, veewee does
    	not provide an option to do that. In order to keep a 'cleanly
    	veewee-only' build possible, and work with any recent veewee
    	version, in this change we restor to using shar
    	(http://en.wikipedia.org/wiki/Shar) to produce an archive which
    	can execute as a script, which we feed to veewee to execute.
    
    In order to avoid having to re-do this cleanup twice, I also ended up
    merging the systemvm and systemvm64 template definitions, factoring out
    their small differences by inspecting the os architecture.
    
      https://github.com/schubergphilis/cloudstack/commit/f570b3921cd52672f841fc5f99cdd96f9737d629
      https://github.com/schubergphilis/cloudstack/commit/50e91217f90fc952182dccac02a5af06ac33fb45
    
    Everything else…well it pretty much falls into two categories:
      * general code cleanup without functional changes
      * general code defensiveness to survive various jenkins build
        scenarios
    
    All in all it should help with ongoing maintenance, I think.
    
    Most of these commits are now a while old but I wanted to wait with
    sending this upstream until we had sufficiently tested the systemvms
    built with this changed approach locally.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/schubergphilis/cloudstack feature/systemvm-refactor-for-upstream

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/cloudstack/pull/16.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #16
    
----
commit 49cbb4e20b70a19286c626ac8d9111be8752bc3f
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T07:54:13Z

    CLOUDSTACK-7143: upgrade systemvm to latest debian stable, 7.6.0.

commit 35b1875226201d5923dea57db64bbd789d9ad908
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T07:55:37Z

    CLOUDSTACK-7143: split base.sh into its two functions.

commit fb258b506964ca339ef85aa92a6217dc12259811
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T07:57:49Z

    CLOUDSTACK-7143: move network tuning from cleanup.sh to its own script.

commit 50e2c0177b0df04aa80d19264d52c8d55dc02eb3
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T08:36:11Z

    CLOUDSTACK-7143: merge systemvm templates, step 1: remove differences

commit 03dba3d901997619d9095e33afe4b39a423ac3f6
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T08:44:38Z

    CLOUDSTACK-7143: merge systemvm templates, step 2: only one template remains

commit afe696edc32dfcaf7eadcb8272f5e1fe31dc32a8
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T08:52:01Z

    CLOUDSTACK-7143: remove 'manual' from preseed config. No other changes.

commit d7aca05d1a728ffdcecf635cf5e339038ec964e9
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T08:54:57Z

    CLOUDSTACK-7143: reformat definition.rb code. No other changes.

commit 6b3657744ee14118bd8c1c72c7154b936de2298d
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T08:57:32Z

    CLOUDSTACK-7143: move add_backports to apt_upgrade.sh

commit 751a4d20ba31386f70987e10059328c718bf1d6e
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T08:58:57Z

    CLOUDSTACK-7143: cleanup apt_upgrade.sh code

commit 6129113983a7f5c8c91e98f63d567469d1fcdd22
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T09:01:15Z

    CLOUDSTACK-7143: add vagrant_box_build_time file that's standard with current veewee templates

commit 58a11d37f6441807a9cf65b7440910ec780fc5ee
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T09:01:56Z

    CLOUDSTACK-7143: cleanup configure_gruh.sh code
    
    Adding a check whether GRUB_TIMEOUT has already been set, and if it has,
    skip re-running update-grub.

commit 82dedcc0f6fc085961409f55f23cfb568f06558a
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T09:09:41Z

    CLOUDSTACK-7143: split login config out from postinstall.sh

commit 3bf9ddf208593a95843ebb728b735bdb32f91ef0
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T09:10:59Z

    CLOUDSTACK-7143: cleanup configure_login.sh code
    
    Had to change various things to make this code re-entrant. In particular,
    the sed-based manipulation of /etc/sudoers is gone and replaced with a
    simpler, minimal (but compatible) sudoers file.
    
    Remove the sshd_config tuning since sshd_config is overwritten when we
    apply the cloud_scripts overlay (from build.sh).

commit ae65bf132457313b6ec826bbfde78f939337ab88
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T09:15:23Z

    CLOUDSTACK-7143: split locale config into its own file

commit d87d60a1a8dfd887d4b37f23e89cee8354538d0a
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T09:16:48Z

    CLOUDSTACK-7143: cleanup configure_locale.sh code

commit 77677a3318f14fb51cdf4574717269adeaef906e
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T09:19:03Z

    CLOUDSTACK-7143: move fix_acpid to its own file

commit db8eb125caef9b18e3954d70ba024a426651c085
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T09:19:48Z

    CLOUDSTACK-7143: cleanup configure_acpid.sh code

commit 5003845b77abb97ba63d3cd596b0e946d4532411
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T09:21:03Z

    CLOUDSTACK-7143: cleanup cleanup.sh code

commit 6e714dc5cd231c9a96b62e304d292017a33be25c
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T09:27:16Z

    CLOUDSTACK-7143: split connection tracking config into its own file

commit 0addd5a1bf0f6810e901f2b4261c4c5a0ec113e6
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T09:41:37Z

    CLOUDSTACK-7143: move networking logic from postinstall.sh to configure_networking.sh

commit fc87fc4a9ed0b536789500eba573156cadfc6383
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T09:42:27Z

    CLOUDSTACK-7143: cleanup configure_networking.sh code

commit 95d77c2a4674b09546c436beeb1fdef2534a1eee
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T09:46:02Z

    CLOUDSTACK-7143: move install_packages from postinstall.sh to its own file

commit cdb30cc59d0bbe90c03d6a1a56c93570f97f2947
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T10:02:13Z

    CLOUDSTACK-7143: consider vhdutil a software package

commit 0e62aab3507e43eaf7598bf3f41f76cced97524e
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T10:07:53Z

    CLOUDSTACK-7143: reorg of install_systemvm_packages.sh (no logic changes, yet)

commit 3ac947fa36cd20a9a7bcd9f467aff53f9af71ace
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T10:17:06Z

    CLOUDSTACK-7143: collapse apt-get install onto one line
    
    In theory this _could_ have changed behavior (apt coming up with a different
    solution, or one of the packages configuring a new apt repository), but in my
    testing, the end result is the same.

commit 207c50aa20b92a99783fcedb0ac51fb2ec112de6
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T10:28:32Z

    CLOUDSTACK-7143: rename postinstall.sh to represent what remains

commit b9f606b4f9ab8943533a7b6d3ca43fecc3cdf5b2
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T11:03:56Z

    CLOUDSTACK-7143: cleanup configure_systemvm_services.sh code

commit 30fe938af9e958972bceaea60275c6b2dfada848
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T11:06:34Z

    CLOUDSTACK-7143: tune timeouts to expect about <=~60 min build

commit 15673808c7c9500b44e715a3f28f8b21aba0311a
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T11:08:35Z

    CLOUDSTACK-7143: cleanup zerodisk.sh code

commit 44a888aa1a9d8f5a14699b87091ef2e34520208e
Author: Leo Simons <lsimons@schubergphilis.com>
Date:   2014-07-21T11:09:14Z

    CLOUDSTACK-7143: cleanup configure_conntrack.sh code

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message