incubator-cloudstack-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ke4...@apache.org
Subject [2/5] fixing release notes name
Date Mon, 22 Oct 2012 16:22:08 GMT
http://git-wip-us.apache.org/repos/asf/incubator-cloudstack/blob/026b8d62/docs/en-US/Release_Notes.xml
----------------------------------------------------------------------
diff --git a/docs/en-US/Release_Notes.xml b/docs/en-US/Release_Notes.xml
new file mode 100644
index 0000000..849dc5a
--- /dev/null
+++ b/docs/en-US/Release_Notes.xml
@@ -0,0 +1,2921 @@
+<?xml version='1.0' encoding='utf-8' ?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
+<!ENTITY % BOOK_ENTITIES SYSTEM "cloudstack.ent">
+%BOOK_ENTITIES;
+]>
+<!-- Licensed to the Apache Software Foundation (ASF) under one
+  or more contributor license agreements.  See the NOTICE file
+  distributed with this work for additional information
+  regarding copyright ownership.  The ASF licenses this file
+  to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance
+  with the License.  You may obtain a copy of the License at
+  http://www.apache.org/licenses/LICENSE-2.0
+  Unless required by applicable law or agreed to in writing,
+  software distributed under the License is distributed on an
+  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  KIND, either express or implied.  See the License for the
+  specific language governing permissions and limitations
+  under the License.
+-->
+<book>
+  <xi:include href="Book_Info_Release_Notes_4.0.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
+  <chapter id="submitting-feedback">
+    <title>Submitting Feedback and Getting Help</title>
+    <para>The Apache CloudStack project has mailing lists for users and developers. These are the
+      official channels of communication for the project and are the best way to get answers about
+      using and contributing to CloudStack. It's a good idea to subscribe to the cloudstack-users
+      mailing list if you've deployed or are deploying CloudStack into production, and even for test
+      deployments.</para>
+    <para>The CloudStack developer's mailing list (cloudstack-dev) is for discussions about
+      CloudStack development, and is the best list for discussing possible bugs in CloudStack.
+      Anyone contributing to CloudStack should be on this mailing list.</para>
+    <para>You can also report bugs in CloudStack using the <ulink
+        url="https://issues.apache.org/jira/secure/CreateIssue!default.jspa">Apache Defect Tracking
+        System</ulink></para>
+    <para>To posts to the lists, you'll need to be subscribed. See the <ulink
+        url="http://incubator.apache.org/cloudstack/mailing-lists.html">CloudStack Web site</ulink>
+      for instructions.</para>
+  </chapter>
+  <chapter id="upgrade-instructions">
+    <title>Upgrade Instructions</title>
+    <section id="upgrade-from-3.0.2-to-4.0">
+      <title>Upgrade from 3.0.2 to 4.0.0-incubating</title>
+      <para>Perform the following to upgrade from version 3.0.2 to version 4.0.0-incubating. Note
+        that some of the steps here are only required if you're using a specific hypervisor. The
+        steps that are hypervisor-specific are called out with a note.</para>
+      <orderedlist>
+        <listitem>
+          <para>Ensure that you query your IP address usage records and process them or make a
+            backup. During the upgrade you will lose the old IP address usage records.</para>
+          <para>Starting in 3.0.2, the usage record format for IP addresses is the same as the rest
+            of the usage types. Instead of a single record with the assignment and release dates,
+            separate records are generated per aggregation period with start and end dates. After
+            upgrading, any existing IP address usage records in the old format will no longer be
+            available.</para>
+        </listitem>
+        <listitem>
+          <note>
+            <para>The following upgrade instructions apply only if you're using VMware hosts. If
+              you're not using VMware hosts, skip this step and move on to step 3: stopping all
+              usage servers.</para>
+          </note>
+          <para>In each zone that includes VMware hosts, you need to add a new system VM template. </para>
+          <orderedlist numeration="loweralpha">
+            <listitem>
+              <para>While running the existing 3.0.2 system, log in to the UI as root
+                administrator.</para>
+            </listitem>
+            <listitem>
+              <para>In the left navigation bar, click Templates.</para>
+            </listitem>
+            <listitem>
+              <para>In Select view, click Templates.</para>
+            </listitem>
+            <listitem>
+              <para>Click Register template.</para>
+              <para>The Register template dialog box is displayed.</para>
+            </listitem>
+            <listitem>
+              <para>In the Register template dialog box, specify the following values (do not change
+                these):</para>
+              <informaltable>
+                <tgroup cols="2" align="left" colsep="1" rowsep="1">
+                  <colspec colwidth="1*" colname="1" colnum="1"/>
+                  <colspec colwidth="2*" colname="2" colnum="2"/>
+                  <thead>
+                    <row>
+                      <entry><para>Field</para></entry>
+                      <entry><para>Value</para></entry>
+                    </row>
+                  </thead>
+                  <tbody>
+                    <row>
+                      <entry><para>Name</para></entry>
+                      <entry><para>systemvm-vmware-3.0.5</para></entry>
+                    </row>
+                    <row>
+                      <entry><para>Description</para></entry>
+                      <entry><para>systemvm-vmware-3.0.5</para></entry>
+                    </row>
+                    <row>
+                      <entry><para>URL</para></entry>
+                      <entry><para>http://download.cloud.com/templates/burbank/burbank-systemvm-08012012.ova</para></entry>
+                    </row>
+                    <row>
+                      <entry><para>Zone</para></entry>
+                      <entry><para>Choose the zone where this hypervisor is used</para></entry>
+                    </row>
+                    <row>
+                      <entry><para>Hypervisor</para></entry>
+                      <entry><para>VMware</para></entry>
+                    </row>
+                    <row>
+                      <entry><para>Format</para></entry>
+                      <entry><para>OVA</para></entry>
+                    </row>
+                    <row>
+                      <entry><para>OS Type</para></entry>
+                      <entry><para>Debian GNU/Linux 5.0 (32-bit)</para></entry>
+                    </row>
+                    <row>
+                      <entry><para>Extractable</para></entry>
+                      <entry><para>no</para></entry>
+                    </row>
+                    <row>
+                      <entry><para>Password Enabled</para></entry>
+                      <entry><para>no</para></entry>
+                    </row>
+                    <row>
+                      <entry><para>Public</para></entry>
+                      <entry><para>no</para></entry>
+                    </row>
+                    <row>
+                      <entry><para>Featured</para></entry>
+                      <entry><para>no</para></entry>
+                    </row>
+                  </tbody>
+                </tgroup>
+              </informaltable>
+            </listitem>
+            <listitem>
+              <para>Watch the screen to be sure that the template downloads successfully and enters
+                the READY state. Do not proceed until this is successful.</para>
+            </listitem>
+          </orderedlist>
+        </listitem>
+        <listitem>
+          <para>Stop all Usage Servers if running. Run this on all Usage Server hosts.</para>
+          <programlisting># service cloud-usage stop</programlisting>
+        </listitem>
+        <listitem>
+          <para>Stop the Management Servers. Run this on all Management Server hosts.</para>
+          <programlisting># service cloud-management stop</programlisting>
+        </listitem>
+        <listitem>
+          <para>On the MySQL master, take a backup of the MySQL databases. We recommend performing
+            this step even in test upgrades. If there is an issue, this will assist with
+            debugging.</para>
+          <para>In the following commands, it is assumed that you have set the root password on the
+            database, which is a CloudStack recommended best practice. Substitute your own MySQL
+            root password.</para>
+          <programlisting><prompt>#</prompt> <command>mysqldump</command> -u root -p<replaceable>mysql_password</replaceable> cloud > <filename>cloud-backup.dmp</filename>
+<prompt>#</prompt> <command>mysqldump</command> -u root -p<replaceable>mysql_password</replaceable> cloud_usage > <filename>cloud-usage-backup.dmp</filename></programlisting>
+        </listitem>
+        <listitem>
+          <para>Either build RPM/DEB packages as detailed in the Installation Guide, or use one of
+            the community provided yum/apt repositories to gain access to the &PRODUCT;
+            binaries.</para>
+        </listitem>
+        <listitem>
+          <para>After you have configured an appropriate yum or apt repository, you may execute the
+            one of the following commands as appropriate for your environment in order to upgrade
+            &PRODUCT;: <programlisting><prompt>#</prompt> <command>yum</command> update cloud-*</programlisting>
+            <programlisting><prompt>#</prompt> <command>apt-get</command> update
+<prompt>#</prompt> <command>apt-get</command> upgrade cloud-*</programlisting>
+          </para>
+          <note>
+            <para>If the upgrade output includes a message similar to the following, then some
+              custom content was found in your old components.xml, and you need to merge the two
+              files:</para>
+            <programlisting>warning: /etc/cloud/management/components.xml created as /etc/cloud/management/components.xml.rpmnew </programlisting>
+            <para>Instructions follow in the next step.</para>
+          </note>
+        </listitem>
+        <listitem>
+          <para>If you have made changes to your copy of
+              <filename>/etc/cloud/management/components.xml</filename> the changes will be
+            preserved in the upgrade. However, you need to do the following steps to place these
+            changes in a new version of the file which is compatible with version
+            4.0.0-incubating.</para>
+          <orderedlist numeration="loweralpha">
+            <listitem>
+              <para>Make a backup copy of <filename>/etc/cloud/management/components.xml</filename>.
+                For example:</para>
+              <programlisting># mv /etc/cloud/management/components.xml /etc/cloud/management/components.xml-backup</programlisting>
+            </listitem>
+            <listitem>
+              <para>Copy <filename>/etc/cloud/management/components.xml.rpmnew</filename> to create
+                a new <filename>/etc/cloud/management/components.xml</filename>:</para>
+              <programlisting># cp -ap /etc/cloud/management/components.xml.rpmnew /etc/cloud/management/components.xml</programlisting>
+            </listitem>
+            <listitem>
+              <para>Merge your changes from the backup file into the new
+                  <filename>components.xml</filename>.</para>
+              <programlisting># vi /etc/cloud/management/components.xml</programlisting>
+            </listitem>
+          </orderedlist>
+          <note>
+            <para>If you have more than one management server node, repeat the upgrade steps on each
+              node.</para>
+          </note>
+        </listitem>
+        <listitem>
+          <para>Start the first Management Server. Do not start any other Management Server nodes
+            yet.</para>
+          <programlisting># service cloud-management start</programlisting>
+          <para>Wait until the databases are upgraded. Ensure that the database upgrade is complete.
+            After confirmation, start the other Management Servers one at a time by running the same
+            command on each node.</para>
+          <note>
+            <para>Failing to restart the Management Server indicates a problem in the upgrade.
+              Having the Management Server restarted without any issues indicates that the upgrade
+              is successfully completed.</para>
+          </note>
+        </listitem>
+        <listitem>
+          <para>Start all Usage Servers (if they were running on your previous version). Perform
+            this on each Usage Server host.</para>
+          <para><command># service cloud-usage start</command></para>
+        </listitem>
+        <listitem>
+          <note>
+            <para>Additional steps are required for each KVM host. These steps will not affect
+              running guests in the cloud. These steps are required only for clouds using KVM as
+              hosts and only on the KVM hosts.</para>
+          </note>
+          <orderedlist numeration="loweralpha">
+            <listitem>
+              <para>Configure a yum or apt respository containing the &PRODUCT; packages as outlined
+                in the Installation Guide.</para>
+            </listitem>
+            <listitem>
+              <para>Stop the running agent.</para>
+              <para><command># service cloud-agent stop</command></para>
+            </listitem>
+            <listitem>
+              <para>Update the agent software with one of the following command sets as appropriate
+                for your environment.</para>
+              <para><command># yum update cloud-*</command></para>
+              <para><command># apt-get update</command></para>
+              <para><command># apt-get upgrade cloud-*</command></para>
+            </listitem>
+            <listitem>
+              <para>Start the agent.</para>
+              <programlisting># service cloud-agent start</programlisting>
+            </listitem>
+            <listitem>
+              <para>Edit <filename>/etc/cloud/agent/agent.properties</filename> to change the
+                resource parameter from
+                "com.cloud.agent.resource.computing.LibvirtComputingResource" to
+                "com.cloud.hypervisor.kvm.resource.LibvirtComputingResource".</para>
+            </listitem>
+            <listitem>
+              <para>Start the cloud agent and cloud management services.</para>
+            </listitem>
+            <listitem>
+              <para>When the Management Server is up and running, log in to the CloudStack UI and
+                restart the virtual router for proper functioning of all the features.</para>
+            </listitem>
+          </orderedlist>
+        </listitem>
+        <listitem>
+          <para>Log in to the CloudStack UI as administrator, and check the status of the hosts. All
+            hosts should come to Up state (except those that you know to be offline). You may need
+            to wait 20 or 30 minutes, depending on the number of hosts.</para>
+          <note>
+            <para>Troubleshooting: If login fails, clear your browser cache and reload the
+              page.</para>
+          </note>
+          <para/>
+          <para>Do not proceed to the next step until the hosts show in Up state.</para>
+        </listitem>
+        <listitem>
+          <para>If you are upgrading from 3.0.2, perform the following:</para>
+          <orderedlist numeration="loweralpha">
+            <listitem>
+              <para>Ensure that the admin port is set to 8096 by using the "integration.api.port"
+                global parameter.</para>
+              <para>This port is used by the cloud-sysvmadm script at the end of the upgrade
+                procedure. For information about how to set this parameter, see "Setting Global
+                Configuration Parameters" in the Installation Guide.</para>
+            </listitem>
+            <listitem>
+              <para>Restart the Management Server.</para>
+              <note>
+                <para>If you don't want the admin port to remain open, you can set it to null after
+                  the upgrade is done and restart the management server.</para>
+              </note>
+            </listitem>
+          </orderedlist>
+        </listitem>
+        <listitem>
+          <para>Run the <command>cloud-sysvmadm</command> script to stop, then start, all Secondary
+            Storage VMs, Console Proxy VMs, and virtual routers. Run the script once on each
+            management server. Substitute your own IP address of the MySQL instance, the MySQL user
+            to connect as, and the password to use for that user. In addition to those parameters,
+            provide the <command>-c</command> and <command>-r</command> arguments. For
+            example:</para>
+          <para><command># nohup cloud-sysvmadm -d 192.168.1.5 -u cloud -p password -c -r >
+              sysvm.log 2>&amp;1 &amp;</command></para>
+          <para><command># tail -f sysvm.log</command></para>
+          <para>This might take up to an hour or more to run, depending on the number of accounts in
+            the system.</para>
+        </listitem>
+        <listitem>
+          <para>If needed, upgrade all Citrix XenServer hypervisor hosts in your cloud to a version
+            supported by CloudStack 4.0.0-incubating. The supported versions are XenServer 5.6 SP2
+            and 6.0.2. Instructions for upgrade can be found in the CloudStack 4.0.0-incubating
+            Installation Guide.</para>
+        </listitem>
+        <listitem>
+          <para>Now apply the XenServer hotfix XS602E003 (and any other needed hotfixes) to
+            XenServer v6.0.2 hypervisor hosts.</para>
+          <orderedlist numeration="loweralpha">
+            <listitem>
+              <para>Disconnect the XenServer cluster from CloudStack.</para>
+              <para>In the left navigation bar of the CloudStack UI, select Infrastructure. Under
+                Clusters, click View All. Select the XenServer cluster and click Actions -
+                Unmanage.</para>
+              <para>This may fail if there are hosts not in one of the states Up, Down,
+                Disconnected, or Alert. You may need to fix that before unmanaging this
+                cluster.</para>
+              <para>Wait until the status of the cluster has reached Unmanaged. Use the CloudStack
+                UI to check on the status. When the cluster is in the unmanaged state, there is no
+                connection to the hosts in the cluster.</para>
+            </listitem>
+            <listitem>
+              <para>To clean up the VLAN, log in to one XenServer host and run:</para>
+              <para><command>/opt/xensource/bin/cloud-clean-vlan.sh</command></para>
+            </listitem>
+            <listitem>
+              <para>Now prepare the upgrade by running the following on one XenServer host:</para>
+              <para><command>/opt/xensource/bin/cloud-prepare-upgrade.sh</command></para>
+              <para>If you see a message like "can't eject CD", log in to the VM and unmount the CD,
+                then run this script again.</para>
+            </listitem>
+            <listitem>
+              <para>Upload the hotfix to the XenServer hosts. Always start with the Xen pool master,
+                then the slaves. Using your favorite file copy utility (e.g. WinSCP), copy the
+                hotfixes to the host. Place them in a temporary folder such as /tmp. </para>
+              <para>On the Xen pool master, upload the hotfix with this command:</para>
+              <para><command>xe patch-upload file-name=XS602E003.xsupdate</command></para>
+              <para>Make a note of the output from this command, which is a UUID for the hotfix
+                file. You'll need it in another step later.</para>
+              <note>
+                <para>(Optional) If you are applying other hotfixes as well, you can repeat the
+                  commands in this section with the appropriate hotfix number. For example,
+                  XS602E004.xsupdate.</para>
+              </note>
+            </listitem>
+            <listitem>
+              <para>Manually live migrate all VMs on this host to another host. First, get a list of
+                the VMs on this host:</para>
+              <para><command># xe vm-list</command></para>
+              <para>Then use this command to migrate each VM. Replace the example host name and VM
+                name with your own:</para>
+              <para><command># xe vm-migrate live=true host=<replaceable>host-name</replaceable>
+                    vm=<replaceable>VM-name</replaceable></command></para>
+              <note>
+                <title>Troubleshooting</title>
+                <para>If you see a message like "You attempted an operation on a VM which requires
+                  PV drivers to be installed but the drivers were not detected," run:</para>
+                <para><command>/opt/xensource/bin/make_migratable.sh
+                    b6cf79c8-02ee-050b-922f-49583d9f1a14</command>.</para>
+              </note>
+            </listitem>
+            <listitem>
+              <para>Apply the hotfix. First, get the UUID of this host:</para>
+              <programlisting># xe host-list</programlisting>
+              <para>Then use the following command to apply the hotfix. Replace the example host
+                UUID with the current host ID, and replace the hotfix UUID with the output from the
+                patch-upload command you ran on this machine earlier. You can also get the hotfix
+                UUID by running xe patch-list. </para>
+              <programlisting><command>xe</command> patch-apply host-uuid=<replaceable>host-uuid</replaceable> uuid=<replaceable>hotfix-uuid</replaceable></programlisting>
+            </listitem>
+            <listitem>
+              <para>Copy the following files from the CloudStack Management Server to the
+                host.</para>
+              <informaltable>
+                <tgroup cols="2" align="left" colsep="1" rowsep="1">
+                  <colspec colwidth="1*" colname="1" colnum="1"/>
+                  <colspec colwidth="2*" colname="2" colnum="2"/>
+                  <thead>
+                    <row>
+                      <entry><para>Copy from here...</para></entry>
+                      <entry><para>...to here</para></entry>
+                    </row>
+                  </thead>
+                  <tbody>
+                    <row>
+                      <entry><para>/usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/xenserver60/NFSSR.py</para></entry>
+                      <entry><para>/opt/xensource/sm/NFSSR.py</para></entry>
+                    </row>
+                    <row>
+                      <entry><para>/usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/setupxenserver.sh</para></entry>
+                      <entry><para>/opt/xensource/bin/setupxenserver.sh</para></entry>
+                    </row>
+                    <row>
+                      <entry><para>/usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/make_migratable.sh</para></entry>
+                      <entry><para>/opt/xensource/bin/make_migratable.sh</para></entry>
+                    </row>
+                  </tbody>
+                </tgroup>
+              </informaltable>
+            </listitem>
+            <listitem>
+              <para>(Only for hotfixes XS602E005 and XS602E007) You need to apply a new Cloud
+                Support Pack.</para>
+              <itemizedlist>
+                <listitem>
+                  <para>Download the CSP software onto the XenServer host from one of the following
+                    links:</para>
+                  <para>For hotfix XS602E005: <ulink
+                      url="http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E005/56710/xe-phase-2/xenserver-cloud-supp.tgz"
+                      >http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E005/56710/xe-phase-2/xenserver-cloud-supp.tgz</ulink></para>
+                  <para>For hotfix XS602E007: <ulink
+                      url="http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E007/57824/xe-phase-2/xenserver-cloud-supp.tgz"
+                      >http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E007/57824/xe-phase-2/xenserver-cloud-supp.tgz</ulink></para>
+                </listitem>
+                <listitem>
+                  <para>Extract the file:</para>
+                  <programlisting># tar xf xenserver-cloud-supp.tgz</programlisting>
+                </listitem>
+                <listitem>
+                  <para>Run the following script:</para>
+                  <programlisting># xe-install-supplemental-pack xenserver-cloud-supp.iso</programlisting>
+                </listitem>
+                <listitem>
+                  <para>If the XenServer host is part of a zone that uses basic networking, disable
+                    Open vSwitch (OVS):</para>
+                  <programlisting># xe-switch-network-backend  bridge</programlisting>
+                </listitem>
+              </itemizedlist>
+            </listitem>
+            <listitem>
+              <para>Reboot this XenServer host.</para>
+            </listitem>
+            <listitem>
+              <para>Run the following:</para>
+              <programlisting>/opt/xensource/bin/setupxenserver.sh</programlisting>
+              <note>
+                <para>If the message "mv: cannot stat `/etc/cron.daily/logrotate': No such file or
+                  directory" appears, you can safely ignore it.</para>
+              </note>
+            </listitem>
+            <listitem>
+              <para>Run the following:</para>
+              <programlisting>for pbd in `xe pbd-list currently-attached=false| grep ^uuid | awk '{print $NF}'`; do xe pbd-plug uuid=$pbd ; </programlisting>
+            </listitem>
+            <listitem>
+              <para>On each slave host in the Xen pool, repeat these steps, starting from "manually
+                live migrate VMs."</para>
+            </listitem>
+          </orderedlist>
+        </listitem>
+      </orderedlist>
+      <note>
+        <title>Troubleshooting Tip</title>
+        <para>If passwords which you know to be valid appear not to work after upgrade, or other UI
+          issues are seen, try clearing your browser cache and reloading the UI page.</para>
+      </note>
+    </section>
+    <section id="upgrade-from-2.2.x-to-4.0">
+      <title>Upgrade from 2.2.14 to 4.0.0-incubating</title>
+      <orderedlist>
+        <listitem>
+          <para>Ensure that you query your IPaddress usage records and process them; for example,
+            issue invoices for any usage that you have not yet billed users for.</para>
+          <para>Starting in 3.0.2, the usage record format for IP addresses is the same as the rest
+            of the usage types. Instead of a single record with the assignment and release dates,
+            separate records are generated per aggregation period with start and end dates. After
+            upgrading to 4.0.0-incubating, any existing IP address usage records in the old format
+            will no longer be available.</para>
+        </listitem>
+        <listitem>
+          <para>If you are using version 2.2.0 - 2.2.13, first upgrade to 2.2.14 by using the
+            instructions in the 2.2.14 Release Notes.</para>
+          <warning>
+            <title>KVM Hosts</title>
+            <para>If KVM hypervisor is used in your cloud, be sure you completed the step to insert
+              a valid username and password into the host_details table on each KVM node as
+              described in the 2.2.14 Release Notes. This step is critical, as the database will be
+              encrypted after the upgrade to 4.0.0-incubating.</para>
+          </warning>
+        </listitem>
+        <listitem>
+          <para>While running the 2.2.14 system, log in to the UI as root administrator.</para>
+        </listitem>
+        <listitem>
+          <para>Using the UI, add a new System VM template for each hypervisor type that is used in
+            your cloud. In each zone, add a system VM template for each hypervisor used in that
+            zone</para>
+          <orderedlist>
+            <listitem>
+              <para>In the left navigation bar, click Templates.</para>
+            </listitem>
+            <listitem>
+              <para>In Select view, click Templates.</para>
+            </listitem>
+            <listitem>
+              <para>Click Register template.</para>
+              <para>The Register template dialog box is displayed.</para>
+            </listitem>
+            <listitem>
+              <para>In the Register template dialog box, specify the following values depending on
+                the hypervisor type (do not change these):</para>
+              <informaltable>
+                <tgroup cols="2" align="left" colsep="1" rowsep="1">
+                  <colspec colwidth="1*" colname="1" colnum="1"/>
+                  <colspec colwidth="2*" colname="2" colnum="2"/>
+                  <thead>
+                    <row>
+                      <entry><para>Hypervisor</para></entry>
+                      <entry><para>Description</para></entry>
+                    </row>
+                  </thead>
+                  <tbody>
+                    <row>
+                      <entry><para>XenServer</para></entry>
+                      <entry><para>Name: systemvm-xenserver-3.0.0</para>
+                        <para>Description: systemvm-xenserver-3.0.0</para>
+                        <para>URL:
+                          http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2</para>
+                        <para>Zone: Choose the zone where this hypervisor is used</para>
+                        <para>Hypervisor: XenServer</para>
+                        <para>Format: VHD</para>
+                        <para>OS Type: Debian GNU/Linux 5.0 (32-bit)</para>
+                        <para>Extractable: no</para>
+                        <para>Password Enabled: no</para>
+                        <para>Public: no</para>
+                        <para>Featured: no</para>
+                      </entry>
+                    </row>
+                    <row>
+                      <entry><para>KVM</para></entry>
+                      <entry><para>Name: systemvm-kvm-3.0.0</para>
+                        <para>Description: systemvm-kvm-3.0.0</para>
+                        <para>URL:
+                          http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.bz2</para>
+                        <para>Zone: Choose the zone where this hypervisor is used</para>
+                        <para>Hypervisor: KVM</para>
+                        <para>Format: QCOW2</para>
+                        <para>OS Type: Debian GNU/Linux 5.0 (32-bit)</para>
+                        <para>Extractable: no</para>
+                        <para>Password Enabled: no</para>
+                        <para>Public: no</para>
+                        <para>Featured: no</para>
+                      </entry>
+                    </row>
+                    <row>
+                      <entry><para>VMware</para></entry>
+                      <entry><para>Name: systemvm-vmware-3.0.5</para>
+                        <para>Description: systemvm-vmware-3.0.5</para>
+                        <para>URL:
+                          http://download.cloud.com/templates/burbank/burbank-systemvm-08012012.ova</para>
+                        <para>Zone: Choose the zone where this hypervisor is used</para>
+                        <para>Hypervisor: VMware</para>
+                        <para>Format: OVA</para>
+                        <para>OS Type: Debian GNU/Linux 5.0 (32-bit)</para>
+                        <para>Extractable: no</para>
+                        <para>Password Enabled: no</para>
+                        <para>Public: no</para>
+                        <para>Featured: no</para>
+                      </entry>
+                    </row>
+                  </tbody>
+                </tgroup>
+              </informaltable>
+            </listitem>
+          </orderedlist>
+        </listitem>
+        <listitem>
+          <para>Watch the screen to be sure that the template downloads successfully and enters the
+            READY state. Do not proceed until this is successful</para>
+        </listitem>
+        <listitem>
+          <para><emphasis role="bold">WARNING</emphasis>: If you use more than one type of
+            hypervisor in your cloud, be sure you have repeated these steps to download the system
+            VM template for each hypervisor type. Otherwise, the upgrade will fail.</para>
+        </listitem>
+        <listitem>
+          <para>Stop all Usage Servers if running. Run this on all Usage Server hosts.</para>
+          <programlisting># service cloud-usage stop</programlisting>
+        </listitem>
+        <listitem>
+          <para>Stop the Management Servers. Run this on all Management Server hosts.</para>
+          <programlisting># service cloud-management stop</programlisting>
+        </listitem>
+        <listitem>
+          <para>On the MySQL master, take a backup of the MySQL databases. We recommend performing
+            this step even in test upgrades. If there is an issue, this will assist with
+            debugging.</para>
+          <para>In the following commands, it is assumed that you have set the root password on the
+            database, which is a CloudStack recommended best practice. Substitute your own MySQL
+            root password.</para>
+          <programlisting><prompt>#</prompt> <command>mysqldump</command> -u root -p<replaceable>mysql_password</replaceable> cloud > <filename>cloud-backup.dmp</filename>
+<prompt>#</prompt> <command>mysqldump</command> -u root -p<replaceable>mysql_password</replaceable> cloud_usage > <filename>cloud-usage-backup.dmp</filename>
+</programlisting>
+        </listitem>
+        <listitem>
+          <para> Either build RPM/DEB packages as detailed in the Installation Guide, or use one of
+            the community provided yum/apt repositories to gain access to the &PRODUCT; binaries.
+          </para>
+        </listitem>
+        <listitem>
+          <para> After you have configured an appropriate yum or apt repository, you may execute the
+            one of the following commands as appropriate for your environment in order to upgrade
+            &PRODUCT;: <programlisting><prompt>#</prompt> <command>yum</command> update cloud-*</programlisting>
+            <programlisting><prompt>#</prompt> <command>apt-get</command> update
+<prompt>#</prompt> <command>apt-get</command> upgrade cloud-*</programlisting>
+          </para>
+        </listitem>
+        <listitem>
+          <para>If you have made changes to your existing copy of the file components.xml in your
+            previous-version CloudStack installation, the changes will be preserved in the upgrade.
+            However, you need to do the following steps to place these changes in a new version of
+            the file which is compatible with version 4.0.0-incubating.</para>
+          <note>
+            <para>How will you know whether you need to do this? If the upgrade output in the
+              previous step included a message like the following, then some custom content was
+              found in your old components.xml, and you need to merge the two files:</para>
+          </note>
+          <programlisting>warning: /etc/cloud/management/components.xml created as /etc/cloud/management/components.xml.rpmnew </programlisting>
+          <orderedlist numeration="loweralpha">
+            <listitem>
+              <para>Make a backup copy of your
+                  <filename>/etc/cloud/management/components.xml</filename> file. For
+                example:</para>
+              <programlisting><prompt>#</prompt> <command>mv</command> <filename>/etc/cloud/management/components.xml</filename> <filename>/etc/cloud/management/components.xml-backup</filename></programlisting>
+            </listitem>
+            <listitem>
+              <para>Copy <filename>/etc/cloud/management/components.xml.rpmnew</filename> to create
+                a new <filename>/etc/cloud/management/components.xml</filename>:</para>
+              <programlisting><prompt>#</prompt> <command>cp</command> -ap <filename>/etc/cloud/management/components.xml.rpmnew</filename> <filename>/etc/cloud/management/components.xml</filename></programlisting>
+            </listitem>
+            <listitem>
+              <para>Merge your changes from the backup file into the new components.xml file.</para>
+              <programlisting><prompt>#</prompt> <command>vi</command> <filename>/etc/cloud/management/components.xml</filename>
+              </programlisting>
+            </listitem>
+          </orderedlist>
+        </listitem>
+        <listitem>
+          <para>If you have made changes to your existing copy of the
+              <filename>/etc/cloud/management/db.properties</filename> file in your previous-version
+            CloudStack installation, the changes will be preserved in the upgrade. However, you need
+            to do the following steps to place these changes in a new version of the file which is
+            compatible with version 4.0.0-incubating.</para>
+          <orderedlist>
+            <listitem>
+              <para>Make a backup copy of your file
+                  <filename>/etc/cloud/management/db.properties</filename>. For example:</para>
+              <programlisting><prompt>#</prompt> <command>mv</command> <filename>/etc/cloud/management/db.properties</filename> <filename>/etc/cloud/management/db.properties-backup</filename></programlisting>
+            </listitem>
+            <listitem>
+              <para>Copy <filename>/etc/cloud/management/db.properties.rpmnew</filename> to create a
+                new <filename>/etc/cloud/management/db.properties</filename>:</para>
+              <programlisting><prompt>#</prompt> <command>cp</command> -ap <filename>/etc/cloud/management/db.properties.rpmnew</filename> <filename>etc/cloud/management/db.properties</filename></programlisting>
+            </listitem>
+            <listitem>
+              <para>Merge your changes from the backup file into the new db.properties file.</para>
+              <programlisting><prompt>#</prompt> <command>vi</command> <filename>/etc/cloud/management/db.properties</filename></programlisting>
+            </listitem>
+          </orderedlist>
+        </listitem>
+        <listitem>
+          <para>On the management server node, run the following command. It is recommended that you
+            use the command-line flags to provide your own encryption keys. See Password and Key
+            Encryption in the Installation Guide.</para>
+          <programlisting><prompt>#</prompt> <command>cloud-setup-encryption</command> -e <replaceable>encryption_type</replaceable> -m <replaceable>management_server_key</replaceable> -k <replaceable>database_key</replaceable></programlisting>
+          <para>When used without arguments, as in the following example, the default encryption
+            type and keys will be used:</para>
+          <itemizedlist>
+            <listitem>
+              <para>(Optional) For encryption_type, use file or web to indicate the technique used
+                to pass in the database encryption password. Default: file.</para>
+            </listitem>
+            <listitem>
+              <para>(Optional) For management_server_key, substitute the default key that is used to
+                encrypt confidential parameters in the properties file. Default: password. It is
+                highly recommended that you replace this with a more secure value</para>
+            </listitem>
+            <listitem>
+              <para>(Optional) For database_key, substitute the default key that is used to encrypt
+                confidential parameters in the CloudStack database. Default: password. It is highly
+                recommended that you replace this with a more secure value.</para>
+            </listitem>
+          </itemizedlist>
+        </listitem>
+        <listitem>
+          <para>Repeat steps 10 - 14 on every management server node. If you provided your own
+            encryption key in step 14, use the same key on all other management servers.</para>
+        </listitem>
+        <listitem>
+          <para>Start the first Management Server. Do not start any other Management Server nodes
+            yet.</para>
+          <programlisting># service cloud-management start</programlisting>
+          <para>Wait until the databases are upgraded. Ensure that the database upgrade is complete.
+            You should see a message like "Complete! Done." After confirmation, start the other
+            Management Servers one at a time by running the same command on each node.</para>
+        </listitem>
+        <listitem>
+          <para>Start all Usage Servers (if they were running on your previous version). Perform
+            this on each Usage Server host.</para>
+          <programlisting># service cloud-usage start</programlisting>
+        </listitem>
+        <listitem>
+          <para>(KVM only) Additional steps are required for each KVM host. These steps will not
+            affect running guests in the cloud. These steps are required only for clouds using KVM
+            as hosts and only on the KVM hosts.</para>
+          <orderedlist numeration="loweralpha">
+            <listitem>
+              <para> Configure your CloudStack package repositories as outlined in the Installation
+                Guide </para>
+            </listitem>
+            <listitem>
+              <para>Stop the running agent.</para>
+              <programlisting># service cloud-agent stop</programlisting>
+            </listitem>
+            <listitem>
+              <para>Update the agent software with one of the following command sets as
+                appropriate.</para>
+              <programlisting><prompt>#</prompt> <command>yum</command> update cloud-*</programlisting>
+              <programlisting>
+                 <prompt>#</prompt> <command>apt-get</command> update
+<prompt>#</prompt> <command>apt-get</command> upgrade cloud-*
+              </programlisting>
+            </listitem>
+            <listitem>
+              <para>Start the agent.</para>
+              <programlisting># service cloud-agent start</programlisting>
+            </listitem>
+            <listitem>
+              <para> Copy the contents of the <filename>agent.properties</filename> file to the new
+                  <filename>agent.properties</filename> file by using the following command</para>
+              <programlisting><command>sed</command> -i 's/com.cloud.agent.resource.computing.LibvirtComputingResource/com.cloud.hypervisor.kvm.resource.LibvirtComputingResource/g' <filename>/etc/cloud/agent/agent.properties</filename></programlisting>
+            </listitem>
+            <listitem>
+              <para>Start the cloud agent and cloud management services.</para>
+            </listitem>
+            <listitem>
+              <para>When the Management Server is up and running, log in to the CloudStack UI and
+                restart the virtual router for proper functioning of all the features.</para>
+            </listitem>
+          </orderedlist>
+        </listitem>
+        <listitem>
+          <para>Log in to the CloudStack UI as admin, and check the status of the hosts. All hosts
+            should come to Up state (except those that you know to be offline). You may need to wait
+            20 or 30 minutes, depending on the number of hosts.</para>
+          <para>Do not proceed to the next step until the hosts show in the Up state. If the hosts
+            do not come to the Up state, contact support.</para>
+        </listitem>
+        <listitem>
+          <para>Run the following script to stop, then start, all Secondary Storage VMs, Console
+            Proxy VMs, and virtual routers.</para>
+          <orderedlist numeration="loweralpha">
+            <listitem>
+              <para>Run the command once on one management server. Substitute your own IP address of
+                the MySQL instance, the MySQL user to connect as, and the password to use for that
+                user. In addition to those parameters, provide the "-c" and "-r" arguments. For
+                example:</para>
+              <programlisting><prompt>#</prompt> <command>nohup cloud-sysvmadm</command> -d <replaceable>192.168.1.5</replaceable> -u cloud -p <replaceable>password</replaceable> -c -r > sysvm.log 2>&amp;1 &amp;
+<prompt>#</prompt> <command>tail</command> -f <filename>sysvm.log</filename></programlisting>
+              <para>This might take up to an hour or more to run, depending on the number of
+                accounts in the system.</para>
+            </listitem>
+            <listitem>
+              <para>After the script terminates, check the log to verify correct execution:</para>
+              <programlisting><prompt>#</prompt> <command>tail</command> -f <filename>sysvm.log</filename></programlisting>
+              <para>The content should be like the following:</para>
+              <programlisting>
+Stopping and starting 1 secondary storage vm(s)...
+Done stopping and starting secondary storage vm(s)
+Stopping and starting 1 console proxy vm(s)...
+Done stopping and starting console proxy vm(s).
+Stopping and starting 4 running routing vm(s)...
+Done restarting router(s).
+						</programlisting>
+            </listitem>
+          </orderedlist>
+        </listitem>
+        <listitem>
+          <para>If you would like additional confirmation that the new system VM templates were
+            correctly applied when these system VMs were rebooted, SSH into the System VM and check
+            the version.</para>
+          <para>Use one of the following techniques, depending on the hypervisor.</para>
+          <formalpara>
+            <title>XenServer or KVM:</title>
+            <para>SSH in by using the link local IP address of the system VM. For example, in the
+              command below, substitute your own path to the private key used to log in to the
+              system VM and your own link local IP.</para>
+          </formalpara>
+          <para>Run the following commands on the XenServer or KVM host on which the system VM is
+            present:</para>
+          <programlisting><prompt>#</prompt> <command>ssh</command> -i <replaceable>private-key-path</replaceable> <replaceable>link-local-ip</replaceable> -p 3922
+# cat /etc/cloudstack-release</programlisting>
+          <para>The output should be like the following:</para>
+          <programlisting>Cloudstack Release 4.0.0-incubating Mon Oct 9 15:10:04 PST 2012</programlisting>
+          <formalpara>
+            <title>ESXi</title>
+            <para>SSH in using the private IP address of the system VM. For example, in the command
+              below, substitute your own path to the private key used to log in to the system VM and
+              your own private IP.</para>
+          </formalpara>
+          <para>Run the following commands on the Management Server:</para>
+          <programlisting><prompt>#</prompt> <command>ssh</command> -i <replaceable>private-key-path</replaceable> <replaceable>private-ip</replaceable> -p 3922
+<prompt>#</prompt> <command>cat</command> <filename>/etc/cloudstack-release</filename>
+</programlisting>
+          <para>The output should be like the following:</para>
+          <programlisting>Cloudstack Release 4.0.0-incubating Mon Oct 9 15:10:04 PST 2012</programlisting>
+        </listitem>
+        <listitem>
+          <para>If needed, upgrade all Citrix XenServer hypervisor hosts in your cloud to a version
+            supported by CloudStack 4.0.0-incubating. The supported versions are XenServer 5.6 SP2
+            and 6.0.2. Instructions for upgrade can be found in the CloudStack 4.0.0-incubating
+            Installation Guide.</para>
+        </listitem>
+        <listitem>
+          <para>Apply the XenServer hotfix XS602E003 (and any other needed hotfixes) to XenServer
+            v6.0.2 hypervisor hosts.</para>
+          <orderedlist numeration="loweralpha">
+            <listitem>
+              <para>Disconnect the XenServer cluster from CloudStack.</para>
+              <para>In the left navigation bar of the CloudStack UI, select Infrastructure. Under
+                Clusters, click View All. Select the XenServer cluster and click Actions -
+                Unmanage.</para>
+              <para>This may fail if there are hosts not in one of the states Up, Down,
+                Disconnected, or Alert. You may need to fix that before unmanaging this
+                cluster.</para>
+              <para>Wait until the status of the cluster has reached Unmanaged. Use the CloudStack
+                UI to check on the status. When the cluster is in the unmanaged state, there is no
+                connection to the hosts in the cluster.</para>
+            </listitem>
+            <listitem>
+              <para>To clean up the VLAN, log in to one XenServer host and run:</para>
+              <programlisting>/opt/xensource/bin/cloud-clean-vlan.sh</programlisting>
+            </listitem>
+            <listitem>
+              <para>Prepare the upgrade by running the following on one XenServer host:</para>
+              <programlisting>/opt/xensource/bin/cloud-prepare-upgrade.sh</programlisting>
+              <para>If you see a message like "can't eject CD", log in to the VM and umount the CD,
+                then run this script again.</para>
+            </listitem>
+            <listitem>
+              <para>Upload the hotfix to the XenServer hosts. Always start with the Xen pool master,
+                then the slaves. Using your favorite file copy utility (e.g. WinSCP), copy the
+                hotfixes to the host. Place them in a temporary folder such as /root or /tmp. </para>
+              <para>On the Xen pool master, upload the hotfix with this command:</para>
+              <programlisting>xe patch-upload file-name=XS602E003.xsupdate</programlisting>
+              <para>Make a note of the output from this command, which is a UUID for the hotfix
+                file. You'll need it in another step later.</para>
+              <note>
+                <para>(Optional) If you are applying other hotfixes as well, you can repeat the
+                  commands in this section with the appropriate hotfix number. For example,
+                  XS602E004.xsupdate.</para>
+              </note>
+            </listitem>
+            <listitem>
+              <para>Manually live migrate all VMs on this host to another host. First, get a list of
+                the VMs on this host:</para>
+              <programlisting># xe vm-list</programlisting>
+              <para>Then use this command to migrate each VM. Replace the example host name and VM
+                name with your own:</para>
+              <programlisting><prompt>#</prompt> <command>xe</command> vm-migrate live=true host=<replaceable>host-name</replaceable> vm=<replaceable>VM-name</replaceable></programlisting>
+              <note>
+                <title>Troubleshooting</title>
+                <para>If you see a message like "You attempted an operation on a VM which requires
+                  PV drivers to be installed but the drivers were not detected," run:</para>
+                <para><command>/opt/xensource/bin/make_migratable.sh
+                    b6cf79c8-02ee-050b-922f-49583d9f1a14</command>.</para>
+              </note>
+            </listitem>
+            <listitem>
+              <para>Apply the hotfix. First, get the UUID of this host:</para>
+              <para><command># xe host-list</command></para>
+              <para>Then use the following command to apply the hotfix. Replace the example host
+                UUID with the current host ID, and replace the hotfix UUID with the output from the
+                patch-upload command you ran on this machine earlier. You can also get the hotfix
+                UUID by running xe patch-list. </para>
+              <para><command>xe patch-apply host-uuid=<replaceable>host-uuid</replaceable>
+                    uuid=<replaceable>hotfix-uuid</replaceable></command></para>
+            </listitem>
+            <listitem>
+              <para>Copy the following files from the CloudStack Management Server to the
+                host.</para>
+              <informaltable>
+                <tgroup cols="2" align="left" colsep="1" rowsep="1">
+                  <colspec colwidth="1*" colname="1" colnum="1"/>
+                  <colspec colwidth="2*" colname="2" colnum="2"/>
+                  <thead>
+                    <row>
+                      <entry><para>Copy from here...</para></entry>
+                      <entry><para>...to here</para></entry>
+                    </row>
+                  </thead>
+                  <tbody>
+                    <row>
+                      <entry><para><filename>/usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/xenserver60/NFSSR.py</filename></para></entry>
+                      <entry><para><filename>/opt/xensource/sm/NFSSR.py</filename></para></entry>
+                    </row>
+                    <row>
+                      <entry><para><filename>/usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/setupxenserver.sh</filename></para></entry>
+                      <entry><para><filename>/opt/xensource/bin/setupxenserver.sh</filename></para></entry>
+                    </row>
+                    <row>
+                      <entry><para><filename>/usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/make_migratable.sh</filename></para></entry>
+                      <entry><para><filename>/opt/xensource/bin/make_migratable.sh</filename></para></entry>
+                    </row>
+                  </tbody>
+                </tgroup>
+              </informaltable>
+            </listitem>
+            <listitem>
+              <para>(Only for hotfixes XS602E005 and XS602E007) You need to apply a new Cloud
+                Support Pack.</para>
+              <itemizedlist>
+                <listitem>
+                  <para>Download the CSP software onto the XenServer host from one of the following
+                    links:</para>
+                  <para>For hotfix XS602E005: <ulink
+                      url="http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E005/56710/xe-phase-2/xenserver-cloud-supp.tgz"
+                      >http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E005/56710/xe-phase-2/xenserver-cloud-supp.tgz</ulink></para>
+                  <para>For hotfix XS602E007: <ulink
+                      url="http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E007/57824/xe-phase-2/xenserver-cloud-supp.tgz"
+                      >http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E007/57824/xe-phase-2/xenserver-cloud-supp.tgz</ulink></para>
+                </listitem>
+                <listitem>
+                  <para>Extract the file:</para>
+                  <para><command># tar xf xenserver-cloud-supp.tgz</command></para>
+                </listitem>
+                <listitem>
+                  <para>Run the following script:</para>
+                  <para><command># xe-install-supplemental-pack
+                    xenserver-cloud-supp.iso</command></para>
+                </listitem>
+                <listitem>
+                  <para>If the XenServer host is part of a zone that uses basic networking, disable
+                    Open vSwitch (OVS):</para>
+                  <para><command># xe-switch-network-backend bridge</command></para>
+                </listitem>
+              </itemizedlist>
+            </listitem>
+            <listitem>
+              <para>Reboot this XenServer host.</para>
+            </listitem>
+            <listitem>
+              <para>Run the following:</para>
+              <para><command>/opt/xensource/bin/setupxenserver.sh</command></para>
+              <note>
+                <para>If the message "mv: cannot stat `/etc/cron.daily/logrotate': No such file or
+                  directory" appears, you can safely ignore it.</para>
+              </note>
+            </listitem>
+            <listitem>
+              <para>Run the following:</para>
+              <para><command>for pbd in `xe pbd-list currently-attached=false| grep ^uuid | awk
+                  '{print $NF}'`; do xe pbd-plug uuid=$pbd ; </command>
+              </para>
+            </listitem>
+            <listitem>
+              <para>On each slave host in the Xen pool, repeat these steps, starting from "manually
+                live migrate VMs."</para>
+            </listitem>
+          </orderedlist>
+        </listitem>
+      </orderedlist>
+    </section>
+  </chapter>
+  <chapter id="version-4.0">
+    <title>Version 4.0.0-incubating</title>
+    <section id="what-new-in-4.0">
+      <title>What’s New in 4.0.0-incubating</title>
+      <para>Apache CloudStack 4.0.0-incubating includes the following new features:</para>
+      <section id="inter-vlan-routing">
+        <title>Inter-VLAN Routing</title>
+        <para>Inter-VLAN Routing is the capability to route network traffic between VLANs. This
+          feature enables you to set up Virtual Private Clouds (VPC) that can hold multi-tier
+          applications. These tiers are deployed on different VLANs that can communicate with each
+          other. You can provision VLANs to the tiers your create, and VMs can be deployed on
+          different tiers, such as Web, Application, or Database. The VLANs are connected to a
+          virtual router, which facilitates communication between the VMs. In effect, you can
+          segment VMs by means of VLANs into different networks that can host multi-tier
+          applications. Such segmentation by means of VLANs logically separate application VMs for
+          higher security and lower broadcasts, while remaining physically connected to the same
+          device.</para>
+        <para>This feature is supported on XenServer and VMware hypervisors.</para>
+      </section>
+      <section id="site-to-site-vpn">
+        <title>Site-to-Site VPN</title>
+        <para>A Site-to-Site VPN connection helps you establish a secure connection from an
+          enterprise datacenter to the cloud infrastructure. This allows users to access the guest
+          VMs by establishing a VPN connection to the virtual router of the account from a device in
+          the datacenter of the enterprise. Having this facility eliminates the need to establish
+          VPN connections to individual VMs.</para>
+        <para>The supported endpoints on the remote datacenters are:</para>
+        <itemizedlist>
+          <listitem>
+            <para>Cisco ISR with IOS 12.4 or later</para>
+          </listitem>
+          <listitem>
+            <para>Juniper J-Series routers with JunOS 9.5 or later</para>
+          </listitem>
+        </itemizedlist>
+      </section>
+      <section id="local-storage-support">
+        <title>Local Storage Support for Data Volumes</title>
+        <para>You can now create data volumes on local storage. The data volume is placed on the
+          same XenServer host as the VM instance that is attached to the data volume. These local
+          data volumes can be attached to virtual machines, detached, re-attached, and deleted just
+          as with the other types of data volume. In earlier releases of CloudStack, only the root
+          disk could be placed in local storage.</para>
+        <para>Local storage is ideal for scenarios where persistence of data volumes and HA is not
+          required. Some of the benefits include reduced disk I/O latency and cost reduction from
+          using inexpensive local disks.</para>
+        <para>In order for local volumes to be used, the feature must be enabled for the
+          zone.</para>
+        <para>You can create a data disk offering for local storage. When a user creates a new VM,
+          they can select this disk offering in order to cause the data disk volume to be placed in
+          local storage.</para>
+        <para>You can not migrate a VM that has a volume in local storage to a different host, nor
+          migrate the volume itself away to a different host. If you want to put a host into
+          maintenance mode, you must first stop any VMs with local data volumes on that host.</para>
+        <para>Local storage support for volumes is available for XenServer, KVM, and VMware
+          hypervisors.</para>
+      </section>
+      <section id="tags">
+        <title>Tags</title>
+        <para>A tag is a key-value pair that stores metadata about a resource in the cloud. Tags are
+          useful for categorizing resources. For example, you can tag a user VM with a value that
+          indicates the user's city of residence. In this case, the key would be "city" and the
+          value might be "Toronto" or "Tokyo." You can then request CloudStack to find all resources
+          that have a given tag; for example, VMs for users in a given city.</para>
+        <para>You can tag a user virtual machine, volume, snapshot, guest network, template, ISO,
+          firewall rule, port forwarding rule, public IP address, security group, load balancer
+          rule, project, VPC, network ACL, or static route. You can not tag a remote access
+          VPN.</para>
+        <para>You can work with tags through the UI or through the new API commands createTags,
+          deleteTags, and listTags. You can define multiple tags for each resource. There is no
+          limit on the number of tags you can define. Each tag can be up to 255 characters long.
+          Users can define tags on the resources they own, and administrators can define tags on any
+          resources in the cloud.</para>
+        <para>A new optional input parameter, "tags," has been added to many of the list* API
+          commands. The following example shows how to use this new parameter to find all the
+          volumes having tag region=canada OR tag city=Toronto:</para>
+        <programlisting>command=listVolumes
+&amp;listAll=true
+&amp;tags[0].key=region
+&amp;tags[0].value=canada
+&amp;tags[1].key=city
+&amp;tags[1].value=Toronto</programlisting>
+        <para>The following API commands have the new "tags" input parameter:</para>
+        <itemizedlist>
+          <listitem>
+            <para>listVirtualMachines</para>
+          </listitem>
+          <listitem>
+            <para>listVolumes</para>
+          </listitem>
+          <listitem>
+            <para>listSnapshots</para>
+          </listitem>
+          <listitem>
+            <para>listNetworks</para>
+          </listitem>
+          <listitem>
+            <para>listTemplates</para>
+          </listitem>
+          <listitem>
+            <para>listIsos</para>
+          </listitem>
+          <listitem>
+            <para>listFirewallRules</para>
+          </listitem>
+          <listitem>
+            <para>listPortForwardingRules</para>
+          </listitem>
+          <listitem>
+            <para>listPublicIpAddresses</para>
+          </listitem>
+          <listitem>
+            <para>listSecurityGroups</para>
+          </listitem>
+          <listitem>
+            <para>listLoadBalancerRules</para>
+          </listitem>
+          <listitem>
+            <para>listProjects</para>
+          </listitem>
+          <listitem>
+            <para>listVPCs</para>
+          </listitem>
+          <listitem>
+            <para>listNetworkACLs</para>
+          </listitem>
+          <listitem>
+            <para>listStaticRoutes</para>
+          </listitem>
+        </itemizedlist>
+      </section>
+      <section id="aws-tags">
+        <title>AWS API Changes for Tags</title>
+        <para>Some changes have been made to the Amazon Web Services API compatibility support in
+          order to accommodate the new tagging feature.</para>
+        <para>New APIs:</para>
+        <informaltable>
+          <tgroup cols="2" align="left" colsep="1" rowsep="1">
+            <colspec colwidth="1.0*" colname="1" colnum="1"/>
+            <colspec colwidth="4.3*" colname="2" colnum="2"/>
+            <thead>
+              <row>
+                <entry>
+                  <para>New API</para>
+                </entry>
+                <entry>
+                  <para>Description</para>
+                </entry>
+              </row>
+            </thead>
+            <tbody>
+              <row>
+                <entry>
+                  <para>ec2-create-tags </para>
+                </entry>
+                <entry>
+                  <para>Add tags to one or more resources.</para>
+                </entry>
+              </row>
+              <row>
+                <entry>
+                  <para>ec2-delete-tags</para>
+                </entry>
+                <entry>
+                  <para>Remove tags from one or more resources.</para>
+                </entry>
+              </row>
+              <row>
+                <entry>ec2-describe-tags</entry>
+                <entry>
+                  <para>Show currently defined tags.</para>
+                </entry>
+              </row>
+            </tbody>
+          </tgroup>
+        </informaltable>
+        <para>Changed APIs:</para>
+        <informaltable>
+          <tgroup cols="2" align="left" colsep="1" rowsep="1">
+            <colspec colwidth="1" colname="1" colnum="1"/>
+            <colspec colwidth="2*" colname="2" colnum="2"/>
+            <thead>
+              <row>
+                <entry>
+                  <para>Changed API</para>
+                </entry>
+                <entry>
+                  <para>Description</para>
+                </entry>
+              </row>
+            </thead>
+            <tbody>
+              <row>
+                <entry>ec2-describe-images</entry>
+                <entry>
+                  <para>Output now shows tags defined for each image.</para>
+                </entry>
+              </row>
+              <row>
+                <entry>
+                  <para>ec2-describe-instances </para>
+                </entry>
+                <entry>
+                  <para>Output now shows tags defined for each image.</para>
+                  <para>The following filters can now be passed in to limit the output result set:
+                    tag-key, tag-value and tag:key</para>
+                </entry>
+              </row>
+              <row>
+                <entry>
+                  <para>ec2-describe-snapshots</para>
+                </entry>
+                <entry>
+                  <para>Output now shows tags defined for each image.</para>
+                  <para>The following filters can now be passed in to limit the output result set:
+                    tag-key, tag-value and tag:key</para>
+                </entry>
+              </row>
+              <row>
+                <entry>ec2-describe-volumes</entry>
+                <entry>
+                  <para>Output now shows tags defined for each image.</para>
+                  <para>The following filters can now be passed in to limit the output result set:
+                    tag-key, tag-value and tag:key</para>
+                </entry>
+              </row>
+            </tbody>
+          </tgroup>
+        </informaltable>
+      </section>
+      <section id="secure-console-access-on-xenserver">
+        <title>Secure Console Access on XenServer</title>
+        <para>With the addition of Secure Console feature, users can now securely access the VM
+          consoles on the XenServer hypervisor. You can either SSH or use the View Console option in
+          the Management Server to securely connect to the VMs on the XenServer host. The Management
+          Server uses the xapi API to stream the VM consoles. However, there is no change in the way
+          you can access the console of a VM. This feature is supported on XenServer 5.6 and 6.0
+          versions.</para>
+      </section>
+      <section id="release-note-stopped-vm">
+        <title>Stopped VM</title>
+        <para>This release supports creating VMs without starting them on the backend. You can
+          determine whether the VM needs to be started as part of the VM deployment. A VM can be
+          deployed in two ways: create and start a VM (the default method); create a VM and leave it
+          in the stopped state.</para>
+        <para>A new request parameter, startVM, is introduced in the deployVm API to support the
+          stopped VM feature. The possible values are:</para>
+        <itemizedlist>
+          <listitem>
+            <para>true - The VM starts as a part of the VM deployment</para>
+          </listitem>
+          <listitem>
+            <para>false - The VM is left in stopped state at the end of the VM deployment</para>
+          </listitem>
+        </itemizedlist>
+      </section>
+      <section id="release-note-upload-existing-volume-to-vm">
+        <title>Uploading an Existing Volume to a Virtual Machine</title>
+        <para>Existing data can now be made accessible to a virtual machine. This is called
+          uploading a volume to the VM. For example, this is useful to upload data from a local file
+          system and attach it to a VM. Root administrators, domain administrators, and end users
+          can all upload existing volumes to VMs. The upload is performed by using HTTP. The
+          uploaded volume is placed in the zone's secondary storage.</para>
+        <para>This functionality is supported for the following hypervisors:</para>
+        <itemizedlist>
+          <listitem>
+            <para>Hypervisor : Disk Image Format</para>
+          </listitem>
+          <listitem>
+            <para>XenServer : VHD</para>
+          </listitem>
+          <listitem>
+            <para>VMware : OVA</para>
+          </listitem>
+          <listitem>
+            <para>KVM : QCOW2</para>
+          </listitem>
+          <!--          <listitem>
+            <para>OVM : RAW</para>
+        </listitem> -->
+        </itemizedlist>
+      </section>
+      <section id="dedicated-ha-hosts">
+        <title>Dedicated High-Availability Hosts</title>
+        <para>One or more hosts can now be designated for use only by high-availability (HA) enabled
+          VMs that are restarted due to a host failure. Setting up a pool of such dedicated HA hosts
+          as the recovery destination for all HA-enabled VMs make it easier to determine which VMs
+          are restarted as part of the high-availability function. You can designate a host as a
+          dedicated-HA restart node only if the Dedicated HA Hosts feature is enabled by setting the
+          appropriate global configuration parameter.</para>
+      </section>
+      <section id="support-for-aws-api">
+        <title>Support for Amazon Web Services API</title>
+        <para>This release supports Amazon Web Services APIs, including Elastic Compute Cloud (EC2)
+          API. Fidelity with the EC2 API and the installation experience for this functionality are
+          both enhanced. In prior releases, users were required to install a separate component
+          called CloudBridge, in addition to installing the Management Server. For new installations
+          of CloudStack 4.0.0-incubating, this software is installed automatically along with
+          CloudStack and runs in a more closely integrated fashion. The feature is disabled by
+          default, but can be easily enabled by setting the appropriate global configuration
+          parameter and performing a few setup steps.</para>
+      </section>
+      <section id="nicira-nvp-plugin">
+        <title>The Nicira NVP Plugin</title>
+        <para>The Nicira NVP plug-in allows CloudStack to use the Nicira solution for virtualized
+          network as a provider for CloudStack networks and services. In CloudStack 4.0.0-incubating
+          this plug-in supports the Connectivity service. This service is responsible for creating
+          Layer 2 networks supporting the networks created by guests. When a tenant creates a new
+          network, instead of a traditional VLAN, a logical network will be created by sending the
+          appropriate calls to the Nicira NVP Controller. The plug-in has been tested with Nicira
+          NVP versions 2.1.0, 2.2.0 and 2.2.1.</para>
+      </section>
+      <section id="castor-support">
+        <title>Support for CAStor Cluster</title>
+        <para>CloudStack 4.0.0-incubating supports using a CAStor cluster as the back-end storage
+          system for a CloudStack S3 front-end. The CAStor back-end storage for CloudStack extends
+          the existing storage classes and allows the storage configuration attribute to point to a
+          CAStor cluster. This feature makes use of the CloudStack server's local disk to spool
+          files before writing them to CAStor when handling the PUT operations. However, a file must
+          be successfully written into the CAStor cluster prior to the return of a success code to
+          the S3 client to ensure that the transaction outcome is correctly reported.</para>
+        <para>The S3 multipart file upload is not supported in this release. You are prompted with
+          proper error message if a multipart upload is attempted.</para>
+      </section>
+      <section id="clvm-support-kvm">
+        <title>Clustered Logical Volume Manager Support for KVM</title>
+        <para>This release adds Clustered Logical Volume Manager (CLVM) storage support for KVM
+          hosts. With this support, you can use CLVM as primary storage.</para>
+        <para>The CLVM support for KVM allows root and data disks (primary storage) to reside on
+          Linux logical volumes. The administrators are required to configure CLVM on the KVM hosts
+          independent of CloudStack. When the volume groups are available, an administrator can
+          simply add primary storage of type CLVM, providing the volume group name. Then CloudStack
+          creates and manages logical volumes as needed.</para>
+        <para>CLVM also supports Snapshots. CloudStack creates an LVM snapshot, copy the applicable
+          logical volume to the secondary storage in the qcow2 format, and then delete the LVM
+          snapshot. </para>
+      </section>
+      <section id="rbd-support-kvm">
+        <title>Rados Block Device Support for KVM</title>
+        <para>You can now use Rados Block Device (RBD) to run instances on Apache CloudStack
+          4.0.0-incubating. This can be done by adding a RBD pool as primary storage. Before using
+          RBD, ensure that Qemu is compiled with RBD enabled, and the libvirt version is at least
+          0.10 with RBD enabled on the KVM host </para>
+        <para>Create a disk offering for RBD so that you can ensure that StoragePoolAllocator
+          chooses the RBD pool to deploy instances.</para>
+      </section>
+    </section>
+    <section id="issues-fixed-4.0">
+      <title>Issues Fixed in 4.0.0-incubating</title>
+      <para>Many bugs include a defect number that reflects the bug number that was held in the bug
+        tracker run by Citrix (bugs.cloudstack.org). The Apache CloudStack project now uses <ulink
+          url="https://issues.apache.org/jira/browse/CLOUDSTACK">Jira</ulink> to manage its bugs, so
+        some of the bugs that are referenced here may not be available to view. However, we are
+        still including them for completeness.</para>
+      <informaltable>
+        <tgroup cols="2" align="left" colsep="1" rowsep="1">
+          <colspec colwidth="1*" colname="1" colnum="1"/>
+          <colspec colwidth="2*" colname="2" colnum="2"/>
+          <thead>
+            <row>
+              <entry>
+                <para>Defect</para>
+              </entry>
+              <entry>
+                <para>Description</para>
+              </entry>
+            </row>
+          </thead>
+          <tbody>
+            <row>
+              <entry><para>Many</para></entry>
+              <entry><para>vSphere 5.0 now has GA support. Formerly only Beta support was
+                  provided.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-16135</para></entry>
+              <entry><para>Creating volumes after upgrading from snapshot taken in 2.2.14 no longer
+                  deletes the snapshot physically from the secondary storage.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-16122</para></entry>
+              <entry><para>In a site-to-site VPN setup, alerts are generated when the VPC virtual
+                  router is rebooted with multiple vpn connections.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-16022</para></entry>
+              <entry><para>If host connection fails due to a database error, host now disconnects
+                  and the Managerment Server id is removed.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-16011</para></entry>
+              <entry><para>Name of network offering is no longer truncated due to too-narrow field
+                  width in Add Guest Network dialog box. </para>
+              </entry>
+            </row>
+            <row>
+              <entry><para>CS-15978</para></entry>
+              <entry><para>When the virtual router and its host go down, the high availability
+                  mechanism now works for the virtual router.</para>
+              </entry>
+            </row>
+            <row>
+              <entry><para>CS-15921</para></entry>
+              <entry><para>The 2.2.x security group script now accounts for the VMs created in the
+                  version 2.1 timeframe.</para>
+              </entry>
+            </row>
+            <row>
+              <entry><para>CS-15919</para></entry>
+              <entry><para>A level parameter is added to the listVolumes command; therefore queries
+                  return the response more quickly.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15904</para></entry>
+              <entry><para>Upgrade from version 2.2.14 to CloudStack-3.0.5-0.2944-rhel5 works as
+                  expected. The upgrade script,
+                  /usr/share/cloud/setup/db/schema-2214to30-cleanup.sql, works as
+                expected.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15879</para></entry>
+              <entry><para>The database upgrade from version 3.0.4 to 3.0.5 works as
+                  expected.</para>
+              </entry>
+            </row>
+            <row>
+              <entry><para>CS-15807</para></entry>
+              <entry><para>Network label for OVM now available in UI.</para>
+              </entry>
+            </row>
+            <row>
+              <entry><para>CS-15779</para></entry>
+              <entry><para>When the thumbnail is requested, the console session will not be
+                  terminated.</para>
+              </entry>
+            </row>
+            <row>
+              <entry><para>CS-15778</para></entry>
+              <entry><para>Fetching a VM thumbnail now gets a thumbnail of appropriate visual
+                  dimensions.</para>
+              </entry>
+            </row>
+            <row>
+              <entry><para>CS-15734</para></entry>
+              <entry><para>KVM Snapshots no longer shows incorrect disk usage.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15733</para></entry>
+              <entry><para>The domainId parameter for the listNetworks command now lists the
+                  resources belonging to the domain specified.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15676</para></entry>
+              <entry><para>Stopping the router no longer fails with the null pointer
+                  exception.</para>
+              </entry>
+            </row>
+            <row>
+              <entry><para>CS-15648</para></entry>
+              <entry><para>If creating a volume from a snapshot fails, the error is reported on the
+                  UI but the volume is stuck in the creating state.</para>
+              </entry>
+            </row>
+            <row>
+              <entry><para>CS-15646</para></entry>
+              <entry><para>createFirewallRule API no longer causes null pointer
+                exception.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15628</para></entry>
+              <entry><para>In a KVM host, the high availability mechanism no longer takes a long
+                  time to migrate VMs to another KVM host if there are multiple storage
+                  pools.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15627</para></entry>
+              <entry><para>Metadata instance-id and vm-id for existing VMs stays the same after
+                  upgrade.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15621</para></entry>
+              <entry><para>Solved difficulty with allocating disk volumes when running multiple VM
+                  deployment in parallel.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15603</para></entry>
+              <entry><para>CloudStack now stop the VMs when destroyVM command is
+                called.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15586</para></entry>
+              <entry><para>Public Vlan for an account no longer fails if multiple physical networks
+                  are present.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15582</para></entry>
+              <entry><para>The dns-name filter is now supported for ec2-describe-instances in the
+                  Amazon Web Services API compatibility commands. The filter maps to the name of a
+                  user VM.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15503</para></entry>
+              <entry><para>An IP address which has static NAT rules can now be released.
+                  Subsequently, restarting this network after it was shutdown can
+                succeed.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15464</para></entry>
+              <entry><para>Can now delete static route whose state is set to Revoke.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15443</para></entry>
+              <entry><para>Creating a firewall rule no longer fails with an internal server
+                  error.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15398</para></entry>
+              <entry><para>Corrected technique for programming DNS on the user VMs.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15356</para></entry>
+              <entry><para>Internal DNS 2 entry now correctly shown in UI.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15335</para></entry>
+              <entry><para>The CloudBridge S3 Engine now connects to the database by using the
+                  deciphered password in the db.properties file.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15318</para></entry>
+              <entry><para>UI now correctly prevents the user from stopping a VM that is in the
+                  Starting state.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15307</para></entry>
+              <entry><para>Fixed Japanese localization of instance statuses in the Instances
+                  menu.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15278</para></entry>
+              <entry><para>The deployment planner no longer takes long time to locate a suitable
+                  host to deploy VMs when large number of clusters are present.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15274</para></entry>
+              <entry><para>Creating a VLAN range using Zone ID without network ID now
+                  succeeds.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15243</para></entry>
+              <entry><para>Now check to be sure source NAT and VPN have same
+                provider.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15232</para></entry>
+              <entry><para>Ensure that networks using external load balancer/firewall in 2.2.14 or
+                  earlier can properly upgrade.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15200</para></entry>
+              <entry><para>No exception when trying to attach the same volume while attaching the
+                  first volume is in progress.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15173</para></entry>
+              <entry><para>Additional cluster can no longer be added with same VSM IP address as
+                  another cluster.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15167</para></entry>
+              <entry><para>AWS API calls now honor the admin account's ability to view or act on the
+                  resources owned by the regular users.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15163</para></entry>
+              <entry><para>The minimum limit is not honored when there is not enough capacity to
+                  deploy all the VMs and the ec2-run-instances command with the -n &gt;n1 -n2&gt;
+                  option is used to deploy multiple VMs.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15157</para></entry>
+              <entry><para>Can now add/enable service providers for multiple physical networks
+                  through the UI.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15145</para></entry>
+              <entry><para>AWS API call ec2-register has better error handling for negative
+                  cases.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15122</para></entry>
+              <entry><para>Filters now supported for AWS API call
+                  ec2-describe-availability-zones.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15120</para></entry>
+              <entry><para>Actions column in UI of Volume page now shows action
+                links.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15099</para></entry>
+              <entry><para>Buttons no longer overlap text on Account Deletion confirmation page in
+                  UI.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15095</para></entry>
+              <entry><para>Ensures you can not create a VM with a CPU frequency greater than the
+                  host CPU frequency.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15094</para></entry>
+              <entry><para>CPU cap now set properly in VMware.</para></entry>
+            </row>
+            <row>
+              <entry><para>CS-15077</para></entry>
+              <entry><para>NullPointerException is no longer observed 

<TRUNCATED>

Mime
View raw message