22.2. Working with Server Logs

- The CloudStack Management Server logs all web site, middle tier, and database activities for diagnostics purposes in /var/log/cloud/management/. The CloudStack logs a variety of error messages. We recommend this command to find the problematic output in the Management Server log:. -

Note

- When copying and pasting a command, be sure the command has pasted as a single line before executing. Some document viewers may introduce unwanted line breaks in copied text. -

-        grep -i -E 'exception|unable|fail|invalid|leak|warn|error' /var/log/cloud/management/management-server.log
-

- The CloudStack processes requests with a Job ID. If you find an error in the logs and you are interested in debugging the issue you can grep for this job ID in the management server log. For example, suppose that you find the following ERROR message: -

-        2010-10-04 13:49:32,595 ERROR [cloud.vm.UserVmManagerImpl] (Job-Executor-11:job-1076) Unable to find any host for [User|i-8-42-VM-untagged]
-

- Note that the job ID is 1076. You can track back the events relating to job 1076 with the following grep: -

-        grep "job-1076)" management-server.log
-

- The CloudStack Agent Server logs its activities in /var/log/cloud/agent/. -

Chapter 22. Troubleshooting

22.1. Events

22.1.1. Event Logs

- There are two types of events logged in the CloudStack Event Log. Standard events log the success or failure of an event and can be used to identify jobs or processes that have failed. There are also long running job events. Events for asynchronous jobs log when a job is scheduled, when it starts, and when it completes. Other long running synchronous jobs log when a job starts, and when it completes. Long running synchronous and asynchronous event logs can be used to gain more information on the status of a pending job or can be used to identify a job that is hanging or has not started. The following sections provide more information on these events.. -

22.1.2. Standard Events

- The events log records three types of standard events. -

- INFO. This event is generated when an operation has been successfully performed. -
- WARN. This event is generated in the following circumstances. -
- - When a network is disconnected while monitoring a template download. -
- - When a template download is abandoned. -
- - When an issue on the storage server causes the volumes to fail over to the mirror storage server. -
- ERROR. This event is generated when an operation has not been successfully performed -

22.1.3. Long Running Job Events

- The events log records three types of standard events. -

- INFO. This event is generated when an operation has been successfully performed. -
- WARN. This event is generated in the following circumstances. -
- - When a network is disconnected while monitoring a template download. -
- - When a template download is abandoned. -
- - When an issue on the storage server causes the volumes to fail over to the mirror storage server. -
- ERROR. This event is generated when an operation has not been successfully performed -

22.1.4. Event Log Queries

- Database logs can be queried from the user interface. The list of events captured by the system includes: -

- Virtual machine creation, deletion, and on-going management operations -
- Virtual router creation, deletion, and on-going management operations -
- Template creation and deletion -
- Network/load balancer rules creation and deletion -
- Storage volume creation and deletion -
- User login and logout -

Chapter 21. Tuning

- This section provides tips on how to improve the performance of your cloud. -

21.1. Performance Monitoring

- Host and guest performance monitoring is available to end users and administrators. This allows the user to monitor their utilization of resources and determine when it is appropriate to choose a more powerful service offering or larger disk. -

Chapter 5. User Interface

5.1. Log In to the UI

- CloudStack provides a web-based UI that can be used by both administrators and end users. The appropriate version of the UI is displayed depending on the credentials used to log in. The UI is available in popular browsers including IE7, IE8, IE9, Firefox 3.5+, Firefox 4, Safari 4, and Safari 5. The URL is: (substitute your own management server IP address) -

http://<management-server-ip-address>:8080/client

- On a fresh Management Server installation, a guided tour splash screen appears. On later visits, you’ll see a login screen where you specify the following to proceed to your Dashboard: -

Username

- The user ID of your account. The default username is admin. -

Password

- The password associated with the user ID. The password for the default username is password. -

Domain

- If you are a root user, leave this field blank. -

- If you are a user in the sub-domains, enter the full path to the domain, excluding the root domain. -

- For example, suppose multiple levels are created under the root domain, such as Comp1/hr. The users in the Comp1 domain should enter Comp1 in the Domain field, whereas the users in the Comp1/sales domain should enter Comp1/sales. -

- For more guidance about the choices that appear when you log in to this UI, see Logging In as the Root Administrator. -

5.1.1. End User's UI Overview

- The CloudStack UI helps users of cloud infrastructure to view and use their cloud resources, including virtual machines, templates and ISOs, data volumes and snapshots, guest networks, and IP addresses. If the user is a member or administrator of one or more CloudStack projects, the UI can provide a project-oriented view. -

5.1.2. Root Administrator's UI Overview

- The CloudStack UI helps the CloudStack administrator provision, view, and manage the cloud infrastructure, domains, user accounts, projects, and configuration settings. The first time you start the UI after a fresh Management Server installation, you can choose to follow a guided tour to provision your cloud infrastructure. On subsequent logins, the dashboard of the logged-in user appears. The various links in this screen and the navigation bar on the left provide access to a variety of administrative functions. The root administrator can also use the UI to perform all the same tasks that are present in the end-user’s UI. -

5.1.3. Logging In as the Root Administrator

- After the Management Server software is installed and running, you can run the CloudStack user interface. This UI is there to help you provision, view, and manage your cloud infrastructure. -

- Open your favorite Web browser and go to this URL. Substitute the IP address of your own Management Server: -
```
http://<management-server-ip-address>:8080/client
```
- On a fresh Management Server installation, a guided tour splash screen appears. On later visits, you’ll see a login screen where you can enter a user ID and password and proceed to your Dashboard. -
- If you see the first-time splash screen, choose one of the following. -
- - Continue with basic setup. Choose this if you're just trying CloudStack, and you want a guided walkthrough of the simplest possible configuration so that you can get started right away. We'll help you set up a cloud with the following features: a single machine that runs CloudStack software and uses NFS to provide storage; a single machine running VMs under the XenServer or KVM hypervisor; and a shared public network. -
  - The prompts in this guided tour should give you all the information you need, but if you want just a bit more detail, you can follow along in the Trial Installation Guide. -
- - I have used CloudStack before. Choose this if you have already gone through a design phase and planned a more sophisticated deployment, or you are ready to start scaling up a trial cloud that you set up earlier with the basic setup screens. In the Administrator UI, you can start using the more powerful features of CloudPlatform, such as advanced VLAN networking, high availability, additional network elements such as load balancers and firewalls, and support for multiple hypervisors including Citrix XenServer, KVM, and VMware vSphere. -
  - The root administrator Dashboard appears. -
- You should set a new root administrator password. If you chose basic setup, you’ll be prompted to create a new password right away. If you chose experienced user, use the steps in Section 5.1.4, “Changing the Root Password”. -

Warning

- You are logging in as the root administrator. This account manages the CloudStack deployment, including physical infrastructure. The root administrator can modify configuration settings to change basic functionality, create or delete user accounts, and take many actions that should be performed only by an authorized person. Please change the default password to a new, unique password. -

5.1.4. Changing the Root Password

- During installation and ongoing cloud administration, you will need to log in to the UI as the root administrator. The root administrator account manages the CloudStack deployment, including physical infrastructure. The root administrator can modify configuration settings to change basic functionality, create or delete user accounts, and take many actions that should be performed only by an authorized person. When first installing CloudStack, be sure to change the default password to a new, unique value. -

- Open your favorite Web browser and go to this URL. Substitute the IP address of your own Management Server: -
```
http://<management-server-ip-address>:8080/client
```
- Log in to the UI using the current root user ID and password. The default is admin, password. -
- Click Accounts. -
- Click the admin account name. -
- Click View Users. -
- Click the admin user name. -
- Click the Change Password button. - -
- Type the new password, and click OK. -

12.8. Uploading Templates

vSphere Templates and ISOs

- If you are uploading a template that was created using vSphere Client, be sure the OVA file does not contain an ISO. If it does, the deployment of VMs from the template will fail. -

- Templates are uploaded based on a URL. HTTP is the supported access protocol. Templates are frequently large files. You can optionally gzip them to decrease upload times. -

- To upload a template: -

- In the left navigation bar, click Templates. -
- Click Create Template. -
- Provide the following: -
- - Name and Display Text. These will be shown in the UI, so choose something descriptive. -
- - URL. The Management Server will download the file from the specified URL, such as http://my.web.server/filename.vhd.gz. -
- - Zone. Choose the zone where you want the template to be available, or All Zones to make it available throughout CloudStack. -
- - OS Type: This helps CloudStack and the hypervisor perform certain operations and make assumptions that improve the performance of the guest. Select one of the following: -
  - - If the operating system of the stopped VM is listed, choose it. -
  - - If the OS type of the stopped VM is not listed, choose Other. -
    Note
    - You should not choose an older version of the OS than the version in the image. For example, choosing CentOS 5.4 to support a CentOS 6.2 image will in general not work. In those cases you should choose Other. -
- - Hypervisor -
- - Format. The format of the template upload file, such as VHD or OVA. -
- - Password Enabled. Choose Yes if your template has the CloudStack password change script installed. See Adding Password Management to Your Templates -
- - Extractable. Choose Yes if the template is available for extraction. If this option is selected, end users can download a full image of a template. -
- - Public. Choose Yes to make this template accessible to all users of this CloudStack installation. The template will appear in the Community Templates list. See Section 12.5, “Private and Public Templates” -
- - Featured. Choose Yes if you would like this template to be more prominent for users to select. The template will appear in the Featured Templates list. Only an administrator can make a template Featured. -

20.3. User Data and Meta Data

- CloudStack provides API access to attach user data to a deployed VM. Deployed VMs also have access to instance metadata via the virtual router. -

- User data can be accessed once the IP address of the virtual router is known. Once the IP address is known, use the following steps to access the user data: -

- Run the following command to find the virtual router. -

# cat /var/lib/dhclient/dhclient-eth0.leases | grep dhcp-server-identifier | tail -1

- Access user data by running the following command using the result of the above command -
```
# curl http://10.1.1.1/latest/user-data
```

- Meta Data can be accessed similarly, using a URL of the form http://10.1.1.1/latest/meta-data/{metadata type}. (For backwards compatibility, the previous URL http://10.1.1.1/latest/{metadata type} is also supported.) For metadata type, use one of the following: -

- service-offering. A description of the VMs service offering -
- availability-zone. The Zone name -
- local-ipv4. The guest IP of the VM -
- local-hostname. The hostname of the VM -
- public-ipv4. The first public IP for the router. (E.g. the first IP of eth2) -
- public-hostname. This is the same as public-ipv4 -
- instance-id. The instance name of the VM -

22.3. Data Loss on Exported Primary Storage

Symptom

Cause

Solution

More Information

22.8. Load balancer rules fail after changing network offering

Symptom

Cause

Solution

22.5. Maintenance mode not working on vCenter

Symptom

Cause

Solution

More Information

22.4. Recovering a Lost Virtual Router

Symptom

Cause

Solution

22.6. Unable to deploy VMs from uploaded vSphere template

Symptom

Cause

Solution

22.7. Unable to power on virtual machine on VMware

Symptom

Cause

Solution

22.2. Working with Server Logs

Note

Chapter 22. Troubleshooting

22.1. Events

22.1.1. Event Logs

22.1.2. Standard Events

22.1.3. Long Running Job Events

22.1.4. Event Log Queries

Chapter 21. Tuning

21.1. Performance Monitoring

Chapter 5. User Interface

5.1. Log In to the UI

Username

Password

Domain

5.1.1. End User's UI Overview

5.1.2. Root Administrator's UI Overview

5.1.3. Logging In as the Root Administrator

Warning

5.1.4. Changing the Root Password

12.8. Uploading Templates

vSphere Templates and ISOs

Note

6.7. Using the Project View

20.3. User Data and Meta Data