cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pawit Pornkitprasan <>
Subject PCI-Passthrough with CloudStack (Improved)
Date Thu, 20 Jun 2013 04:12:14 GMT

Following my previous post about implementing PCI Passthrough on
CloudStack (KVM), I have taken Edison Su’s and others’ comments into
account and came up with an improved design.

Because the devices available at each agent may be different, the
available devices for passthrough are now configured at the agent
configuration file (/etc/cloudstack/agent/
Configuration is a comma separated list of available PCI devices and
its given name.


At agent startup, the list of PCI devices is parsed and sent together
with StartupRoutingCommand (in a new field, not in details). The
management server then stores it in a new table “op_host_pci_devices”.
If a device is added, removed, or renamed, the table is updated
accordingly. The current schema has the following fields

id (auto-increment)
host_id (host that this device belongs to)
name (given name of the PCI device)
domain (PCI ID - domain)
bus (PCI ID - bus)
slot (PCI ID - slot)
function (PCI ID - function)
instance_id (ID of the VM using the PCI device, NULL if not in use)

The “name” of the PCI device is what is used to assign a device. In a
compute offering, the user can specify the name of one or more PCI
devices (as a comma-separated list) and CloudStack will find a host
with the PCI device of the specified name available and assign it.

A new manager, PciDeviceManager, is then created to handle the
allocation of PCI device. The manager implements StateListener and
assigns PCI devices on state change to “starting” and also release the
devices VM stop. First fit allocator and first fit planner are also
modified to check for PCI device availability accordingly.

For migration, there are 2 approaches. The first approach is to forbid
migration and is straightforward. The second approach is to PCI
Hotplug to detach the device, migrate and attach it again at the other
end. This will interrupt whatever is using the device on the VM.
However, it may be desirable for networking devices where the VM can
use a bonding device to channel network traffic through a standard
virtualized network device while the PCI Passthrough device is down.

The design mentioned here (including detach-attach migration) has been
implemented in code and is working. Again, comments and suggestions
are welcomed.

Best Regards,

View raw message