Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Received-SPF: pass (nike.apache.org: domain of donal.lafferty@citrix.com
 designates 46.33.159.39 as permitted sender)
From: Donal Lafferty <donal.lafferty@citrix.com>
To: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>
CC: "cloudstack-dev@incubator.apache.org"
	<cloudstack-dev@incubator.apache.org>
Subject: RE: Update on Hyper-V plugin work
Thread-Topic: Update on Hyper-V plugin work
Thread-Index: Ac5YI73ZIkMIXjZkQ5el/mUB/povIwAkQUyAADmGKHA=
Date: Mon, 27 May 2013 16:38:23 +0000
Message-ID: <9ADDE3F979256644BED8F0D244BE51F0018D24@LONPEX01CL02.citrite.net>
References: <9ADDE3F979256644BED8F0D244BE51F001769F@LONPEX01CL02.citrite.net>
 <20130524202743.GG90457@USLT-205755.sungardas.corp>
In-Reply-To: <20130524202743.GG90457@USLT-205755.sungardas.corp>
Accept-Language: en-GB, en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

 Hi Chip,

Thanks for clarifying the IP info required.  I'll take a closer look ASAP.

WRT collaboration, on the agent-side, I did the design work, and used colle=
agues working on CloudStack features for learning and some discussion.  Rev=
iews requests have flowed through this mailing list.  However, the final ar=
chitecture meant to ease scaling out development.  The stateless agent and =
HTTP request model should allow developers to add commands without stepping=
 on each other's toes.
=20
On the System VM side, the community updated the storage architecture such =
that CloudStack 4.2 will expect Hyper-V to use S3, and they updated the sys=
tem VM such that it is using a kernel with Hyper-V PV drivers.  Citrix have=
 provided some effort for a suitable console proxy, which needs to be integ=
rated into the System VM.  (Volunteers?)

DL


-----Original Message-----
From: Chip Childers [mailto:chip.childers@sungard.com]=20
Sent: 24 May 2013 1:28 PM
To: dev@cloudstack.apache.org
Cc: cloudstack-dev@incubator.apache.org
Subject: Re: Update on Hyper-V plugin work

Donal,

First, thank you for sending this out!  It seems like really good progress.=
  Who are you working with on this?  Are you running solo?

The technical elements of this seem sound at first review.

The licensing implications are going to take some time to parse through tho=
ugh.  One of the things that could help us on that front, would be to under=
stand which of the licenses are runtime vs. buildtime.  It would also be go=
od to highlight which dependencies would easily be considered "system depen=
dencies" (as in: expected to be installed prior to our software being insta=
lled).

Again, thanks!

-chip

On Fri, May 24, 2013 at 02:22:41AM +0000, Donal Lafferty wrote:
> Reproduced from=20
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Progress
>=20
> Apologies for the length - skip to the heading that interests you.
>=20
> h1. Update on Hyper-V plugin work (2013-05-23):
>=20
> h2. New plugin Architecture:
>=20
> https://cwiki.apache.org/confluence/download/attachments/31822487/Phas
> e2_HyperV_Remote_Agent.png
>=20
> h3. How does is the new architecture an improvement?
>=20
> The plugin's architecture uses JSON over HTTP.  This allows the plugin ag=
ent to be written in C# rather than Java.  .NET has tools to create WMI wra=
ppers that allows the Hyper-V API calls to be quickly rewritten in C#.  The=
 result is easier to maintain and does not suffer the IP problems of the Py=
thon used in phase 1.  An all C# agent can be contained in a single C#/.NET=
 Windows Service.  The result is a more robust design, because there are fa=
r fewer points of failure.
>=20
> Switching from CloudStack Message bus to JSON over HTTP communications al=
lows the plugin's remote agent to be implemented in C#.  Previously, this a=
gent was written in Java to take advantage to the existing CloudStack messa=
ge bus agent.  The CloudStack message bus is difficult to port due it's rel=
iance on the Java NIO API. The NIO API is peculiar to Java.  As a consequen=
ce, porting to even similar languages such as C# is difficult.  Design work=
 is required that would consume significant time. In contrast, JSON over HT=
TP passes kernel commands as JSON in the body of an HTTP POST.  This soluti=
on is RPC-like and not RESTful, because the destination URI that correspond=
s to the command and not the resource being manipulated.  The HTTP stack re=
moves the dependency on NIO.  HTTP stacks themselves are easy to come by re=
gardless of the language or platform.  Using an RPC approach avoids underta=
king design work to translate incoming kernel commands to a suitable RESTfu=
l equivalent.  Therefore, JSON over HTTP minimises the effort required to a=
dopt the best language for a platform.
>=20
> C# code to call the Hyper-V's API directly removes IP restrictions and co=
mplexity of the existing Python code.  Hyper-V is controlled through Window=
s Management Interface (WMI).  WMI provides a uniform framework through whi=
ch services and operating subsystems can provide information and register c=
ontrol APIs.  All Hyper-V control packages ultimately call this API. Phase =
1 of the plugin controlled Hyper-V using a port of cloud.com's OpenStack dr=
iver for Hyper-V.  The driver could be easily repurposed, because the OpenS=
tack information model is similar to that of CloudStack.  However, this Pyt=
hon code has IP, support, and coding drawbacks.  The code is a derivative o=
f an existing work, which prevented immediate donation to ASF. The code use=
d a third party WMI wrapper package that limited the version of Python that=
 could be used.  Finally, the python wrapper did not address discover of WM=
I objects, properties and methods.  In contrast, the C# code can be transfe=
rred to ASF immediately.  The strongly typed wrappers rely on C# and .NET, =
which is well supported. These wrappers are simple to update as they are ma=
chine generated code.  More importantly, they enable property and method di=
scovery through autocomplete.  Therefore, the C# version is easier to maint=
ain and addresses Apache's copyright requirements.
>=20
> Access to native error handling, APIs and services make the C#/.NET agent=
 more robust. Previously, the remote agent created a new process every time=
 Python was invoked.  Communications with this new process were over stdin =
and stdout.  The serialisation requirements meant exceptions could not be u=
sed for error handling.  Also, the design was vulnerable to deadlock.  Shou=
ld the new process not return but not die either, the call would be stuck. =
 Now that the agent is written entirely in C# agent, it can pass errors in =
a single exception rather than a series of method returns.  Furthermore, us=
ing Python along side Java in phase 1 provides access to platform-specific =
APIs not exposed by the Java Runtime Environment (JRE).  In contrast, .NET =
allows direct access to Windows APIs, which means the C# agent has no need =
for external processes.  Finally, additional scripts were required to start=
/stop/restart the phase 1 agent.  With phase 2, the agent is implemented wi=
th hooks that allow it to be launched as a Windows Service.  Using these ho=
oks, the Windows Service Control Manager (SCM) takes care of executing the =
agent as if it were a daemon.  Therefore, switching to a single language an=
d process means the new remote agent has fewer points of failure.
>=20
> An additional strength of this design is that pushes the hypervisor-speci=
fic code out of the server component.  Previously, a hypervisor's remote ag=
ent would have to register itself with the management server.  This meant t=
hat the agent had to somehow know its zone, pod, and cluster location befor=
e it joined itself to the cloud.  For example, the KVM server component use=
s SSH to launch the remote agent with the necessary configuration data.  In=
 our new design, the remote agent can be told its cloud configuration after=
 it has been started.  This removes the need for a hypervisor-specific cont=
rol to launch remote agent.  Therefore, the resulting server component is m=
uch more reusable.
>=20
> To summarise, the plugin has an updated communications stack called JSON =
over HTTP.  This has simplified the implementation of the remote agent and =
the corresponding server component.  The result is a more robust and reusab=
le design, and a new implementation that meets Apache CloudStack's IP requi=
rements.
>=20
>=20
> h2. Implementation Status:
>=20
> A VisualStudio 2012 SP1 solution on GitHub ([https://github.com/lafferty/=
cshv3/tree/master/plugins/hypervisors/hyperv/DotNet/ServerResource]) holds =
the plugin's new remote agent.  VM and volume creation / deletion commands =
are mostly complete. Additional storage commands are required to allow tran=
sfer of template and volume images.  Attach / detach commands should be add=
ed to allow System VM patching.
>=20
> The solution includes unit tests.  The tests are a port of the functional=
 tests from phase 1.  At the time of writing, the port was still being comp=
leted.
>=20
> The implementation avoids using features that would require the hyperviso=
r to run Windows Server 2012 in the parent partition.  The image store uses=
 S3 for template storage, because the Windows NFS client is an advanced fea=
ture that is not available with the basic and free Hyper-V Server 2012.  Li=
kewise, the remote agent hosts its own HTTP stack to avoid any dependency o=
f Microsoft's Internet Information Services (IIS) web server.
>=20
> A Server component is available that uses JSON over HTTP to communicate t=
he remote agent.  The existing unit tests are being updated to call this se=
rver component.  Additional scripting is required to launch the remote agen=
t so that HTTP requests can be properly received and acted on.  The server =
component's Discoverer algorithm needs to be updated.
>=20
> The build and test system has dependencies on Microsoft tools and platfor=
ms.  To build the system with free tools, the remote agent should be compil=
ed against VisualStudio express.  To build with Linux, the solution should =
be compiled with Mono.  However, neither of these options have been tried.
>=20
>=20
> h2. IP Dependencies
>=20
> The C# source code and configuration files that are input for code genera=
te are in the process of being marked with the Apache header ([http://www.a=
pache.org/legal/src-headers.html#headers]) to indicate that Apache CloudSta=
ck will hold the copyright.
>=20
> Third party binaries used by the solution are the Microsoft .NET Framewor=
k 4.5, NuGet packages, and AWS .NET SDK.  The AWS SDK is used to access S3 =
storage.  The NuGet packages provide logging, JSON serialisation, and a lig=
ht weight HTTP stack.  The background to NuGet is that it is akin to a Mave=
n repo, but each packages has a corresponding web page that includes detail=
s of the license for use of the package.
>=20
> Licenses for the NuGet packages are enumerated below.  I do not have the =
.NET Framework 4.5 EULA to hand.
>=20
>=20
> Apache License, Version 2.0 ([http://logging.apache.org/log4net/license.h=
tml)]:
>=20
> AWS .NET SDK ([http://aws.amazon.com/sdkfornet/faqs/#13])
> Log4net 2.0.0 ([http://nuget.org/packages/log4net/])
>=20
>=20
> The MIT License (MIT):
>=20
> Newtonsoft.Json 4.5.11 [http://json.codeplex.com/license]
>=20
>=20
> MICROSOFT ASP.NET MODEL VIEW CONTROLLER 4  EULA=20
> ([http://www.microsoft.com/web/webpi/eula/mvc_4_eula_enu.htm])
>=20
> Microsoft ASP.NET Web API Core Libraries 4.0.20710.0 [http://nuget.org/pa=
ckages/Microsoft.AspNet.WebApi.Core/4.0.20710.0] NuGet id=3D"Microsoft.AspN=
et.WebApi.Core"  version=3D"4.0.20710.0"
> Microsoft ASP.NET Web API Client Libraries 4.0.20710.0 [http://nuget.org/=
packages/Microsoft.AspNet.WebApi.Client/4.0.20710.0] NuGet id=3D"Microsoft.=
AspNet.WebApi.Client" version=3D"4.0.20710.0"
> Microsoft .NET Framework 4 HTTP Client Libraries 2.0.20710.0 [http://nuge=
t.org/packages/Microsoft.Net.Http/2.0.20710.0] NuGet id=3D"Microsoft.Net.Ht=
tp" version=3D"2.0.20710.0"
> Microsoft ASP.NET Web API Self Host 4.0.20918.0 [http://nuget.org/package=
s/Microsoft.AspNet.WebApi.SelfHost/4.0.20918.0] NuGet id=3D"Microsoft.AspNe=
t.WebApi.SelfHost" version=3D"4.0.20918.0"
>=20
>=20
>=20
> h2. System VM services:
>=20
> The plugin depends on system VM services that are coded separately.  Syst=
em VM services refer to CloudStack functionality that is offloaded from the=
 management server to VMs that sit in the cloud.  The services in question =
are template/snapshot storage, networking services, and console access.  Tr=
aditionally, CloudStack has implemented these services using system VMs:  s=
econdary storage VM for image persistence, virtual router VM for networking=
 services, and console VM for access to guest VM consoles.  For the Hyper-V=
 plugin to work properly, CloudStack has to be able to offer these same ser=
vices on Hyper-V hypervisors.
>=20
> The image that the CloudStack database points to for Hyper-V system VM ne=
eds to be updated with a version that includes the console proxy.  The late=
st system VM includes Hyper-V kernel modules.  This provides it with the hi=
gh performance network driver to act as a virtual router VM.  The secondary=
 storage VM capabilities are not required.  The secondary storage VM allows=
 CloudStack to upload images to an NFS store.  However, a NFS client is not=
 available in the "Hyper-V Server" that we are targeting.  Therefore, we wi=
ll not be using NFS for image storage.  Finally, the console services must =
be updated.  Hyper-V's guest consoles are accessed through RDP, which means=
 an RDP proxy must be added to the system VM.  Therefore, changes to the sy=
stem VM appear to be limited to the addition of a Hyper-V compatible consol=
e proxy.