cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Donal Lafferty <donal.laffe...@citrix.com>
Subject RE: [Merge] Minimal Hyper-V Plugin
Date Mon, 30 Sep 2013 16:05:25 GMT
Code already builds with Mono.  Indeed, functional testing of the plugin was done using Mono-built
code.

The Mono build is a profile under the plugin's pom.xml

For background on building C# code with Mono, see http://dlafferty.blogspot.co.uk/2013/08/building-your-microsoft-solution-with.html


DL


> -----Original Message-----
> From: Rajesh Battala [mailto:rajesh.battala@citrix.com]
> Sent: 30 September 2013 06:51
> To: dev@cloudstack.apache.org
> Subject: RE: [Merge] Minimal Hyper-V Plugin
> 
> Am looking at building the agent code using mono.
> Will create a wiki after finishing it.
> 
> Thanks
> Rajesh Battala
> 
> -----Original Message-----
> From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com]
> Sent: Thursday, September 26, 2013 2:55 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [Merge] Minimal Hyper-V Plugin
> 
> +1 to put this into a branch off of master. Can merge into master after
> unit tests for the agent API.
> 
> 
> On 9/24/13 12:37 PM, "Donal Lafferty" <donal.lafferty@citrix.com> wrote:
> 
> >On paternity leave, so I don't get to these emails right away...
> >
> >> -----Original Message-----
> >> From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com]
> >> Sent: 20 September 2013 06:40
> >> To: dev@cloudstack.apache.org
> >> Subject: Re: [Merge] Minimal Hyper-V Plugin
> >>
> >> Thanks for the preliminary testing.
> >> Questions:
> >> 1. More for the community: should the C# code be in a separate repo?
> >> According to the merge request, mono and maven can be used to build
> >> the agent.
> >
> >Silence == acceptance?
> >
> >> 2. Packaging: how is the C# agent installed?
> >
> >The agent is implemented to as a self-contained Windows Service, which
> >is the Microsoft Windows equivalent of a Linux daemon.
> >
> >To make the agent distributable, package the agent and an app.config
> >consistent with your data center in an MSI.  WiX is the preferred tool
> >(http://en.wikipedia.org/wiki/WiX ).  When executed, the MSI will add
> >the agent to set of Windows Services.
> >
> >To distribute and run this MSI, use Active Directory's GPO (global
> >policy
> >object) service.  In typical deployments machines running Hyper-V will
> >be domain joined.  Where machines are not domain joined, look at
> >something like puppet.
> >
> >> 3. What does minimal mean? What works? What doesn't? Local storage?
> >> Shared storage? Networking modes? Are the hypervisors supposed to be
> >> clustered?
> >
> >Minimal = create / start / stop / destroy a local storage VM in a
> >QuickCloud network offering and CIFS secondary storage.
> >
> >No clustering required.
> >
> >> 4. How does one extend the "minimal" plugin?
> >
> >Each CloudStack command has a corresponding an HTTP URI served by the
> >agent.  These are written in ASP.NET MVC4.  Data received by the agent
> >is kept in a JSON object graph.
> >
> >E.g.
> >
> >        // POST api/HypervResource/ReadyCommand
> >        [HttpPost]
> >        [ActionName(CloudStackTypes.ReadyCommand)]
> >        public JContainer ReadyCommand([FromBody]dynamic cmd)
> >        {
> >            using (log4net.NDC.Push(Guid.NewGuid().ToString()))
> >            {
> >                logger.Info(CloudStackTypes.ReadyCommand +
> >cmd.ToString());
> >                object ansContent = new
> >                {
> >                    result = true,
> >                    details = (string)null
> >                };
> >                return ReturnCloudStackTypedJArray(ansContent,
> >CloudStackTypes.ReadyAnswer);
> >            }
> >
> >        }
> >
> >Therefore, to extend the plugin, add new HTTP URIs corresponding to
> >missing commands, or extend the capabilities of existing commands.
> >
> >I can follow up with an explanation in a blog entry.
> >
> >> 5. Can the unit tests (at least those that test the agent API) be run
> >>in a non-  hyper-v environment?
> >
> >Unit tests start the agent in a local process.  Provided Mono is
> >installed on your system, the unit tests will run.  However, they will
> >complain of bad output.
> >
> >> 6. Is the RDP console you had earlier mentioned included in the merge?
> >
> >Yes, but it serves no purpose at the moment.
> >
> >If there is an IP clearance protocol to follow for this console, I
> >would prefer to remove the console from the submission.
> >
> >> 7. Any known issues?
> >>
> >
> >There seems to be a bug with local paths that include spaces.  I've
> >asked Rajesh to provide a bug report, but it's unclear where to put
> >this.  Can we use JIRA for code not merged, or should the bug appear in the
> comments.
> >
> >
> >> On 9/11/13 8:00 AM, "Donal Lafferty" <donal.lafferty@citrix.com> wrote:
> >>
> >> >Hi Rajesh,
> >> >
> >> >Thanks for spotting this problem with the Hyper-V Agent.  Sounds
> >> >like it should first URL decode the field.
> >> >
> >> >Can you update the review with details of your testing?
> >> >
> >> >I would need to know the command and which incoming field is causing
> >> >problems.  Also, can you add a serialised example of the instruction
> >> >that fails?  There should be an example in the agent's log file.  By
> >> >default, the log file is in the same folder as the agent executable.
> >> >I will use this to update the automated tests.
> >> >
> >> >If you want to go ahead and made the fixes from a git clone, send a
> >> >Pull Request.  As long as there is an appropriate automated test,
> >> >I'll update the feature branch with your changes.
> >> >
> >> >
> >> >DL
> >> >
> >> >> -----Original Message-----
> >> >> From: Rajesh Battala [mailto:rajesh.battala@citrix.com]
> >> >> Sent: 11 September 2013 09:08
> >> >> To: dev@cloudstack.apache.org
> >> >> Subject: RE: [Merge] Minimal Hyper-V Plugin
> >> >>
> >> >> Hi Donal,
> >> >> I had figured out the issue why "+" is coming in the path value.
> >> >> The root cause is while encoding the URI, we use
> >> >> URLEncoder.encode(path)
> >> >>
> >> >> The encode method is converting/replace "space" with "+".
> >> >>
> >> >> API doc:
> >> >> When encoding a String, the following rules apply:
> >> >>
> >> >> The alphanumeric characters "a" through "z", "A" through "Z" and "0"
> >> >> through "9" remain the same.
> >> >> The special characters ".", "-", "*", and "_" remain the same.
> >> >> The space character " " is converted into a plus sign "+".
> >> >> All other characters are unsafe and are first converted into one
> >> >>or more  bytes using some encoding scheme. Then each byte is
> >> >>represented by the
> >> >>3-
> >> >> character string "%xy", where xy is the two-digit hexadecimal
> >> >>representation  of the byte. The recommended encoding scheme to
> use
> >> >>is UTF-8. However,  for compatibility reasons, if an encoding is
> >> >>not specified, then the default  encoding of the platform is used.
> >> >>
> >> >> Thanks
> >> >> Rajesh Battala
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> -----Original Message-----
> >> >> From: Donal Lafferty [mailto:donal.lafferty@citrix.com]
> >> >> Sent: Monday, September 2, 2013 3:03 PM
> >> >> To: dev@cloudstack.apache.org
> >> >> Subject: RE: [Merge] Minimal Hyper-V Plugin
> >> >>
> >> >> Hi Rajesh,
> >> >>
> >> >> I'll PM the scripts.
> >> >>
> >> >> WRT problem below, it looks like a CS persists paths with spaces
> >> >> as URL encoded strings.  Is it the Hyper-V code storing the path
> >> >> in the wrong formant?  I can fix that if it's an issue.
> >> >>
> >> >> I would be very reluctant to change the the format of columns in
> >> >> the database. There will be other code reliant on URL encoding.
> >> >>
> >> >> DL
> >> >>
> >> >>
> >> >> > -----Original Message-----
> >> >> > From: Rajesh Battala [mailto:rajesh.battala@citrix.com]
> >> >> > Sent: 02 September 2013 08:37
> >> >> > To: dev@cloudstack.apache.org
> >> >> > Subject: RE: [Merge] Minimal Hyper-V Plugin
> >> >> >
> >> >> > Hi Donal,
> >> >> > One more issue is, currently local storage is discovered when
> >> >> > host is
> >> >>added.
> >> >> > But when mgmt. server got restarted,  there is an exception
> >> >> > happening while add/discovering already existing local storage
> >>pool.
> >> >> >
> >> >> >
> >> >> > Root cause:
> >> >> > ==========
> >> >> > I had debugged and figured out the root cause,  CS not able to
> >> >> > find the storage pool and tries to created it but it fails to
> >> >> > add to DB as the entry is already persisted.
> >> >> > The reason why CS not able to get the existing localstorage is
,
> >> >> > when executing the query to get the storage pool, it's getting
> >>empty set
> >> .
> >> >> > It's because of the storage path issue.
> >> >> > In the db the local storage path is stored as
> >> >> > "C:\Users\Public\Documents\Hyper-V\Virtual+Hard+Disks",  ('+'
> >> >> > symbol in place of space) In the db query its searching without
"+"
> >> >> > and with space which is causing the query to have empty set and
> >> >> > hence the issue.
> >> >> >
> >> >> > I can fix the issue by removing the "+" when we are persisting
> >> >> > the local storage pool from hyperv
> >> >> >
> >> >> >
> >> >> >
> >> >> > Exception:
> >> >> > =========
> >> >> > WARN  [c.c.r.ResourceManagerImpl]
> >> >> > (AgentTaskPool-14:ctx-ddbbf089) Unable to connect due to
> >> >> > com.cloud.exception.ConnectionException: Unable to setup the
> >> >> > local storage pool for Host[-1-Routing]
> >> >> >         at
> >> >> >
> >> >>
> >>
> com.cloud.storage.StorageManagerImpl.createLocalStorage(StorageManage
> >> >> > rImpl.java:598)
> >> >> >         at
> >> >> >
> >> >>
> >>
> com.cloud.utils.component.ComponentInstantiationPostProcessor$Interce
> >> >> p
> >> >> >
> >> torDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> >> >> >         at
> >> >> > com.cloud.storage.LocalStoragePoolListener.processConnect(LocalS
> >> >> > tor
> >> >> > age
> >> >> > Po
> >> >> > olListener.java:86)
> >> >> >         at
> >> >> >
> >> >>
> >>
> com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(
> >> >> > AgentManagerImpl.java:519)
> >> >> >         at
> >> >> >
> >> >>
> >>
> com.cloud.agent.manager.AgentManagerImpl.handleDirectConnectAgent(A
> >> >> > gentManagerImpl.java:1414)
> >> >> >         at
> >> >> >
> >>
> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(Resource
> >> >> > ManagerImpl.java:1760)
> >> >> >         at
> >> >> >
> >>
> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(Resource
> >> >> > ManagerImpl.java:1922)
> >> >> >         at
> >> >> >
> >> >>
> >>
> com.cloud.agent.manager.AgentManagerImpl$SimulateStartTask.run(Agent
> >> >> > ManagerImpl.java:1063)
> >> >> >         at
> >> >> >
> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecuto
> >> >> > r.j
> >> >> > av
> >> >> > a:1110)
> >> >> >         at
> >> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
> >> >> > ja
> >> >> > va:603)
> >> >> >         at java.lang.Thread.run(Thread.java:722)
> >> >> > Caused by: com.cloud.utils.exception.CloudRuntimeException:
> >> >> > duplicate
> >> >> > uuid: 35aa91ba-c95b-3fc4-91df-9c95dc31b350-HypervResource
> >> >> >         at
> >> >> >
> >>
> org.apache.cloudstack.storage.volume.datastore.PrimaryDataStoreHelper.
> >> >> > cr
> >> >> > eatePrimaryDataStore(PrimaryDataStoreHelper.java:66)
> >> >> >         at
> >> >> > org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrim
> >> >> > ary
> >> >> > Dat
> >> >> > aSto
> >> >> >
> >> >>
> >> >>reLifeCycleImpl.initialize(CloudStackPrimaryDataStoreLifeCycleImpl.
> >> >>jav
> >> >>a:3
> >> >>49)
> >> >> >         at
> >> >> >
> >> >>
> >>
> com.cloud.storage.StorageManagerImpl.createLocalStorage(StorageManage
> >> >> > rImpl.java:589)
> >> >> >         ... 14 more
> >> >> >
> >> >> >
> >> >> >
> >> >> > -----Original Message-----
> >> >> > From: Donal Lafferty [mailto:donal.lafferty@citrix.com]
> >> >> > Sent: Saturday, August 31, 2013 5:19 AM
> >> >> > To: dev@cloudstack.apache.org
> >> >> > Subject: [Merge] Minimal Hyper-V Plugin
> >> >> >
> >> >> > A plugin for Hyper-V control is available for CloudStack.  The
> >> >> > plugin implements basic VM control;  however, its architecture
> >> >> > allows additional functionality to be easily added.
> >> >> > Incorporating the plugin in CloudStack will allow the community
> >> >> > to participate in improving the features available with Hyper-V.
> >> >> >
> >> >> > The plugin uses a Director Connect Agent architecture described
> >>here:
> >> >> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Progress
> >> >> >
> >> >> > This links provides details of third party libraries used by the
> >> >> > plugin including their licenses.  No source for these libraries
> >> >> > is used, and the binaries are downloaded from their distributors
> >> >> > at
> >> >>build time.
> >> >> >
> >> >> > No proprietary tools are required for the build.  For instance,
> >> >> > C# compiled with Mono has been tested.  Therefore, the plugin
> >> >> > has been added to the default build and the default deployment.
> >> >> >
> >> >> > The plugin includes unit and functional tests that can be
> >> >> > triggered at build time.  See the link above for details.
> >> >> >
> >> >> > The Apache header is applied to source and where feasible to
> >> >> > build config files.
> >> >> >
> >> >> > The source is the
> >> >> > https://github.com/lafferty/cloudstack/tree/hyperv_plugin
> >> >> >
> >> >> > Finally, the review request is at
> >> >> > https://reviews.apache.org/r/13922/
> >> >> >
> >> >> > Testing?
> >> >> >
> >> >> > Integration test:  created zone with CIFS secondary storage,
> >> >> > added Hyper-V host, registered templates,
> create/stop/start/destroy VM.
> >> >> > Scripts for zone setup are available on request.
> >> >> >
> >> >> > Functional & unit tests:  the Java versions are part of the
build.
> >> >> > The C# versions were run inside Visual Studio's Test Explorer.
> >


Mime
View raw message