cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajesh Battala <rajesh.batt...@citrix.com>
Subject RE: [Merge] Minimal Hyper-V Plugin
Date Sat, 28 Sep 2013 15:18:00 GMT
+1
I had used the code and Agent code works fine.
Adding HyperV host, discovering local storage and Advance Networking for user VMs works fine
( in mixed environment ).

Thanks
Rajesh Battala

-----Original Message-----
From: Sebastien Goasguen [mailto:runseb@gmail.com] 
Sent: Thursday, September 26, 2013 12:29 PM
To: dev@cloudstack.apache.org
Subject: Re: [Merge] Minimal Hyper-V Plugin

+1

I would love to see a screencast of how this works, once it's put in its own branch (and you
back from paternity leave...congratulations :) )

On Sep 25, 2013, at 5:25 PM, Chiradeep Vittal <Chiradeep.Vittal@citrix.com> wrote:

> +1 to put this into a branch off of master. Can merge into master 
> +after
> unit tests for the agent API.
> 
> 
> On 9/24/13 12:37 PM, "Donal Lafferty" <donal.lafferty@citrix.com> wrote:
> 
>> On paternity leave, so I don't get to these emails right away...
>> 
>>> -----Original Message-----
>>> From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com]
>>> Sent: 20 September 2013 06:40
>>> To: dev@cloudstack.apache.org
>>> Subject: Re: [Merge] Minimal Hyper-V Plugin
>>> 
>>> Thanks for the preliminary testing.
>>> Questions:
>>> 1. More for the community: should the C# code be in a separate repo?
>>> According to the merge request, mono and maven can be used to build 
>>> the agent.
>> 
>> Silence == acceptance?
>> 
>>> 2. Packaging: how is the C# agent installed?
>> 
>> The agent is implemented to as a self-contained Windows Service, 
>> which is the Microsoft Windows equivalent of a Linux daemon.
>> 
>> To make the agent distributable, package the agent and an app.config 
>> consistent with your data center in an MSI.  WiX is the preferred 
>> tool (http://en.wikipedia.org/wiki/WiX ).  When executed, the MSI 
>> will add the agent to set of Windows Services.
>> 
>> To distribute and run this MSI, use Active Directory's GPO (global 
>> policy
>> object) service.  In typical deployments machines running Hyper-V 
>> will be domain joined.  Where machines are not domain joined, look at 
>> something like puppet.
>> 
>>> 3. What does minimal mean? What works? What doesn't? Local storage?
>>> Shared storage? Networking modes? Are the hypervisors supposed to be 
>>> clustered?
>> 
>> Minimal = create / start / stop / destroy a local storage VM in a 
>> QuickCloud network offering and CIFS secondary storage.
>> 
>> No clustering required.
>> 
>>> 4. How does one extend the "minimal" plugin?
>> 
>> Each CloudStack command has a corresponding an HTTP URI served by the 
>> agent.  These are written in ASP.NET MVC4.  Data received by the 
>> agent is kept in a JSON object graph.
>> 
>> E.g.
>> 
>>       // POST api/HypervResource/ReadyCommand
>>       [HttpPost]
>>       [ActionName(CloudStackTypes.ReadyCommand)]
>>       public JContainer ReadyCommand([FromBody]dynamic cmd)
>>       {
>>           using (log4net.NDC.Push(Guid.NewGuid().ToString()))
>>           {
>>               logger.Info(CloudStackTypes.ReadyCommand + 
>> cmd.ToString());
>>               object ansContent = new
>>               {
>>                   result = true,
>>                   details = (string)null
>>               };
>>               return ReturnCloudStackTypedJArray(ansContent,
>> CloudStackTypes.ReadyAnswer);
>>           }
>> 
>>       }
>> 
>> Therefore, to extend the plugin, add new HTTP URIs corresponding to 
>> missing commands, or extend the capabilities of existing commands.
>> 
>> I can follow up with an explanation in a blog entry.
>> 
>>> 5. Can the unit tests (at least those that test the agent API) be 
>>> run in a non- hyper-v environment?
>> 
>> Unit tests start the agent in a local process.  Provided Mono is 
>> installed on your system, the unit tests will run.  However, they 
>> will complain of bad output.
>> 
>>> 6. Is the RDP console you had earlier mentioned included in the merge?
>> 
>> Yes, but it serves no purpose at the moment.
>> 
>> If there is an IP clearance protocol to follow for this console, I 
>> would prefer to remove the console from the submission.
>> 
>>> 7. Any known issues?
>>> 
>> 
>> There seems to be a bug with local paths that include spaces.  I've 
>> asked Rajesh to provide a bug report, but it's unclear where to put 
>> this.  Can we use JIRA for code not merged, or should the bug appear in the comments.
>> 
>> 
>>> On 9/11/13 8:00 AM, "Donal Lafferty" <donal.lafferty@citrix.com> wrote:
>>> 
>>>> Hi Rajesh,
>>>> 
>>>> Thanks for spotting this problem with the Hyper-V Agent.  Sounds 
>>>> like it should first URL decode the field.
>>>> 
>>>> Can you update the review with details of your testing?
>>>> 
>>>> I would need to know the command and which incoming field is 
>>>> causing problems.  Also, can you add a serialised example of the 
>>>> instruction that fails?  There should be an example in the agent's 
>>>> log file.  By default, the log file is in the same folder as the 
>>>> agent executable.  I will use this to update the automated tests.
>>>> 
>>>> If you want to go ahead and made the fixes from a git clone, send a 
>>>> Pull Request.  As long as there is an appropriate automated test, 
>>>> I'll update the feature branch with your changes.
>>>> 
>>>> 
>>>> DL
>>>> 
>>>>> -----Original Message-----
>>>>> From: Rajesh Battala [mailto:rajesh.battala@citrix.com]
>>>>> Sent: 11 September 2013 09:08
>>>>> To: dev@cloudstack.apache.org
>>>>> Subject: RE: [Merge] Minimal Hyper-V Plugin
>>>>> 
>>>>> Hi Donal,
>>>>> I had figured out the issue why "+" is coming in the path value.
>>>>> The root cause is while encoding the URI, we use
>>>>> URLEncoder.encode(path)
>>>>> 
>>>>> The encode method is converting/replace "space" with "+".
>>>>> 
>>>>> API doc:
>>>>> When encoding a String, the following rules apply:
>>>>> 
>>>>> The alphanumeric characters "a" through "z", "A" through "Z" and "0"
>>>>> through "9" remain the same.
>>>>> The special characters ".", "-", "*", and "_" remain the same.
>>>>> The space character " " is converted into a plus sign "+".
>>>>> All other characters are unsafe and are first converted into one 
>>>>> or more  bytes using some encoding scheme. Then each byte is 
>>>>> represented by the
>>>>> 3-
>>>>> character string "%xy", where xy is the two-digit hexadecimal 
>>>>> representation  of the byte. The recommended encoding scheme to 
>>>>> use is UTF-8. However,  for compatibility reasons, if an encoding 
>>>>> is not specified, then the default  encoding of the platform is used.
>>>>> 
>>>>> Thanks
>>>>> Rajesh Battala
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> -----Original Message-----
>>>>> From: Donal Lafferty [mailto:donal.lafferty@citrix.com]
>>>>> Sent: Monday, September 2, 2013 3:03 PM
>>>>> To: dev@cloudstack.apache.org
>>>>> Subject: RE: [Merge] Minimal Hyper-V Plugin
>>>>> 
>>>>> Hi Rajesh,
>>>>> 
>>>>> I'll PM the scripts.
>>>>> 
>>>>> WRT problem below, it looks like a CS persists paths with spaces 
>>>>> as URL encoded strings.  Is it the Hyper-V code storing the path 
>>>>> in the wrong formant?  I can fix that if it's an issue.
>>>>> 
>>>>> I would be very reluctant to change the the format of columns in 
>>>>> the database. There will be other code reliant on URL encoding.
>>>>> 
>>>>> DL
>>>>> 
>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Rajesh Battala [mailto:rajesh.battala@citrix.com]
>>>>>> Sent: 02 September 2013 08:37
>>>>>> To: dev@cloudstack.apache.org
>>>>>> Subject: RE: [Merge] Minimal Hyper-V Plugin
>>>>>> 
>>>>>> Hi Donal,
>>>>>> One more issue is, currently local storage is discovered when 
>>>>>> host is
>>>>> added.
>>>>>> But when mgmt. server got restarted,  there is an exception 
>>>>>> happening while add/discovering already existing local storage
>>> pool.
>>>>>> 
>>>>>> 
>>>>>> Root cause:
>>>>>> ==========
>>>>>> I had debugged and figured out the root cause,  CS not able to 
>>>>>> find the storage pool and tries to created it but it fails to add

>>>>>> to DB as the entry is already persisted.
>>>>>> The reason why CS not able to get the existing localstorage is ,

>>>>>> when executing the query to get the storage pool, it's getting
>>> empty set
>>> .
>>>>>> It's because of the storage path issue.
>>>>>> In the db the local storage path is stored as 
>>>>>> "C:\Users\Public\Documents\Hyper-V\Virtual+Hard+Disks",  ('+'
>>>>>> symbol in place of space) In the db query its searching without "+"
>>>>>> and with space which is causing the query to have empty set and 
>>>>>> hence the issue.
>>>>>> 
>>>>>> I can fix the issue by removing the "+" when we are persisting 
>>>>>> the local storage pool from hyperv
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Exception:
>>>>>> =========
>>>>>> WARN  [c.c.r.ResourceManagerImpl] (AgentTaskPool-14:ctx-ddbbf089)

>>>>>> Unable to connect due to
>>>>>> com.cloud.exception.ConnectionException: Unable to setup the 
>>>>>> local storage pool for Host[-1-Routing]
>>>>>>        at
>>>>>> 
>>>>> 
>>> com.cloud.storage.StorageManagerImpl.createLocalStorage(StorageManag
>>> e
>>>>>> rImpl.java:598)
>>>>>>        at
>>>>>> 
>>>>> 
>>> com.cloud.utils.component.ComponentInstantiationPostProcessor$Interc
>>> e
>>>>> p
>>>>>> 
>>> torDispatcher.intercept(ComponentInstantiationPostProcessor.java:125
>>> )
>>>>>>        at
>>>>>> com.cloud.storage.LocalStoragePoolListener.processConnect(LocalSt
>>>>>> or
>>>>>> age
>>>>>> Po
>>>>>> olListener.java:86)
>>>>>>        at
>>>>>> 
>>>>> 
>>> com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(
>>>>>> AgentManagerImpl.java:519)
>>>>>>        at
>>>>>> 
>>>>> 
>>> com.cloud.agent.manager.AgentManagerImpl.handleDirectConnectAgent(A
>>>>>> gentManagerImpl.java:1414)
>>>>>>        at
>>>>>> 
>>> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(Resource
>>>>>> ManagerImpl.java:1760)
>>>>>>        at
>>>>>> 
>>> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(Resource
>>>>>> ManagerImpl.java:1922)
>>>>>>        at
>>>>>> 
>>>>> 
>>> com.cloud.agent.manager.AgentManagerImpl$SimulateStartTask.run(Agent
>>>>>> ManagerImpl.java:1063)
>>>>>>        at
>>>>>> 
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecuto
>>>>>> r.j
>>>>>> av
>>>>>> a:1110)
>>>>>>        at
>>>>>> 
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
>>>>>> ja
>>>>>> va:603)
>>>>>>        at java.lang.Thread.run(Thread.java:722)
>>>>>> Caused by: com.cloud.utils.exception.CloudRuntimeException:
>>>>>> duplicate
>>>>>> uuid: 35aa91ba-c95b-3fc4-91df-9c95dc31b350-HypervResource
>>>>>>        at
>>>>>> 
>>> org.apache.cloudstack.storage.volume.datastore.PrimaryDataStoreHelper.
>>>>>> cr
>>>>>> eatePrimaryDataStore(PrimaryDataStoreHelper.java:66)
>>>>>>        at
>>>>>> org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrima
>>>>>> ry
>>>>>> Dat
>>>>>> aSto
>>>>>> 
>>>>> 
>>>>> reLifeCycleImpl.initialize(CloudStackPrimaryDataStoreLifeCycleImpl
>>>>> .jav
>>>>> a:3
>>>>> 49)
>>>>>>        at
>>>>>> 
>>>>> 
>>> com.cloud.storage.StorageManagerImpl.createLocalStorage(StorageManag
>>> e
>>>>>> rImpl.java:589)
>>>>>>        ... 14 more
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Donal Lafferty [mailto:donal.lafferty@citrix.com]
>>>>>> Sent: Saturday, August 31, 2013 5:19 AM
>>>>>> To: dev@cloudstack.apache.org
>>>>>> Subject: [Merge] Minimal Hyper-V Plugin
>>>>>> 
>>>>>> A plugin for Hyper-V control is available for CloudStack.  The 
>>>>>> plugin implements basic VM control;  however, its architecture 
>>>>>> allows additional functionality to be easily added.  
>>>>>> Incorporating the plugin in CloudStack will allow the community 
>>>>>> to participate in improving the features available with Hyper-V.
>>>>>> 
>>>>>> The plugin uses a Director Connect Agent architecture described
>>> here:
>>>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Progress
>>>>>> 
>>>>>> This links provides details of third party libraries used by the

>>>>>> plugin including their licenses.  No source for these libraries 
>>>>>> is used, and the binaries are downloaded from their distributors

>>>>>> at
>>>>> build time.
>>>>>> 
>>>>>> No proprietary tools are required for the build.  For instance, 
>>>>>> C# compiled with Mono has been tested.  Therefore, the plugin has

>>>>>> been added to the default build and the default deployment.
>>>>>> 
>>>>>> The plugin includes unit and functional tests that can be 
>>>>>> triggered at build time.  See the link above for details.
>>>>>> 
>>>>>> The Apache header is applied to source and where feasible to 
>>>>>> build config files.
>>>>>> 
>>>>>> The source is the
>>>>>> https://github.com/lafferty/cloudstack/tree/hyperv_plugin
>>>>>> 
>>>>>> Finally, the review request is at 
>>>>>> https://reviews.apache.org/r/13922/
>>>>>> 
>>>>>> Testing?
>>>>>> 
>>>>>> Integration test:  created zone with CIFS secondary storage, 
>>>>>> added Hyper-V host, registered templates, create/stop/start/destroy
VM.
>>>>>> Scripts for zone setup are available on request.
>>>>>> 
>>>>>> Functional & unit tests:  the Java versions are part of the build.
>>>>>> The C# versions were run inside Visual Studio's Test Explorer.
>> 
> 


Mime
View raw message