cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sailaja Mada (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CLOUDSTACK-6576) No Error Handling while deploying GPU unsupported instances on a GPU enabled host (Ex: Cent OS instance)
Date Mon, 05 May 2014 11:17:14 GMT
Sailaja Mada created CLOUDSTACK-6576:
----------------------------------------

             Summary: No Error Handling while deploying GPU unsupported instances on a GPU
enabled host (Ex: Cent OS instance)
                 Key: CLOUDSTACK-6576
                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-6576
             Project: CloudStack
          Issue Type: Bug
      Security Level: Public (Anyone can view this level - this is the default.)
          Components: Management Server, XenServer
    Affects Versions: 4.4.0
            Reporter: Sailaja Mada
            Priority: Minor


Steps:

1.  Install and configure Adv zone with GPU enabled XenServer 6.2.5 

2.  Try to deploy default Cent OS instance using GPU enabled offering( K2 - 200 profile) 

Observation:
1.  Xenserver does not support vGPU with Linux instances.   
2.  Cloudstack throws deployment failure after throwing lot of exceptions by XenServer.

Expected Results:

CloudStack should parse the failure provided by XS and provide proper deployment failure as
unsupported deployment.

2014-05-05 16:38:50,730 DEBUG [c.c.a.t.Request] (Work-Job-Executor-18:job-58/job-59 ctx-4a14e747)
Seq 1-7036029992836286502: Sending  { Cmd , MgmtId: 55355881446856, via: 1(cldstk-R720-66),
Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StartCommand":{"vm":{"id":9,"name":"i-3-9-VM","bootloader":"PyGrub","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":2147483648,"maxRam":2147483648,"arch":"x86_64","os":"CentOS
5.6 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"1439f05b06e154d2","params":{},"uuid":"00069b3f-abf0-4f0d-a12c-6ae9d1a3ea81","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"e92dcd30-3775-4762-bb83-b6c9d901644b","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e8ccf63a-577b-3378-9741-16ce76631992","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Sailaja/goletagpups1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Sailaja/goletagpups1/?ROLE=Primary&STOREUUID=e8ccf63a-577b-3378-9741-16ce76631992"}},"name":"ROOT-9","size":21474836480,"path":"8d273d72-877c-4776-b4b5-9e52072ea45f","volumeId":11,"vmName":"i-3-9-VM","accountId":3,"format":"VHD","id":11,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"8d273d72-877c-4776-b4b5-9e52072ea45f","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"91973ed0-97f6-47bf-94ea-25751da81ae9","ip":"10.1.1.220","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:03:4b:00:07","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://770","isolationUri":"vlan://770","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group
of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID K200"}},"hostIp":"10.102.192.66","executeInSequence":false,"wait":0}}]
}
2014-05-05 16:38:50,732 DEBUG [c.c.a.t.Request] (Work-Job-Executor-18:job-58/job-59 ctx-4a14e747)
Seq 1-7036029992836286502: Executing:  { Cmd , MgmtId: 55355881446856, via: 1(cldstk-R720-66),
Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StartCommand":{"vm":{"id":9,"name":"i-3-9-VM","bootloader":"PyGrub","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":2147483648,"maxRam":2147483648,"arch":"x86_64","os":"CentOS
5.6 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"1439f05b06e154d2","params":{},"uuid":"00069b3f-abf0-4f0d-a12c-6ae9d1a3ea81","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"e92dcd30-3775-4762-bb83-b6c9d901644b","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e8ccf63a-577b-3378-9741-16ce76631992","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Sailaja/goletagpups1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Sailaja/goletagpups1/?ROLE=Primary&STOREUUID=e8ccf63a-577b-3378-9741-16ce76631992"}},"name":"ROOT-9","size":21474836480,"path":"8d273d72-877c-4776-b4b5-9e52072ea45f","volumeId":11,"vmName":"i-3-9-VM","accountId":3,"format":"VHD","id":11,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"8d273d72-877c-4776-b4b5-9e52072ea45f","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"91973ed0-97f6-47bf-94ea-25751da81ae9","ip":"10.1.1.220","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:03:4b:00:07","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://770","isolationUri":"vlan://770","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group
of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID K200"}},"hostIp":"10.102.192.66","executeInSequence":false,"wait":0}}]
}
2014-05-05 16:38:50,732 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-296:ctx-251a7963)
Seq 1-7036029992836286502: Executing request
2014-05-05 16:38:50,743 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
1. The VM i-3-9-VM is in Starting state.
2014-05-05 16:38:50,780 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
Created VM 2b2d13d8-42ea-301d-6b65-0ad093195826 for i-3-9-VM
2014-05-05 16:38:50,794 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
Creating VGPU for of VGPU type: GRID K200 in GPU group Group of NVIDIA Corporation GK104GL
[GRID K2] GPUs for VM i-3-9-VM
2014-05-05 16:38:50,794 DEBUG [c.c.h.x.r.XenServer620SP1Resource] (DirectAgent-296:ctx-251a7963)
Creating VGPU of VGPU type [ GRID K200 ] in gpu groupGroup of NVIDIA Corporation GK104GL [GRID
K2] GPUs for VM i-3-9-VM
2014-05-05 16:38:50,818 DEBUG [c.c.h.x.r.XenServer620SP1Resource] (DirectAgent-296:ctx-251a7963)
Created VGPU of VGPU type [ GRID K200 ] for VM i-3-9-VM
2014-05-05 16:38:50,954 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
VBD 74e87e65-0066-f400-eb20-e322c0c64ebe created for com.cloud.agent.api.to.DiskTO@61ba9994
2014-05-05 16:38:50,975 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
VBD d4e27c31-8ef4-49ca-661c-712f83568a03 created for com.cloud.agent.api.to.DiskTO@21b8f5ac
2014-05-05 16:38:50,975 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
Creating VIF for i-3-9-VM on nic [Nic:Guest-10.1.1.220-vlan://770]
2014-05-05 16:38:51,000 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
Found a network called VLAN-0a144be7-8521-934e-ff53-c62c59e75d69-770 on host=10.102.192.66;
 Network=fa62257b-e53e-1e82-69f0-963e63f853e0; pif=4fb4b5fd-c11c-5827-8198-8861506bda56
2014-05-05 16:38:51,011 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
Created a vif c6b0df48-0a58-9640-5d23-302cf3c3d7b4 on 0
2014-05-05 16:38:52,033 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
Task failed! Task record:                 uuid: fe8bd262-4b47-9a13-ac8e-1c2cafbea4fb
           nameLabel: Async.VM.start_on
     nameDescription:
   allowedOperations: []
   currentOperations: {}
             created: Mon May 05 15:28:17 IST 2014
status: failure
          residentOn: com.xensource.xenapi.Host@be7936ed
            progress: 1.0
                type: <none/>
              result:
           errorInfo: [FEATURE_REQUIRES_HVM, vGPU- and GPU-passthrough needs HVM]
         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

2014-05-05 16:38:52,041 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
Unable to start VM(i-3-9-VM) on host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed!
Task record:                 uuid: fe8bd262-4b47-9a13-ac8e-1c2cafbea4fb
           nameLabel: Async.VM.start_on
     nameDescription:
   allowedOperations: []
   currentOperations: {}
             created: Mon May 05 15:28:17 IST 2014
            finished: Mon May 05 15:28:17 IST 2014
              status: failure
          residentOn: com.xensource.xenapi.Host@be7936ed
            progress: 1.0
                type: <none/>
              result:
           errorInfo: [FEATURE_REQUIRES_HVM, vGPU- and GPU-passthrough needs HVM]
         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

Task failed! Task record:                 uuid: fe8bd262-4b47-9a13-ac8e-1c2cafbea4fb
           nameLabel: Async.VM.start_on
     nameDescription:
   allowedOperations: []
   currentOperations: {}
             created: Mon May 05 15:28:17 IST 2014
            finished: Mon May 05 15:28:17 IST 2014
              status: failure
          residentOn: com.xensource.xenapi.Host@be7936ed
            progress: 1.0
                type: <none/>
              result:
           errorInfo: [FEATURE_REQUIRES_HVM, vGPU- and GPU-passthrough needs HVM]
         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []
 at com.cloud.hypervisor.xen.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:3346)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:3458)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1812)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:500)
        at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:60)
        at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:93)
        at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:64)
        at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
2014-05-05 16:38:52,046 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
unable to destroy task(com.xensource.xenapi.Task@d080c3e6) on host(76520b97-5ce8-4c71-9572-b0834b62dc35)
due to You gave an invalid object reference.  The object may have recently been deleted. 
The class parameter gives the type of reference given, and the handle parameter echoes the
bad value given.
2014-05-05 16:38:52,046 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
Catch Exception: class com.cloud.utils.exception.CloudRuntimeException due to com.cloud.utils.exception.CloudRuntimeException:
Unable to start VM(i-3-9-VM) on host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed!
Task record:                 uuid: fe8bd262-4b47-9a13-ac8e-1c2cafbea4fb
           nameLabel: Async.VM.start_on
     nameDescription:
   allowedOperations: []
   currentOperations: {}
             created: Mon May 05 15:28:17 IST 2014
            finished: Mon May 05 15:28:17 IST 2014
              status: failure
          residentOn: com.xensource.xenapi.Host@be7936ed
            progress: 1.0
                type: <none/>
              result:
           errorInfo: [FEATURE_REQUIRES_HVM, vGPU- and GPU-passthrough needs HVM]
         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

com.cloud.utils.exception.CloudRuntimeException: Unable to start VM(i-3-9-VM) on host(76520b97-5ce8-4c71-9572-b0834b62dc35)
due to Task failed! Task record:                 uuid: fe8bd262-4b47-9a13-ac8e-1c2cafbea4fb
 nameLabel: Async.VM.start_on
     nameDescription:
   allowedOperations: []
   currentOperations: {}
             created: Mon May 05 15:28:17 IST 2014
            finished: Mon May 05 15:28:17 IST 2014
              status: failure
          residentOn: com.xensource.xenapi.Host@be7936ed
            progress: 1.0
                type: <none/>
              result:
           errorInfo: [FEATURE_REQUIRES_HVM, vGPU- and GPU-passthrough needs HVM]
         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:3477)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1812)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:500)
        at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:60)
        at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:93)
        at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:64)
        at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
2014-05-05 16:38:52,048 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
Unable to start i-3-9-VM due to
com.cloud.utils.exception.CloudRuntimeException: Unable to start VM(i-3-9-VM) on host(76520b97-5ce8-4c71-9572-b0834b62dc35)
due to Task failed! Task record:                 uuid: fe8bd262-4b47-9a13-ac8e-1c2cafbea4fb
           nameLabel: Async.VM.start_on
     nameDescription:
   allowedOperations: []
   currentOperations: {}
             created: Mon May 05 15:28:17 IST 2014
            finished: Mon May 05 15:28:17 IST 2014
              status: failure
          residentOn: com.xensource.xenapi.Host@be7936ed
            progress: 1.0
                type: <none/>
              result:
 errorInfo: [FEATURE_REQUIRES_HVM, vGPU- and GPU-passthrough needs HVM]
         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:3477)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1812)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:500)
        at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:60)
        at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:93)
        at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:64)
        at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
2014-05-05 16:38:52,060 DEBUG [c.c.a.ApiServlet] (catalina-exec-13:ctx-1d7ac3bf) ===START===
 10.144.6.6 -- GET  command=queryAsyncJobResult&jobId=48d1d975-c105-4823-99e1-6fc5f51f180e&response=json&sessionkey=TDXLIOmV8uKaEsb9Mr5o96exjw0%3D&_=1399288191562
2014-05-05 16:38:52,083 DEBUG [c.c.a.ApiServlet] (catalina-exec-13:ctx-1d7ac3bf ctx-6f979a7b)
===END===  10.144.6.6 -- GET  command=queryAsyncJobResult&jobId=48d1d975-c105-4823-99e1-6fc5f51f180e&response=json&sessionkey=TDXLIOmV8uKaEsb9Mr5o96exjw0%3D&_=1399288191562
2014-05-05 16:38:52,105 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
Unable to clean up VBD due to
You gave an invalid object reference.  The object may have recently been deleted.  The class
parameter gives the type of reference given, and the handle parameter echoes the bad value
given.
        at com.xensource.xenapi.Types.checkResponse(Types.java:693)
        at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
        at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
        at com.xensource.xenapi.VBD.unplug(VBD.java:1059)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1542)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1892)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:500)
        at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:60)
        at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:93)
        at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:64)
        at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
2014-05-05 16:38:52,112 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
Unable to clean up VBD due to
You gave an invalid object reference.  The object may have recently been deleted.  The class
parameter gives the type of reference given, and the handle parameter echoes the bad value
given.
        at com.xensource.xenapi.Types.checkResponse(Types.java:693)
        at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
        at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
        at com.xensource.xenapi.VBD.unplug(VBD.java:1059)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1542)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1892)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:500)
        at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:60)
        at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:93)
        at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:64)
        at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
2014-05-05 16:38:52,119 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
Unable to cleanup VIF
You gave an invalid object reference.  The object may have recently been deleted.  The class
parameter gives the type of reference given, and the handle parameter echoes the bad value
given.
        at com.xensource.xenapi.Types.checkResponse(Types.java:693)
        at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
        at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
        at com.xensource.xenapi.VIF.unplug(VIF.java:878)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1550)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1892)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:500)
        at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:60)
        at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:93)
        at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:64)
        at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
2014-05-05 16:38:52,137 INFO  [c.c.h.x.r.XenServer56Resource] (DirectAgent-296:ctx-251a7963)
Catch com.xensource.xenapi.Types$VifInUse: failed to destory VLAN eth0 on host 76520b97-5ce8-4c71-9572-b0834b62dc35
due to Network has active VIFs
2014-05-05 16:38:52,137 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963)
The VM is in stopped state, detected problem during startup : i-3-9-VM
2014-05-05 16:38:52,137 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-296:ctx-251a7963)
Seq 1-7036029992836286502: Response Received:
2014-05-05 16:38:52,139 DEBUG [c.c.a.t.Request] (DirectAgent-296:ctx-251a7963) Seq 1-7036029992836286502:
Processing:  { Ans: , MgmtId: 55355881446856, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":9,"name":"i-3-9-VM","bootloader":"PyGrub","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":2147483648,"maxRam":2147483648,"arch":"x86_64","os":"CentOS
5.6 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"1439f05b06e154d2","params":{},"uuid":"00069b3f-abf0-4f0d-a12c-6ae9d1a3ea81","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"e92dcd30-3775-4762-bb83-b6c9d901644b","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e8ccf63a-577b-3378-9741-16ce76631992","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Sailaja/goletagpups1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Sailaja/goletagpups1/?ROLE=Primary&STOREUUID=e8ccf63a-577b-3378-9741-16ce76631992"}},"name":"ROOT-9","size":21474836480,"path":"8d273d72-877c-4776-b4b5-9e52072ea45f","volumeId":11,"vmName":"i-3-9-VM","accountId":3,"format":"VHD","id":11,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"8d273d72-877c-4776-b4b5-9e52072ea45f","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"91973ed0-97f6-47bf-94ea-25751da81ae9","ip":"10.1.1.220","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:03:4b:00:07","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://770","isolationUri":"vlan://770","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group
of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID K200","groupDetails":{"Group
of NVIDIA Corporation GK104GL [GRID K2] GPUs":{"GRID K200":{"modelName":"GRID K200","maxHeads":2,"videoRam":268435456,"maxResolutionX":1920,"maxResolutionY":1200,"maxVgpuPerGpu":8,"remainingCapacity":16,"maxCapacity":16},"passthrough":{"modelName":"passthrough","maxHeads":0,"videoRam":0,"maxResolutionX":0,"maxResolutionY":0,"maxVgpuPerGpu":1,"remainingCapacity":2,"maxCapacity":2},"GRID
K260Q":{"modelName":"GRID K260Q","maxHeads":4,"videoRam":2013265920,"maxResolutionX":2560,"maxResolutionY":1600,"maxVgpuPerGpu":2,"remainingCapacity":4,"maxCapacity":4},"GRID
K240Q":{"modelName":"GRID K240Q","maxHeads":2,"videoRam":1006632960,"maxResolutionX":2560,"maxResolutionY":1600,"maxVgpuPerGpu":4,"remainingCapacity":8,"maxCapacity":8}}}}},"_iqnToPath":{},"result":false,"details":"Unable
to start i-3-9-VM due to ","wait":0}}] }
2014-05-05 16:38:52,139 DEBUG [c.c.a.t.Request] (Work-Job-Executor-18:job-58/job-59 ctx-4a14e747)
Seq 1-7036029992836286502: Received:  { Ans: , MgmtId: 55355881446856, via: 1, Ver: v1, Flags:
10, { StartAnswer } }
2014-05-05 16:38:52,142 INFO  [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-18:job-58/job-59
ctx-4a14e747) Unable to start VM on Host[-1-Routing] due to Unable to start i-3-9-VM due to
2014-05-05 16:38:52,145 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-18:job-58/job-59
ctx-4a14e747) Cleaning up resources for the vm VM[User|i-3-9-VM] in Starting state
2014-05-05 16:38:52,148 DEBUG [c.c.a.t.Request] (Work-Job-Executor-18:job-58/job-59 ctx-4a14e747)
Seq 1-7036029992836286503: Sending  { Cmd , MgmtId: 55355881446856, via: 1(cldstk-R720-66),
Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-3-9-VM","wait":0}}]
}
2014-05-05 16:38:52,148 DEBUG [c.c.a.t.Request] (Work-Job-Executor-18:job-58/job-59 ctx-4a14e747)
Seq 1-7036029992836286503: Executing:  { Cmd , MgmtId: 55355881446856, via: 1(cldstk-R720-66),
Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-3-9-VM","wait":0}}]
}
2014-05-05 16:38:52,224 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-18:job-58/job-59
ctx-4a14e747) Invocation exception, caused by: com.cloud.exception.InsufficientServerCapacityException:
Unable to create a deployment for VM[User|i-3-9-VM]Scope=interface com.cloud.dc.DataCenter;
id=1
2014-05-05 16:38:52,225 INFO  [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-18:job-58/job-59
ctx-4a14e747) Rethrow exception com.cloud.exception.InsufficientServerCapacityException: Unable
to create a deployment for VM[User|i-3-9-VM]Scope=interface com.cloud.dc.DataCenter; id=1
2014-05-05 16:38:52,226 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-18:job-58/job-59)
Done with run of VM work job: com.cloud.vm.VmWorkStart for VM 9, job origin: 58
2014-05-05 16:38:52,226 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-18:job-58/job-59)
Unable to complete AsyncJobVO {id:59, userId: 3, accountId: 3, instanceType: null, instanceId:
null, cmd: com.cloud.vm.VmWorkStart, cmdInfo: rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAADAAAAAAAAAAMAAAAAAAAACXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAXBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAAFxAH4ACnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
55355881446856, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon May
05 16:38:46 IST 2014}, job origin:58
com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for
VM[User|i-3-9-VM]Scope=interface com.cloud.dc.DataCenter; id=1
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:943)
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5167)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
        at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5312)
        at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:495)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:452)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
2014-05-05 16:38:52,228 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-18:job-58/job-59)
Complete async job-59, jobStatus: FAILED, resultCode: 0, result: rO0ABXNyABpqYXZhLmxhbmcuUnVudGltZUV4Y2VwdGlvbp5fBkcKNIPlAgAAeHIAE2phdmEubGFuZy5FeGNlcHRpb27Q_R8-GjscxAIAAHhyABNqYXZhLmxhbmcuVGhyb3dhYmxl1cY1Jzl3uMsDAARMAAVjYXVzZXQAFUxqYXZhL2xhbmcvVGhyb3dhYmxlO0wADWRldGFpbE1lc3NhZ2V0ABJMamF2YS9sYW5nL1N0cmluZztbAApzdGFja1RyYWNldAAeW0xqYXZhL2xhbmcvU3RhY2tUcmFjZUVsZW1lbnQ7TAAUc3VwcHJlc3NlZEV4Y2VwdGlvbnN0ABBMamF2YS91dGlsL0xpc3Q7eHBxAH4AB3QAT0pvYiBmYWlsZWQgZHVlIHRvIGV4Y2VwdGlvbiBVbmFibGUgdG8gY3JlYXRlIGEgZGVwbG95bWVudCBmb3IgVk1bVXNlcnxpLTMtOS1WTV11cgAeW0xqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnQ7AkYqPDz9IjkCAAB4cAAAAA1zcgAbamF2YS5sYW5nLlN0YWNrVHJhY2VFbGVtZW50YQnFmiY23YUCAARJAApsaW5lTnVtYmVyTAAOZGVjbGFyaW5nQ2xhc3NxAH4ABEwACGZpbGVOYW1lcQB-AARMAAptZXRob2ROYW1lcQB-AAR4cAAAAHJ0ACBjb20uY2xvdWQudm0uVm1Xb3JrSm9iRGlzcGF0Y2hlcnQAGFZtV29ya0pvYkRpc3BhdGNoZXIuamF2YXQABnJ1bkpvYnNxAH4ACwAAAe90AD9vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZnJhbWV3b3JrLmpvYnMuaW1wbC5Bc3luY0pvYk1hbmFnZXJJbXBsJDV0ABhBc3luY0pvYk1hbmFnZXJJbXBsLmphdmF0AAxydW5JbkNvbnRleHRzcQB-AAsAAAAxdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1bm5hYmxlLmphdmF0AANydW5zcQB-AAsAAAA4dABCb3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5pbXBsLkRlZmF1bHRNYW5hZ2VkQ29udGV4dCQxdAAaRGVmYXVsdE1hbmFnZWRDb250ZXh0LmphdmF0AARjYWxsc3EAfgALAAAAZ3QAQG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHRxAH4AGnQAD2NhbGxXaXRoQ29udGV4dHNxAH4ACwAAADVxAH4AHXEAfgAadAAOcnVuV2l0aENvbnRleHRzcQB-AAsAAAAudAA8b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlcQB-ABZxAH4AF3NxAH4ACwAAAcRxAH4AEXEAfgAScQB-ABdzcQB-AAsAAAHXdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuRXhlY3V0b3JzJFJ1bm5hYmxlQWRhcHRlcnQADkV4ZWN1dG9ycy5qYXZhcQB-ABtzcQB-AAsAAAEGdAAfamF2YS51dGlsLmNvbmN1cnJlbnQuRnV0dXJlVGFza3QAD0Z1dHVyZVRhc2suamF2YXEAfgAXc3EAfgALAAAEeXQAJ2phdmEudXRpbC5jb25jdXJyZW50LlRocmVhZFBvb2xFeGVjdXRvcnQAF1RocmVhZFBvb2xFeGVjdXRvci5qYXZhdAAJcnVuV29ya2Vyc3EAfgALAAACZ3QALmphdmEudXRpbC5jb25jdXJyZW50LlRocmVhZFBvb2xFeGVjdXRvciRXb3JrZXJxAH4ALHEAfgAXc3EAfgALAAAC6HQAEGphdmEubGFuZy5UaHJlYWR0AAtUaHJlYWQuamF2YXEAfgAXc3IAJmphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVMaXN0_A8lMbXsjhACAAFMAARsaXN0cQB-AAZ4cgAsamF2YS51dGlsLkNvbGxlY3Rpb25zJFVubW9kaWZpYWJsZUNvbGxlY3Rpb24ZQgCAy173HgIAAUwAAWN0ABZMamF2YS91dGlsL0NvbGxlY3Rpb247eHBzcgATamF2YS51dGlsLkFycmF5TGlzdHiB0h2Zx2GdAwABSQAEc2l6ZXhwAAAAAHcEAAAAAHhxAH4AOHg


CCP should 







--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message