cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Vazquez <nicovazque...@gmail.com>
Subject Re: :[VOTE] Apache Cloudstack 4.10.0.0
Date Wed, 08 Mar 2017 20:45:05 GMT
Hi Mike,

Thanks for reporting this issue! We've been working on it and created this
PR: https://github.com/apache/cloudstack/pull/1994. We've been testing it
and seems to be solving the issues reported, can you please repeat your
tests on it?

Thanks,
Nicolas

2017-03-07 21:52 GMT-03:00 Tutkowski, Mike <Mike.Tutkowski@netapp.com>:

> Here’s the ticket:
>
> https://issues.apache.org/jira/browse/CLOUDSTACK-9827
>
> On 3/7/17, 1:51 PM, "Tutkowski, Mike" <Mike.Tutkowski@netapp.com> wrote:
>
>     Yes, I can open a ticket.
>
>     > On Mar 7, 2017, at 1:50 PM, Sergey Levitskiy <
> Sergey.Levitskiy@autodesk.com> wrote:
>     >
>     > Yes that’s the bug. Can you open a ticket for this? @ nvazquez can
> you take a look?
>     >
>     > On 3/7/17, 12:44 PM, "Tutkowski, Mike" <Mike.Tutkowski@netapp.com>
> wrote:
>     >
>     >    This does seem messed up.
>     >
>     >    If I add a new primary storage and give it a storage tag, the tag
> ends up in storage_pool_details.
>     >
>     >    If I edit an existing storage pool’s storage tags, it places them
> in storage_pool_tags.
>     >
>     >    On 3/7/17, 1:39 PM, "Tutkowski, Mike" <Mike.Tutkowski@netapp.com>
> wrote:
>     >
>     >        I believe I have found another bug (one that we should either
> fix or examine in detail before releasing 4.10).
>     >
>     >        It looks like we have a new table: cloud.storage_pool_tags.
>     >
>     >        The addition of this table seems to have broken the
> listStorageTags API command. When this command runs, it doesn’t pick up any
> storage tags for me (and I know I have one storage tag).
>     >
>     >        This data used to be stored in the cloud.storage_pool_details
> table. It’s good to put it in its own table, but will our upgrade process
> move the existing tags from storage_pool_details to storage_pool_tags?
>     >
>     >        I have not yet opened a ticket for this. I want to examine it
> a bit more before doing so.
>     >
>     >        On 3/7/17, 8:10 AM, "Tutkowski, Mike" <
> Mike.Tutkowski@netapp.com> wrote:
>     >
>     >            No VM snapshot.
>     >
>     >            I tried while the VM was in the Running state and then I
> also tried in the Stopped state. Same results.
>     >
>     >> On Mar 7, 2017, at 7:54 AM, Sergey Levitskiy <
> Sergey.Levitskiy@autodesk.com> wrote:
>     >>
>     >> Is VM has an VMsnaphsot? Is VM in Stopped state?
>     >>
>     >> On 3/6/17, 10:32 PM, "Tutkowski, Mike" <Mike.Tutkowski@netapp.com>
> wrote:
>     >>
>     >>   I seem to have found another blocker:
>     >>
>     >>   https://issues.apache.org/jira/browse/CLOUDSTACK-9822
>     >>
>     >>   On 3/6/17, 9:51 PM, "Rajani Karuturi" <rajani@apache.org> wrote:
>     >>
>     >>       PRs are ready for the blockers. Waiting for reviews and test
>     >>       results. Once they are ready, I will merge them(and a few more
>     >>       bug fixes) and create RC2 (probably tomorrow, Wednesday)
>     >>
>     >>       Thanks,
>     >>
>     >>       ~ Rajani
>     >>
>     >>       http://cloudplatform.accelerite.com/
>     >>
>     >>       On March 3, 2017 at 4:30 PM, Rajani Karuturi (
> rajani@apache.org)
>     >>       wrote:
>     >>
>     >>       I will create RC2 on Monday with the fixes mentioned in my
>     >>       previous mail.
>     >>
>     >>       ~ Rajani
>     >>
>     >>       http://cloudplatform.accelerite.com/
>     >>
>     >>       On March 3, 2017 at 2:36 PM, Rohit Yadav
>     >>       (rohit.yadav@shapeblue.com) wrote:
>     >>
>     >>       Thanks Koushik, I did not realize Kishan had sent this
> already.
>     >>       Let's get either of the PRs merged and kick a RC2.
>     >>
>     >>       Regards.
>     >>
>     >>       ________________________________
>     >>       From: Koushik Das <koushik.das@accelerite.com>
>     >>       Sent: 03 March 2017 14:14:56
>     >>       To: dev@cloudstack.apache.org
>     >>       Subject: Re: :[VOTE] Apache Cloudstack 4.10.0.0
>     >>
>     >>       Looks like there is already a PR for the same issue
>     >>       https://github.com/apache/cloudstack/pull/1982 from Kishan.
>     >>
>     >>       -Koushik
>     >>
>     >>       On 03/03/17, 1:58 PM, "Rohit Yadav" <
> rohit.yadav@shapeblue.com>
>     >>       wrote:
>     >>
>     >>       -1 (binding)
>     >>
>     >>       All, I've found an upgrade blocker. Pre 4.6 users are required
>     >>       to seed 4.6 systemvmtemplate to proceed with the upgrade
>     >>       otherwise upgrade fails, and from 4.9 upgrade to 4.10 does no
>     >>       check/enforcement that 4.10 based systemvmtemplate has been
>     >>       seeded/registered, nor the minimum required systemvmtemplate
>     >>       version is changed from 4.6.0 to 4.10.0.
>     >>
>     >>       After we have merged the strongswan/java8 PR, I had updated
> the
>     >>       upgrade docs on how to upgrade the systemvmtemplate here:
>     >>
>     >>       http://docs.cloudstack.apache.org/projects/cloudstack-
> release-notes/en/4.10/upgrade/upgrade-4.9.html
>     >>
>     >>       Using the above, I've tried to fix these issues here, please
>     >>       review and merge for RC2:
>     >>
>     >>       https://github.com/apache/cloudstack/pull/1983
>     >>
>     >>       <https://github.com/apache/cloudstack/pull/1983>With above
> fix,
>     >>       the aim is that users only seed the 4.10 systemvmtemplate
> before
>     >>       upgrade and post-upgrade the upgrade paths fix the entries,
>     >>       global setting etc.
>     >>
>     >>       Regards.
>     >>
>     >>       ________________________________
>     >>       From: Tutkowski, Mike <Mike.Tutkowski@netapp.com>
>     >>       Sent: 02 March 2017 22:39:08
>     >>       To: dev@cloudstack.apache.org
>     >>       Subject: Re: :[VOTE] Apache Cloudstack 4.10.0.0
>     >>
>     >>       I rolled back to my master branch at
>     >>       da66b06e7d562393da2e4b52206943f8bad49d10 and it works.
>     >>
>     >>       It appears something that went into after that commit has
> broken
>     >>       this. It looks like this SHA is about two weeks old and that
> 43
>     >>       commits have gone into master since it.
>     >>
>     >>       On 3/2/17, 7:06 AM, "Tutkowski, Mike"
>     >>       <Mike.Tutkowski@netapp.com> wrote:
>     >>
>     >>       According to where the code fails, though, it appears to be a
>     >>       networking problem. If I set a breakpoint before the failure
> and
>     >>       change a variable to say that security groups are not being
> used,
>     >>       then the VM starts.
>     >>
>     >>       I think this is a recently introduced problem because I have
>     >>       another branch based off of a slightly older version of master
>     >>       and it works fine here.
>     >>
>     >>> On Mar 2, 2017, at 6:51 AM, Pierre-Luc Dion
>     >>       <pdion@cloudops.com> wrote:
>     >>>
>     >>> Hi Mike,
>     >>> Try vm with at least 512MB for memory.
>     >>>
>     >>>> On Mar 1, 2017 15:01, "Tutkowski, Mike"
>     >>       <Mike.Tutkowski@netapp.com> wrote:
>     >>>>
>     >>>> I see the following exception when trying to deploy a user VM
>     >>       in a Basic
>     >>>> Zone with two XenServer 6.5 hosts in one cluster. My system
>     >>       VMs have all
>     >>>> deployed properly. The user template gets downloaded fine. I
>     >>       can see the
>     >>>> user VM begin to start on a XenServer host, then it goes
>     >>       away. We then
>     >>>> automatically try on the other host. I can see the VM begin
>     >>       to start there
>     >>>> for a moment, then it goes away.
>     >>>>
>     >>>> I am just deploying the user VM’s template and root disk to
>     >>       NFS (same
>     >>>> place where the template and root disks of my system VMs
>     >>       are).
>     >>>>
>     >>>> I am using the built-in XenServer CentOS 5.6 (64 bit)
>     >>       template with 1
>     >>>> vCPU, 500 MHz, and 256 MB memory.
>     >>>>
>     >>>> WARN [c.c.a.r.v.VirtualRoutingResource]
>     >>       (DirectAgent-7:ctx-35aded78)
>     >>>> (logid:aab9c320) Expected 1 answers while executing
>     >>       VmDataCommand but
>     >>>> received 2
>     >>>> WARN [c.c.v.VirtualMachinePowerStateSyncImpl]
>     >>       (DirectAgentCronJob-14:ctx-27fb1ac3)
>     >>>> (logid:2c342f23) VM state was updated but update time is
>     >>       null?! vm id: 6
>     >>>> INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>     >>       (AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce)
>     >>>> (logid:a56a9a8c) Begin cleanup expired async-jobs
>     >>>> INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>     >>       (AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce)
>     >>>> (logid:a56a9a8c) End cleanup expired async-jobs
>     >>>> INFO [c.c.u.AccountManagerImpl]
>     >>       (AccountChecker-1:ctx-383a632c)
>     >>>> (logid:541e9ba5) Found 0 removed accounts to cleanup
>     >>>> INFO [c.c.u.AccountManagerImpl]
>     >>       (AccountChecker-1:ctx-383a632c)
>     >>>> (logid:541e9ba5) Found 0 disabled accounts to cleanup
>     >>>> INFO [c.c.u.AccountManagerImpl]
>     >>       (AccountChecker-1:ctx-383a632c)
>     >>>> (logid:541e9ba5) Found 0 inactive domains to cleanup
>     >>>> INFO [c.c.u.AccountManagerImpl]
>     >>       (AccountChecker-1:ctx-383a632c)
>     >>>> (logid:541e9ba5) Found 0 disabled projects to cleanup
>     >>>> WARN [c.c.h.x.r.CitrixResourceBase]
>     >>       (DirectAgent-16:ctx-7c901443)
>     >>>> (logid:aab9c320) callHostPlugin failed for cmd:
>     >>       default_network_rules with
>     >>>> args secIps: 0:, vmName: i-2-6-VM, vmID: 6, vmIP:
>     >>       10.117.40.53, vmMAC:
>     >>>> 06:b2:f4:00:00:22, due to There was a failure communicating
>     >>       with the
>     >>>> plugin.
>     >>>> WARN [c.c.h.x.r.w.x.CitrixStartCommandWrapper]
>     >>>> (DirectAgent-16:ctx-7c901443) (logid:aab9c320) Catch
>     >>       Exception: class
>     >>>> com.cloud.utils.exception.CloudRuntimeException due to
>     >>>> com.cloud.utils.exception.CloudRuntimeException:
>     >>       callHostPlugin failed
>     >>>> for cmd: default_network_rules with args secIps: 0:, vmName:
>     >>       i-2-6-VM,
>     >>>> vmID: 6, vmIP: 10.117.40.53, vmMAC: 06:b2:f4:00:00:22, due to
>     >>       There was a
>     >>>> failure communicating with the plugin.
>     >>>> com.cloud.utils.exception.CloudRuntimeException:
>     >>       callHostPlugin failed
>     >>>> for cmd: default_network_rules with args secIps: 0:, vmName:
>     >>       i-2-6-VM,
>     >>>> vmID: 6, vmIP: 10.117.40.53, vmMAC: 06:b2:f4:00:00:22, due to
>     >>       There was a
>     >>>> failure communicating with the plugin.
>     >>>> at
>     >>       com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.
>     >>>> callHostPlugin(CitrixResourceBase.java:338)
>     >>>> at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.
>     >>>>
>     >>       CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.
> java:188)
>     >>>> at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.
>     >>>>
>     >>       CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.
> java:53)
>     >>>> at com.cloud.hypervisor.xenserver.resource.wrapper.
>     >>>>
>     >>       xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.
> java:122)
>     >>>> at
>     >>       com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.
>     >>>> executeRequest(CitrixResourceBase.java:1691)
>     >>>> at
>     >>       com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(
>     >>>> DirectAgentAttache.java:315)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.
> ManagedContextRunnable$1.run(
>     >>>> ManagedContextRunnable.java:49)
>     >>>> at org.apache.cloudstack.managed.context.impl.
>     >>>> DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext.
>     >>>> callWithContext(DefaultManagedContext.java:103)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext.
>     >>>> runWithContext(DefaultManagedContext.java:53)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.
> ManagedContextRunnable.run(
>     >>>> ManagedContextRunnable.java:46)
>     >>>> at java.util.concurrent.Executors$RunnableAdapter.
>     >>>> call(Executors.java:511)
>     >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     >>>> at java.util.concurrent.ScheduledThreadPoolExecutor$
>     >>>>
>     >>       ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.
> java:180)
>     >>>> at java.util.concurrent.ScheduledThreadPoolExecutor$
>     >>>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>     >>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(
>     >>>> ThreadPoolExecutor.java:1142)
>     >>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>     >>>> ThreadPoolExecutor.java:617)
>     >>>> at java.lang.Thread.run(Thread.java:745)
>     >>>> WARN [c.c.h.x.r.CitrixResourceBase]
>     >>       (DirectAgent-16:ctx-7c901443)
>     >>>> (logid:aab9c320) Unable to start i-2-6-VM due to
>     >>>> com.cloud.utils.exception.CloudRuntimeException:
>     >>       callHostPlugin failed
>     >>>> for cmd: default_network_rules with args secIps: 0:, vmName:
>     >>       i-2-6-VM,
>     >>>> vmID: 6, vmIP: 10.117.40.53, vmMAC: 06:b2:f4:00:00:22, due to
>     >>       There was a
>     >>>> failure communicating with the plugin.
>     >>>> at
>     >>       com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.
>     >>>> callHostPlugin(CitrixResourceBase.java:338)
>     >>>> at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.
>     >>>>
>     >>       CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.
> java:188)
>     >>>> at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.
>     >>>>
>     >>       CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.
> java:53)
>     >>>> at com.cloud.hypervisor.xenserver.resource.wrapper.
>     >>>>
>     >>       xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.
> java:122)
>     >>>> at
>     >>       com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.
>     >>>> executeRequest(CitrixResourceBase.java:1691)
>     >>>> at
>     >>       com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(
>     >>>> DirectAgentAttache.java:315)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.
> ManagedContextRunnable$1.run(
>     >>>> ManagedContextRunnable.java:49)
>     >>>> at org.apache.cloudstack.managed.context.impl.
>     >>>> DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext.
>     >>>> callWithContext(DefaultManagedContext.java:103)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext.
>     >>>> runWithContext(DefaultManagedContext.java:53)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.
> ManagedContextRunnable.run(
>     >>>> ManagedContextRunnable.java:46)
>     >>>> at java.util.concurrent.Executors$RunnableAdapter.
>     >>>> call(Executors.java:511)
>     >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     >>>> at java.util.concurrent.ScheduledThreadPoolExecutor$
>     >>>>
>     >>       ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.
> java:180)
>     >>>> at java.util.concurrent.ScheduledThreadPoolExecutor$
>     >>>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>     >>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(
>     >>>> ThreadPoolExecutor.java:1142)
>     >>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>     >>>> ThreadPoolExecutor.java:617)
>     >>>> at java.lang.Thread.run(Thread.java:745)
>     >>>> INFO [o.a.c.e.o.NetworkOrchestrator]
>     >>       (Network-Scavenger-1:ctx-2058d5ac)
>     >>>> (logid:bf8885a0) NetworkGarbageCollector uses '20' seconds
>     >>       for GC interval.
>     >>>> WARN [c.c.h.x.r.CitrixResourceBase]
>     >>       (DirectAgent-16:ctx-7c901443)
>     >>>> (logid:aab9c320) Unable to clean up VBD due to
>     >>>> You gave an invalid object reference. The object may have
>     >>       recently been
>     >>>> deleted. The class parameter gives the type of reference
>     >>       given, and the
>     >>>> handle parameter echoes the bad value given.
>     >>>> at com.xensource.xenapi.Types.checkResponse(Types.java:693)
>     >>>> at
>     >>       com.xensource.xenapi.Connection.dispatch(Connection.java:395)
>     >>>> at
>     >>       com.cloud.hypervisor.xenserver.resource.
> XenServerConnectionPool$
>     >>>>
>     >>       XenServerConnection.dispatch(XenServerConnectionPool.java:
> 457)
>     >>>> at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
>     >>>> at
>     >>       com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.
>     >>>> handleVmStartFailure(CitrixResourceBase.java:3576)
>     >>>> at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.
>     >>>>
>     >>       CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.
> java:210)
>     >>>> at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.
>     >>>>
>     >>       CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.
> java:53)
>     >>>> at com.cloud.hypervisor.xenserver.resource.wrapper.
>     >>>>
>     >>       xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.
> java:122)
>     >>>> at
>     >>       com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.
>     >>>> executeRequest(CitrixResourceBase.java:1691)
>     >>>> at
>     >>       com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(
>     >>>> DirectAgentAttache.java:315)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.
> ManagedContextRunnable$1.run(
>     >>>> ManagedContextRunnable.java:49)
>     >>>> at org.apache.cloudstack.managed.context.impl.
>     >>>> DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext.
>     >>>> callWithContext(DefaultManagedContext.java:103)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext.
>     >>>> runWithContext(DefaultManagedContext.java:53)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.
> ManagedContextRunnable.run(
>     >>>> ManagedContextRunnable.java:46)
>     >>>> at java.util.concurrent.Executors$RunnableAdapter.
>     >>>> call(Executors.java:511)
>     >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     >>>> at java.util.concurrent.ScheduledThreadPoolExecutor$
>     >>>>
>     >>       ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.
> java:180)
>     >>>> at java.util.concurrent.ScheduledThreadPoolExecutor$
>     >>>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>     >>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(
>     >>>> ThreadPoolExecutor.java:1142)
>     >>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>     >>>> ThreadPoolExecutor.java:617)
>     >>>> at java.lang.Thread.run(Thread.java:745)
>     >>>> WARN [c.c.h.x.r.CitrixResourceBase]
>     >>       (DirectAgent-16:ctx-7c901443)
>     >>>> (logid:aab9c320) Unable to clean up VBD due to
>     >>>> You gave an invalid object reference. The object may have
>     >>       recently been
>     >>>> deleted. The class parameter gives the type of reference
>     >>       given, and the
>     >>>> handle parameter echoes the bad value given.
>     >>>> at com.xensource.xenapi.Types.checkResponse(Types.java:693)
>     >>>> at
>     >>       com.xensource.xenapi.Connection.dispatch(Connection.java:395)
>     >>>> at
>     >>       com.cloud.hypervisor.xenserver.resource.
> XenServerConnectionPool$
>     >>>>
>     >>       XenServerConnection.dispatch(XenServerConnectionPool.java:
> 457)
>     >>>> at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
>     >>>> at
>     >>       com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.
>     >>>> handleVmStartFailure(CitrixResourceBase.java:3576)
>     >>>> at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.
>     >>>>
>     >>       CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.
> java:210)
>     >>>> at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.
>     >>>>
>     >>       CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.
> java:53)
>     >>>> at com.cloud.hypervisor.xenserver.resource.wrapper.
>     >>>>
>     >>       xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.
> java:122)
>     >>>> at
>     >>       com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.
>     >>>> executeRequest(CitrixResourceBase.java:1691)
>     >>>> at
>     >>       com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(
>     >>>> DirectAgentAttache.java:315)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.
> ManagedContextRunnable$1.run(
>     >>>> ManagedContextRunnable.java:49)
>     >>>> at org.apache.cloudstack.managed.context.impl.
>     >>>> DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext.
>     >>>> callWithContext(DefaultManagedContext.java:103)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext.
>     >>>> runWithContext(DefaultManagedContext.java:53)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.
> ManagedContextRunnable.run(
>     >>>> ManagedContextRunnable.java:46)
>     >>>> at java.util.concurrent.Executors$RunnableAdapter.
>     >>>> call(Executors.java:511)
>     >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     >>>> at java.util.concurrent.ScheduledThreadPoolExecutor$
>     >>>>
>     >>       ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.
> java:180)
>     >>>> at java.util.concurrent.ScheduledThreadPoolExecutor$
>     >>>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>     >>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(
>     >>>> ThreadPoolExecutor.java:1142)
>     >>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>     >>>> ThreadPoolExecutor.java:617)
>     >>>> at java.lang.Thread.run(Thread.java:745)
>     >>>> WARN [c.c.h.x.r.CitrixResourceBase]
>     >>       (DirectAgent-16:ctx-7c901443)
>     >>>> (logid:aab9c320) Unable to cleanup VIF
>     >>>> You gave an invalid object reference. The object may have
>     >>       recently been
>     >>>> deleted. The class parameter gives the type of reference
>     >>       given, and the
>     >>>> handle parameter echoes the bad value given.
>     >>>> at com.xensource.xenapi.Types.checkResponse(Types.java:693)
>     >>>> at
>     >>       com.xensource.xenapi.Connection.dispatch(Connection.java:395)
>     >>>> at
>     >>       com.cloud.hypervisor.xenserver.resource.
> XenServerConnectionPool$
>     >>>>
>     >>       XenServerConnection.dispatch(XenServerConnectionPool.java:
> 457)
>     >>>> at com.xensource.xenapi.VIF.unplug(VIF.java:921)
>     >>>> at
>     >>       com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.
>     >>>> handleVmStartFailure(CitrixResourceBase.java:3584)
>     >>>> at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.
>     >>>>
>     >>       CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.
> java:210)
>     >>>> at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.
>     >>>>
>     >>       CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.
> java:53)
>     >>>> at com.cloud.hypervisor.xenserver.resource.wrapper.
>     >>>>
>     >>       xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.
> java:122)
>     >>>> at
>     >>       com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.
>     >>>> executeRequest(CitrixResourceBase.java:1691)
>     >>>> at
>     >>       com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(
>     >>>> DirectAgentAttache.java:315)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.
> ManagedContextRunnable$1.run(
>     >>>> ManagedContextRunnable.java:49)
>     >>>> at org.apache.cloudstack.managed.context.impl.
>     >>>> DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext.
>     >>>> callWithContext(DefaultManagedContext.java:103)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext.
>     >>>> runWithContext(DefaultManagedContext.java:53)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.
> ManagedContextRunnable.run(
>     >>>> ManagedContextRunnable.java:46)
>     >>>> at java.util.concurrent.Executors$RunnableAdapter.
>     >>>> call(Executors.java:511)
>     >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     >>>> at java.util.concurrent.ScheduledThreadPoolExecutor$
>     >>>>
>     >>       ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.
> java:180)
>     >>>> at java.util.concurrent.ScheduledThreadPoolExecutor$
>     >>>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>     >>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(
>     >>>> ThreadPoolExecutor.java:1142)
>     >>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>     >>>> ThreadPoolExecutor.java:617)
>     >>>> at java.lang.Thread.run(Thread.java:745)
>     >>>> INFO [c.c.v.VirtualMachineManagerImpl]
>     >>       (Work-Job-Executor-2:ctx-bc104380
>     >>>> job-25/job-27 ctx-8c946a5b) (logid:aab9c320) Unable to start
>     >>       VM on
>     >>>> Host[-2-Routing] due to Unable to start i-2-6-VM due to
>     >>>> ERROR [c.c.v.VmWorkJobHandlerProxy]
>     >>       (Work-Job-Executor-2:ctx-bc104380
>     >>>> job-25/job-27 ctx-8c946a5b) (logid:aab9c320) Invocation
>     >>       exception, caused
>     >>>> by: com.cloud.exception.InsufficientServerCapacityException:
>     >>       Unable to
>     >>>> create a deployment for VM[User|i-2-6-VM]Scope=interface
>     >>>> com.cloud.dc.DataCenter; id=1
>     >>>> INFO [c.c.v.VmWorkJobHandlerProxy]
>     >>       (Work-Job-Executor-2:ctx-bc104380
>     >>>> job-25/job-27 ctx-8c946a5b) (logid:aab9c320) Rethrow
>     >>       exception
>     >>>> com.cloud.exception.InsufficientServerCapacityException:
>     >>       Unable to create
>     >>>> a deployment for VM[User|i-2-6-VM]Scope=interface
>     >>>> com.cloud.dc.DataCenter; id=1
>     >>>> ERROR [c.c.v.VmWorkJobDispatcher]
>     >>       (Work-Job-Executor-2:ctx-bc104380
>     >>>> job-25/job-27) (logid:aab9c320) Unable to complete AsyncJobVO
>     >>       {id:27,
>     >>>> userId: 2, accountId: 2, instanceType: null, instanceId:
>     >>       null, cmd:
>     >>>> com.cloud.vm.VmWorkStart, cmdInfo:
>     >>       rO0ABXNyABhjb20uY2xvdWQudm0uVm
>     >>>> 1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2
>     >>>> Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAA
>     >>>> ljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-
>     >>>> AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNp
>     >>>>
>     >>       Y2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-
> AANMAAVwb2RJZHEAfgACTAAGcG9vbE
>     >>>>
>     >>       lkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS
> 91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-
>     >>>>
>     >>       AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-
> ZtlbwJWdrAgAESgAJYWNjb3VudElkS
>     >>>>
>     >>       gAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-
> AAN4cAAAAAAAAAACAAAAAAAAAAIAAA
>     >>>> AAAAAABnQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcH
>     >>>> BwcHBwc3IAEWphdmEudXRpbC5IYXNoTWFwBQfawcMWYNEDAAJGAApsb2FkRm
>     >>>> FjdG9ySQAJdGhyZXNob2xkeHA_QAAAAAAADHcIAAAAEAAAAAF0AApWbV
>     >>>> Bhc3N3b3JkdAAcck8wQUJYUUFEbk5oZG1Wa1gzQmhjM04zYjNKa3hw,
>     >>       cmdVersion: 0,
>     >>>> status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
>     >>       null,
>     >>>> initMsid: 52237617797, completeMsid: null, lastUpdated: null,
>     >>       lastPolled:
>     >>>> null, created: Wed Mar 01 12:51:32 MST 2017}, job origin:25
>     >>>> com.cloud.exception.InsufficientServerCapacityException:
>     >>       Unable to create
>     >>>> a deployment for VM[User|i-2-6-VM]Scope=interface
>     >>>> com.cloud.dc.DataCenter; id=1
>     >>>> at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(
>     >>>> VirtualMachineManagerImpl.java:961)
>     >>>> at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(
>     >>>> VirtualMachineManagerImpl.java:4661)
>     >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>     >>       Method)
>     >>>> at sun.reflect.NativeMethodAccessorImpl.invoke(
>     >>>> NativeMethodAccessorImpl.java:62)
>     >>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>     >>>> DelegatingMethodAccessorImpl.java:43)
>     >>>> at java.lang.reflect.Method.invoke(Method.java:498)
>     >>>> at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(
>     >>>> VmWorkJobHandlerProxy.java:107)
>     >>>> at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(
>     >>>> VirtualMachineManagerImpl.java:4822)
>     >>>> at com.cloud.vm.VmWorkJobDispatcher.runJob(
>     >>>> VmWorkJobDispatcher.java:102)
>     >>>> at
>     >>       org.apache.cloudstack.framework.jobs.impl.
> AsyncJobManagerImpl$5.
>     >>>> runInContext(AsyncJobManagerImpl.java:554)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.
> ManagedContextRunnable$1.run(
>     >>>> ManagedContextRunnable.java:49)
>     >>>> at org.apache.cloudstack.managed.context.impl.
>     >>>> DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext.
>     >>>> callWithContext(DefaultManagedContext.java:103)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext.
>     >>>> runWithContext(DefaultManagedContext.java:53)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.
> ManagedContextRunnable.run(
>     >>>> ManagedContextRunnable.java:46)
>     >>>> at org.apache.cloudstack.framework.jobs.impl.
>     >>>> AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502)
>     >>>> at java.util.concurrent.Executors$RunnableAdapter.
>     >>>> call(Executors.java:511)
>     >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     >>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(
>     >>>> ThreadPoolExecutor.java:1142)
>     >>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>     >>>> ThreadPoolExecutor.java:617)
>     >>>> at java.lang.Thread.run(Thread.java:745)
>     >>>> INFO [o.a.c.f.j.i.AsyncJobMonitor]
>     >>       (Work-Job-Executor-2:ctx-bc104380
>     >>>> job-25/job-27) (logid:aab9c320) Remove job-27 from job
>     >>       monitoring
>     >>>> WARN [o.a.c.alerts] (API-Job-Executor-1:ctx-f787201d job-25
>     >>>> ctx-56356c1a) (logid:aab9c320) alertType:: 8 //
>     >>       dataCenterId:: 1 //
>     >>>> podId:: 1 // clusterId:: null // message:: Failed to deploy
>     >>       Vm with Id: 6,
>     >>>> on Host with Id: null
>     >>>> ERROR [c.c.a.ApiAsyncJobDispatcher]
>     >>       (API-Job-Executor-1:ctx-f787201d
>     >>>> job-25) (logid:aab9c320) Unexpected exception while executing
>     >>>> org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin
>     >>>> com.cloud.utils.exception.CloudRuntimeException: Unable to
>     >>       start a VM due
>     >>>> to insufficient capacity
>     >>>> at com.cloud.vm.VirtualMachineManagerImpl.start(
>     >>>> VirtualMachineManagerImpl.java:623)
>     >>>> at
>     >>       org.apache.cloudstack.engine.cloud.entity.api.
> VMEntityManagerImpl.
>     >>>> deployVirtualMachine(VMEntityManagerImpl.java:242)
>     >>>> at org.apache.cloudstack.engine.cloud.entity.api.
>     >>>>
>     >>       VirtualMachineEntityImpl.deploy(
> VirtualMachineEntityImpl.java:212)
>     >>>> at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(
>     >>>> UserVmManagerImpl.java:4084)
>     >>>> at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(
>     >>>> UserVmManagerImpl.java:3682)
>     >>>> at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(
>     >>>> UserVmManagerImpl.java:3670)
>     >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>     >>       Method)
>     >>>> at sun.reflect.NativeMethodAccessorImpl.invoke(
>     >>>> NativeMethodAccessorImpl.java:62)
>     >>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>     >>>> DelegatingMethodAccessorImpl.java:43)
>     >>>> at java.lang.reflect.Method.invoke(Method.java:498)
>     >>>> at org.springframework.aop.support.AopUtils.
>     >>>> invokeJoinpointUsingReflection(AopUtils.java:333)
>     >>>> at
>     >>       org.springframework.aop.framework.ReflectiveMethodInvocation.
>     >>>> invokeJoinpoint(ReflectiveMethodInvocation.java:190)
>     >>>> at
>     >>       org.springframework.aop.framework.ReflectiveMethodInvocation.
>     >>>> proceed(ReflectiveMethodInvocation.java:157)
>     >>>> at org.apache.cloudstack.network.contrail.management.
>     >>>> EventUtils$EventInterceptor.invoke(EventUtils.java:107)
>     >>>> at
>     >>       org.springframework.aop.framework.ReflectiveMethodInvocation.
>     >>>> proceed(ReflectiveMethodInvocation.java:168)
>     >>>> at com.cloud.event.ActionEventInterceptor.invoke(
>     >>>> ActionEventInterceptor.java:51)
>     >>>> at
>     >>       org.springframework.aop.framework.ReflectiveMethodInvocation.
>     >>>> proceed(ReflectiveMethodInvocation.java:168)
>     >>>> at
>     >>       org.springframework.aop.interceptor.
> ExposeInvocationInterceptor.
>     >>>> invoke(ExposeInvocationInterceptor.java:92)
>     >>>> at
>     >>       org.springframework.aop.framework.ReflectiveMethodInvocation.
>     >>>> proceed(ReflectiveMethodInvocation.java:179)
>     >>>> at org.springframework.aop.framework.JdkDynamicAopProxy.
>     >>>> invoke(JdkDynamicAopProxy.java:213)
>     >>>> at com.sun.proxy.$Proxy186.startVirtualMachine(Unknown
>     >>       Source)
>     >>>> at org.apache.cloudstack.api.command.admin.vm.
>     >>>> DeployVMCmdByAdmin.execute(DeployVMCmdByAdmin.java:50)
>     >>>> at
>     >>       com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150)
>     >>>> at com.cloud.api.ApiAsyncJobDispatcher.runJob(
>     >>>> ApiAsyncJobDispatcher.java:108)
>     >>>> at
>     >>       org.apache.cloudstack.framework.jobs.impl.
> AsyncJobManagerImpl$5.
>     >>>> runInContext(AsyncJobManagerImpl.java:554)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.
> ManagedContextRunnable$1.run(
>     >>>> ManagedContextRunnable.java:49)
>     >>>> at org.apache.cloudstack.managed.context.impl.
>     >>>> DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext.
>     >>>> callWithContext(DefaultManagedContext.java:103)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext.
>     >>>> runWithContext(DefaultManagedContext.java:53)
>     >>>> at
>     >>       org.apache.cloudstack.managed.context.
> ManagedContextRunnable.run(
>     >>>> ManagedContextRunnable.java:46)
>     >>>> at org.apache.cloudstack.framework.jobs.impl.
>     >>>> AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502)
>     >>>> at java.util.concurrent.Executors$RunnableAdapter.
>     >>>> call(Executors.java:511)
>     >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     >>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(
>     >>>> ThreadPoolExecutor.java:1142)
>     >>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>     >>>> ThreadPoolExecutor.java:617)
>     >>>> at java.lang.Thread.run(Thread.java:745)
>     >>>> Caused by:
>     >>       com.cloud.exception.InsufficientServerCapacityException:
>     >>>> Unable to create a deployment for
>     >>       VM[User|i-2-6-VM]Scope=interface
>     >>>> com.cloud.dc.DataCenter; id=1
>     >>>> at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(
>     >>>> VirtualMachineManagerImpl.java:961)
>     >>>> at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(
>     >>>> VirtualMachineManagerImpl.java:4661)
>     >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>     >>       Method)
>     >>>> ... 18 more
>     >>>>
>     >>>> On 3/1/17, 8:52 AM, "Pierre-Luc Dion" <pdion891@apache.org>
>     >>       wrote:
>     >>>>
>     >>>> Do we support centos6 with this 4.10 on jdk8?
>     >>>>
>     >>>> Because I did not had much success to install 4.10 jdk8 on
>     >>       centos6
>     >>>> and it
>     >>>> would make more sense to drop support of centos6 so our
>     >>       packages would
>     >>>> use
>     >>>> centos7 with distro packages for tomcat7 and jdk8. Should
>     >>       also be the
>     >>>> same
>     >>>> with ubuntu 16.04 that use jdk8 and tomcat7 by default ?
>     >>>>
>     >>>>
>     >>>> thanks,
>     >>>>
>     >>>>
>     >>>>> On Wed, Mar 1, 2017 at 9:01 AM, Rene Moser
>     >>       <mail@renemoser.net> wrote:
>     >>>>>
>     >>>>> Hi
>     >>>>>
>     >>>>> While not be directly related to the clodustack java source
>     >>       code, any
>     >>>>> RPM created using the specs from the repo e.g. from
>     >>       packages/centos7
>     >>>>> and proceeding an upgrade, will hit CLOUDSTACK-9765, PR
>     >>>>> https://github.com/apache/cloudstack/pull/1923 fixes the
>     >>       issue.
>     >>>>>
>     >>>>> Regards
>     >>>>> René
>     >>>>>
>     >>>>>
>     >>>>>> On 03/01/2017 02:12 AM, Rajani Karuturi wrote:
>     >>>>>> Hi All,
>     >>>>>>
>     >>>>>> I've created a 4.10.0.0 release, with the following
>     >>       artifacts up
>     >>>> for a
>     >>>>> vote:
>     >>>>>>
>     >>>>>> Git Branch and Commit
>     >>>>>> SH:https://git-wip-us.apache.org/repos/asf?p=cloudstack.
>     >>>>> git;a=shortlog;h=refs/heads/4.10.0.0-RC20170301T0634
>     >>>>>> Commit:7c1d003b5269b375d87f4f6cfff8a144f0608b67
>     >>>>>> <https://git-wip-us.apache.org/repos/asf?p=cloudstack.
>     >>>>> git;a=shortlog;h=refs/heads/4.10.0.0-RC20170301T0634Commit:
>     >>>>> 7c1d003b5269b375d87f4f6cfff8a144f0608b67>
>     >>>>>>
>     >>>>>> Source release (checksums and signatures are available at
>     >>       the same
>     >>>>>>
>     >>       location):https://dist.apache.org/repos/dist/dev/cloudstack/
>     >>>> 4.10.0.0/
>     >>>>>>
>     >>>>>> PGP release keys (signed using
>     >>>>>> CBB44821):https://dist.apache.org/repos/dist/release/
>     >>>> cloudstack/KEYS
>     >>>>>>
>     >>>>>> Vote will be open for 72 hours.
>     >>>>>>
>     >>>>>> For sanity in tallying the vote, can PMC members please be
>     >>       sure to
>     >>>>>> indicate "(binding)" with their vote?
>     >>>>>>
>     >>>>>> [ ] +1 approve
>     >>>>>> [ ] +0 no opinion
>     >>>>>> [ ] -1 disapprove (and reason why)
>     >>>>>>
>     >>>>>>
>     >>>>>>
>     >>>>>> ~Rajani
>     >>>>>> http://cloudplatform.accelerite.com/
>     >>>>>>
>     >>>>>
>     >>>>
>     >>>>
>     >>>>
>     >>
>     >>       rohit.yadav@shapeblue.com
>     >>       www.shapeblue.com<http://www.shapeblue.com
>     >>       ( http://www.shapeblue.com<http://www.shapeblue.com )>
>     >>       53 Chandos Place, Covent Garden, London WC2N 4HSUK
>     >>       @shapeblue
>     >>
>     >>       DISCLAIMER
>     >>       ==========
>     >>       This e-mail may contain privileged and confidential
> information
>     >>       which is the property of Accelerite, a Persistent Systems
>     >>       business. It is intended only for the use of the individual or
>     >>       entity to which it is addressed. If you are not the intended
>     >>       recipient, you are not authorized to read, retain, copy,
> print,
>     >>       distribute or use this message. If you have received this
>     >>       communication in error, please notify the sender and delete
> all
>     >>       copies of this message. Accelerite, a Persistent Systems
> business
>     >>       does not accept any liability for virus infected mails.
>     >>
>     >>       rohit.yadav@shapeblue.com
>     >>       www.shapeblue.com ( http://www.shapeblue.com )
>     >>       53 Chandos Place, Covent Garden, London WC2N 4HSUK
>     >>       @shapeblue
>     >>
>     >>
>     >>
>     >
>     >
>     >
>     >
>     >
>     >
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message