cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Somesh Naidu <Somesh.Na...@citrix.com>
Subject RE: Dynamic Scalable Template issue
Date Tue, 15 Sep 2015 22:38:20 GMT
I believe XS does not allocate more than the dynamic-max to the VM. That task manger, perfmon,
etc. would report incorrect memory usage.

One way to find out how much memory is available to the Windows guest is running the following
in a powershell (with admin privilege):
$a = gwmi -n root\wmi -cl CitrixXenStoreSession
$a[0].getvalue("memory/target").value

The value returned would represent “real” amount of memory (in KB) available to Windows
guest.

Regards,
Somesh

From: Todd Pigram [mailto:todd@toddpigram.com]
Sent: Tuesday, September 15, 2015 4:26 PM
To: Somesh Naidu
Cc: CloudStack Users
Subject: Re: Dynamic Scalable Template issue

I understand this but in the case of MS SQL it will use all the memory that is available to
Windows. As in my case a DDC with SQL was using 31.5gb of ram while looking in perfmon. But
the system offering was limited to 8GB.

Basically Dynamic Scales works but is bad for SQL servers. I just turned it off.

On Tuesday, September 15, 2015, Somesh Naidu <Somesh.Naidu@citrix.com<mailto:Somesh.Naidu@citrix.com>>
wrote:
Well, that defect is considered a doc defect from CCP/CS perspective so there are no code
changes involved. After listing the limitation in the CCP 4.5 release notes the defect is
marked as fixed.

The behavior is controlled/affected by how XS DMC works and how Windows guests view available
memory. Windows guests always see static-max whereas the usable memory (allowed by Xen) for
the guest can never exceed dynamic-max. In case of dynamic scaling, CCP/CS sets static-max
higher (4 times dynamic-min) than dynamic-max. The memory represented by static-max – dynamic-max
is occupied by XS balloon driver (contained in the XS tools) which inflates/deflates based
on dynamic memory scaling requests.

Essentially, what Windows guests reports as available memory is incorrect. The functionality
itself isn’t affected, that is, the actual memory available to the Windows guest is correct.

Somesh
CloudPlatform Escalations
Citrix Systems, Inc.

From: Todd Pigram [mailto:todd@toddpigram.com<javascript:_e(%7B%7D,'cvml','todd@toddpigram.com');>]
Sent: Tuesday, September 15, 2015 11:33 AM
To: Somesh Naidu
Cc: CloudStack Users
Subject: Re: Dynamic Scalable Template issue

Is this fixed in 4.5.1 as the issue is not listed in the release notes under CS-27425/CS-21217?

As a FYI, on the one VM the new tools worked but not on any others. I had to set ‘enable.dynamic.scale.vm’
to false, restart management service. Then I have to shutdown the instance. then start, to
get the right memory. A reboot doesn’t fix it.



Todd Pigram
http://about.me/ToddPigram
www.linkedin.com/in/toddpigram/<http://www.linkedin.com/in/toddpigram/>
@pigram86 on twitter
https://plus.google.com/+ToddPigram86
Mobile - 216-224-5769

PGP Public Key<http://pgp.mit.edu/pks/lookup?op=get&search=0x96B7B0F0C55933BB>

On Sep 15, 2015, at 11:11 AM, Somesh Naidu <Somesh.Naidu@citrix.com<javascript:_e(%7B%7D,'cvml','Somesh.Naidu@citrix.com');>>
wrote:

A similar issue is listed in CCP 4.5 release notes – look for “CS-27425/CS-21217”.

Somesh
CloudPlatform Escalations
Citrix Systems, Inc.

From: Todd Pigram [mailto:todd@toddpigram.com<javascript:_e(%7B%7D,'cvml','todd@toddpigram.com');>]
Sent: Wednesday, September 09, 2015 11:22 AM
To: CloudStack Users
Subject: Re: Dynamic Scalable Template issue

Well its seems to be a XS Tools issue. I am waiting to here back from support on the ‘Official’
word, but with the new tools installed and the Dynamic Scalable option set, I am currently
running like I was before the upgrade to CCP 4.3.0.2.

At this point it is closed.

Thanks for the help.

Todd Pigram
http://about.me/ToddPigram
www.linkedin.com/in/toddpigram/<http://www.linkedin.com/in/toddpigram/<http://www.linkedin.com/in/toddpigram/%3chttp:/www.linkedin.com/in/toddpigram/>>
@pigram86 on twitter
https://plus.google.com/+ToddPigram86
Mobile - 216-224-5769

PGP Public Key<http://pgp.mit.edu/pks/lookup?op=get&search=0x96B7B0F0C55933BB>

On Sep 9, 2015, at 10:53 AM, Vadim Kimlaychuk <vadim@kickcloud.net<mailto:vadim@kickcloud.net<javascript:_e(%7B%7D,'cvml','vadim@kickcloud.net%5Cx3cmailto:vadim@kickcloud.net');>>>
wrote:

Todd,

      Is VM guest shows the correct amount of RAM? Is this issue resolved?

Vadim.

On 2015-09-09 15:34, Todd Pigram wrote:


Vadium,
Yes to both with the new tools from XS62ESP1028. Both Windows and CentOS have 'Dynamic Scalable'
Option selected and in XenCenter they show the correct RAM.
Todd Pigram
http://about.me/ToddPigram [1]
www.linkedin.com/in/toddpigram/<http://www.linkedin.com/in/toddpigram/<http://www.linkedin.com/in/toddpigram/%3chttp:/www.linkedin.com/in/toddpigram/>>
[2] @pigram86 on twitter
https://plus.google.com/+ToddPigram86 [3] Mobile - 216-224-5769
PGP Public Key [4]
On Sep 9, 2015, at 2:49 AM, Vadim Kimlaychuk <vadim@kickcloud.net<mailto:vadim@kickcloud.net<javascript:_e(%7B%7D,'cvml','vadim@kickcloud.net%5Cx3cmailto:vadim@kickcloud.net');>>>
wrote:
Todd,
Have you tried to do the following manual tests on the cluster where you have problem:
1. Dynamically scalable with CentOS ?
2. Dynamically scalable with Windows ?
What do they show as available RAM?
Regards,
On 2015-09-08 19:44, Todd Pigram wrote:
Vadium,
After installing XS62ESP1028 via CLI (no reboot on hosts yet) and building an Centos65 instance
w/o 'Dynamic Scalable' option checked, it showed right in XenCenter. I installed the new tools
from (XS62ESP1028) and still good. I created a template from this instance. Deployed said
template, and the memory is still good in XenCenter.
I tested on a Windows VM and it is the same with the new tools, however, as this particular
tenant bypasses the virtual router, I have to reboot twice and reset networking as the new
XenTools reset the networking stack.
Todd Pigram
http://about.me/ToddPigram [1] [1]
www.linkedin.com/in/toddpigram/<http://www.linkedin.com/in/toddpigram/<http://www.linkedin.com/in/toddpigram/%3chttp:/www.linkedin.com/in/toddpigram/>>
[2] [2] @pigram86 on twitter
https://plus.google.com/+ToddPigram86 [3] [3] Mobile - 216-224-5769
PGP Public Key [4]
On Sep 6, 2015, at 4:57 AM, Vadim Kimlaychuk <vadim@kickcloud.net<mailto:vadim@kickcloud.net<javascript:_e(%7B%7D,'cvml','vadim@kickcloud.net%5Cx3cmailto:vadim@kickcloud.net');>>>
wrote:
Todd,
Can you try Linux template with same dynamic scale option on the same pool? I wonder if there
is a problem with Windows or any guest OS.
Regards,
Vadim.
On 2015-09-05 21:04, Todd Pigram wrote:
Vadium
That makes sense. I will see if I can replicate the issue in a lab. But given the holiday
weekend, might not be until next week
On Saturday, September 5, 2015, Vadim Kimlaychuk <vadim@kickcloud.net<mailto:vadim@kickcloud.net<javascript:_e(%7B%7D,'cvml','vadim@kickcloud.net%5Cx3cmailto:vadim@kickcloud.net');>>>
wrote:
Todd,
I have seeing similar problem with Xen 4.1 (not XenServer). Linux guests were able to see
(and use) entire host resources on any guest VM. That was a bug of configuration. If you think
about what could be different after CS update - it could be VM registration procedure. Still
guest VM should not be able to see static max. Your XenCenter shows that effective dynamic
VM memory size is 8Gb while maximium is 32Gb. So CS configured VM guest correctly. This is
problem of hypervisor <-> guest VM communitcation. That is why I asked you to try to
register VM manually. I believe you will have the same result. Than means your server pool
of XS62ESP1027 is broken. 3 other pools are not. I see no reason to update to 4.5.1, because
I think this is not the problem of CS, but particularly this XenServer pool + this type of
Windows guest (if other templates with dynamic offer are good).
Vadim.
On 2015-09-05 14:45, Todd Pigram wrote:
Vadim
I have 3 other pools (1 XS6.2sp1 and 2x xs65sp1) I have no issue with these.
Based on the Design doc, what i was experiencing is by design. Ok I will turn it off.
But now my question is, why on 4.3 I didn't have this issue but after installing 4.3.0.2 it
changed.
Was 4.3 broken or is 4.3.0.2? Will upgrading to CCP 4.5.1 will be better?
If this is truely by design, I will not be able to use dynamic scalable for my windows instances
On Saturday, September 5, 2015, Vadim Kimlaychuk <vadim@kickcloud.net<mailto:vadim@kickcloud.net<javascript:_e(%7B%7D,'cvml','vadim@kickcloud.net%5Cx3cmailto:vadim@kickcloud.net');>>>
wrote:
Todd,
You may try to create VM at XenServer without CloudStack just using XE tool (or XenCenter).
If your manually created VM with static max <> dynamic max will be OK, then there is
a problem with Cloudstack.
Vadim.
On 2015-09-04 21:51, Todd Pigram wrote:
Latest as of XS62ESP1027. I know XS62ESP1028 comes with new XenTools.
On Friday, September 4, 2015, Vadim Kimlaychuk <vadim@kickcloud.net<mailto:vadim@kickcloud.net<javascript:_e(%7B%7D,'cvml','vadim@kickcloud.net%5Cx3cmailto:vadim@kickcloud.net');>>>
wrote:
I am afraid this issue has nothing to do with Cloudstack. If VM "sees"
maximum memory size available instead of effective -- this is problem of
hypervisor and/or guest utilities. Do you have guest utilities that come
with XenServer version or older in your template?
Vadim.
On 2015-09-04 19:34, Todd Pigram wrote:
Ok. After rebooting the instance still no change after disabling globally. I modified all
my templates to remove 'Dynamic Scalable', then I have to
shutdown every instance and unselect 'Dynamic Scalable', then restart for
it to be correct.
This may need to be looked at closer for a Windows/XenDesktop(XenApp)
environment as MS SQL will consume all the RAM windows sees.
My other question is why CCP 4.3 (which is certified for Citrix Ready for
IaaS for XD) didn't have this issue until I upgraded to 4.3.0.2 in July.
This feature was released in 4.2.....
Thanks for all the help.
Todd Pigram
http://about.me/ToddPigram [1] [1] [1] [1] [1] [1 [1]] <http://about.me/ToddPigram [1]
[1] [1] [1 [1]] [5] [5]<http://about.me/ToddPigram%20%5b1%5d%20%5b1%5d%20%5b1%5d%20%5b1%20%5b1%5d%5d%20%5b5%5d%20%5b5%5d<http://about.me/ToddPigram%20%5b1%5d%20%5b1%5d%20%5b1%5d%20%5b1%20%5b1%5d%5d%20%5b5%5d%20%5b5%5d%3chttp:/about.me/ToddPigram%20%5b1%5d%20%5b1%5d%20%5b1%5d%20%5b1%20%5b1%5d%5d%20%5b5%5d%20%5b5%5d>>>
www.linkedin.com/in/toddpigram/<http://www.linkedin.com/in/toddpigram/<http://www.linkedin.com/in/toddpigram/%3chttp:/www.linkedin.com/in/toddpigram/>>
[2] [2] [2] [2] [2] [2 [2]] <
http://www.linkedin.com/in/toddpigram/ [2] [2] [2] [2] [2] [2 [2]]>
@pigram86 on twitter
https://plus.google.com/+ToddPigram86 [3] [3] [3] [3] [3] [3 [3]] <
https://plus.google.com/+ToddPigram86 [3] [3] [3] [3] [3] [3 [3]]>
Mobile - 216-224-5769
PGP Public Key <
http://pgp.mit.edu/pks/lookup?op=get&search=0x96B7B0F0C55933BB [4] [4] [4] [4] [4] [4
[5]]> On
Sep 4, 2015, at 12:04 PM, Todd Pigram <todd@toddpigram.com<mailto:todd@toddpigram.com<javascript:_e(%7B%7D,'cvml','todd@toddpigram.com%5Cx3cmailto:todd@toddpigram.com');>>>
wrote:
Thanks.
Just funny that on CCP 4.3 it wasn't an issue and only after upgrading to
4.3.0.2 and a host reboot did it change my instances.
This issue is that the instance below is a SQL server. As Windows see
32GB, it shows the used RAM in windows as 31.6GB. The Server only has 8GB.
That is what really is the issue. Causing slowness for my users.
I have set 'enable.dynamic.scale.vm' to false and restarted
cloudstack-management. I will reboot an instance that is out of prod and
let you know the results.
Todd Pigram
http://about.me/ToddPigram [1] [1] [1] [1] [1] [1 [1]] <http://about.me/ToddPigram [1]
[1] [1] [1 [1]] [5] [5]<http://about.me/ToddPigram%20%5b1%5d%20%5b1%5d%20%5b1%5d%20%5b1%20%5b1%5d%5d%20%5b5%5d%20%5b5%5d<http://about.me/ToddPigram%20%5b1%5d%20%5b1%5d%20%5b1%5d%20%5b1%20%5b1%5d%5d%20%5b5%5d%20%5b5%5d%3chttp:/about.me/ToddPigram%20%5b1%5d%20%5b1%5d%20%5b1%5d%20%5b1%20%5b1%5d%5d%20%5b5%5d%20%5b5%5d>>>
www.linkedin.com/in/toddpigram/<http://www.linkedin.com/in/toddpigram/<http://www.linkedin.com/in/toddpigram/%3chttp:/www.linkedin.com/in/toddpigram/>>
[2] [2] [2] [2] [2] [2 [2]] <
http://www.linkedin.com/in/toddpigram/ [2] [2] [2] [2] [2] [2 [2]]>
@pigram86 on twitter
https://plus.google.com/+ToddPigram86 [3] [3] [3] [3] [3] [3 [3]] <
https://plus.google.com/+ToddPigram86 [3] [3] [3] [3] [3] [3 [3]]>
Mobile - 216-224-5769
PGP Public Key <
http://pgp.mit.edu/pks/lookup?op=get&search=0x96B7B0F0C55933BB [4] [4] [4] [4] [4] [4
[5]]> On
Sep 4, 2015, at 11:41 AM, Vadim Kimlaychuk <vadim@kickcloud.net<mailto:vadim@kickcloud.net<javascript:_e(%7B%7D,'cvml','vadim@kickcloud.net%5Cx3cmailto:vadim@kickcloud.net');>>
<mailto:
vadim@kickcloud.net<mailto:vadim@kickcloud.net<javascript:_e(%7B%7D,'cvml','%5Cx0bvadim@kickcloud.net%5Cx3cmailto:vadim@kickcloud.net');>>>>
wrote:
Hello Todd,
If I understood you correctly, you wonder why you have static max as 4x
of the offering, isn't it? This article should answer your question:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scaling+of+CPU+and+RAM#DynamicscalingofCPUandRAM-Xenserver
[6] [6] [5] [5] [6]
[5] <
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scaling+of+CPU+and+RAM#DynamicscalingofCPUandRAM-Xenserver
[6] [6] [5] [5] [6]
[5]>
Look chapter for XenServer changes. As it is stated there: static max =
f_min (4 * service_offering) / memory_overprovisioning_of_cluster
Regards,
Vadim.
On 2015-09-04 17:41, Todd Pigram wrote:
Just wondering if anyone has seen similar. I already have a case open.
But seeing if the forum has an answer?
Issue, Upgraded to CCP 4.3.0.2 back in July. Afterwards everything was
great. Hosts rebooted two weeks ago. Any instance built with a template
with Dynamic Scalable set to yes, shows in XenCenter as min/max memory as
what system offering is but has a Static Max of 4x the RAM.
Screen shot of instance built from template with Dynamic Scalable = yes




--


Todd Pigram
http://about.me/ToddPigram
www.linkedin.com/in/toddpigram/<http://www.linkedin.com/in/toddpigram/>
@pigram86 on twitter
https://plus.google.com/+ToddPigram86
Mobile - 216-224-5769

PGP Public Key<http://pgp.mit.edu/pks/lookup?op=get&search=0x96B7B0F0C55933BB>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message