cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rafael Weingartner <rafaelweingart...@gmail.com>
Subject Re: cloudstack presenting wrong used memory values
Date Fri, 31 Jan 2014 15:53:10 GMT
Well guys, yesterday I took a look at the CS source code and how it was
getting those values. I think that I cracked it.

It seems that there is a whole misunderstanding about the used memory value
and It would be nice an update on the Cloudstack API documentation.

Let's start:

Here is a link for the API doc:
http://cloudstack.apache.org/docs/api/apidocs-4.2/root_admin/listHosts.html

"listHosts" is the method which returns those values about the host (mem,
cpu and etc), the confusion is there.
One looking at the list of values that it can return would find this:

 *memoryallocated*

the amount of the host's memory currently allocated

*memorytotal*

the memory total of the host

*memoryused*

the amount of the host's memory currently used


The first time I looked at it I thought; well, nice I can have the values
about the amount of the total memory available to allocate VMs, the amount
allocated and the amount used by those VMs.However, it turns out that it is
a little more complicated than that.

First of all, the memoryTotal, is the total amount of memory available to
allocate directly to VMs, it is neither the total amount of memory on DOM0,
nor the total amount of memory of the host.

I will give an example; I am use Xen hypervisor and XCP. Well, let's
suppose I have a host with 8GB of RAM, if I let the auto ballooning driver
on it would look like the DOM0 would have available 8GB of RAM (at first
sight), and one could use the "free" command to find out the amount of
memory of that host. However, that is not the proper way to check the total
amount of RAM to allocate to VMs, and each time you instantiate a new VM
that value would decrease and would not appear on used memory. BTW, I do
not use the auto ballooning driver, so, if I use the free command on DOM0
it would present a much smaller amount o RAM.
The total amount of memory available to allocate VMs is calculated like
this, "total host's RAM - DOM0 RAM".  Cloudstack does that math perfectly;
my host that has 8GB of RAM on CS it says that it has 7.5GB, since I set
static values of RAM and VCPUs to my DOM0 (512MB of RAM and 2 VCPUs which
are more than enough to that DOM0). To find out those values, you could you
"xentop", and check where your hosts memory and CPUs are being used.

Second, the memoryAllocated value is the amount of memory allocated to VMs
and CS does that properly. No explanations needed, the value could be
calculated using "xentop" or "xe host-compute-free-memory" and then
decrease that value from the value of total memory, anyways it would give
the exactly amount of memory with no problems.


Third, the memoryused value, here is the misunderstanding. This value
represents the memory used by the DOM0 (the amount that is allocated to it)
and not the memory used by deployed VMs on that Host. However, the way CS
API doc is written can make one think that those values are all about
deployed VMs on hosts, when one of them is not. Thus, there is also another
problem,  there are no info about the metric used to represent such values,
 memoryTotal and memoryAllocated are given in bytes while memoryused is
given in kilobytes and that is why the CS UI is presenting the those pretty
weird values for used memory on infrastructure > Hosts view.


At the end I do believe that the documentation needs update in order to
state clearly what those values means and in what metric they are given.
Thus, the CS UI needs a bug fix since the used memory value is being
converted in a wrong way.

BTW, I also noticed that it those values are taking a while to be updated,
what is the time interval between updates of hosts statistics? I saw a
configuration parameter called "host.stats.interval" that is set to 1(6000
milliseconds) minute. However, on table "op_host_capacity" and it has been
two days that those values are not being updated. Could be a
misconfiguration?A problem?


On Thu, Jan 30, 2014 at 10:57 AM, Rafael Weingartner <
rafaelweingartner@gmail.com> wrote:

> I am using xen-hypervisor 4.1 and XCP 1.6 if I am not mistaken.
>
> I have also never noticed this used memory values until we deployed Zenoss
> to monitor the cloud. I just noticed that because the Zenoss was generating
> a graph that presented whole cloud memory usage and it was a little weird.
> With high allocated values, but pretty low used values.
>
> As in your case, it seems pretty weird to me that we have this huge amount
> of memory allocated (26GB), but just 28MB are being used. It almost does
> not make sense. I also should point out that you used the command "free
> -g", it shows the memory of the DOM0, in your case it just presents the
> whole amount of memory of your hosts because you are using the auto
> ballooning option. But in my case, we disabled this, and set static amount
> of memory and cpus for the DOM0.
>
> Does anyone know from where the CS is getting those used memory values?
>
>
> On Thu, Jan 30, 2014 at 5:48 AM, Marty Sweet <msweet.dev@gmail.com> wrote:
>
>> I should also point out that this is on CS 4.2.0.
>>
>> This issue may have been fixed since then.
>>
>> Marty
>>
>>
>> On Thu, Jan 30, 2014 at 7:41 AM, Marty Sweet <msweet.dev@gmail.com>
>> wrote:
>>
>> > What hypervisor are you using?
>> >
>> > My CS System Capacity on the dashboard shows *Memory 157.05 GB / 1.37
>> TB *(which
>> > works + overprovisioning).
>> >
>> > I think this issue is slightly different to CLOUDSTACK-3048 as it should
>> > not effect VM allocation.
>> > I have never noticed about the memory used section:
>> >
>> >   Memory Allocated 68.50 GB  Memory Used 70.65 MB
>> > $ free -g
>> >              total       used       free     shared    buffers
>> cached
>> > Mem:           251        250          1          0          0
>>  179
>> > -/+ buffers/cache:         70        181
>> > Swap:            5          2          2
>> >
>> >
>> >
>> >  Memory Allocated 22.05 GB  Memory Used 24.06 MB
>> > $ free -g
>> >              total       used       free     shared    buffers
>> cached
>> > Mem:           251         27        223          0          0
>>  3
>> > -/+ buffers/cache:         24        227
>> > Swap:            2          0          2
>> >
>> >
>> >  Memory Allocated 39.53 GB  Memory Used 41.51 MB
>> > $ free -g
>> >              total       used       free     shared    buffers
>> cached
>> > Mem:           251         42        209          0          0
>>  0
>> > -/+ buffers/cache:         41        210
>> > Swap:            5          0          5
>> >
>> >
>> >  Memory Allocated 26.97 GB  Memory Used 28.46 MB
>> > $ free -g
>> >              total       used       free     shared    buffers
>> cached
>> > Mem:           141         29        111          0          0
>>  1
>> > -/+ buffers/cache:         28        113
>> > Swap:            2          0          2
>> >
>> >
>> > It appears that the units used for 'Memory used' are simply a few
>> factors
>> > of 10 out.
>> >
>> > Can you have a look to see if there is a bug specifically for this? If
>> not
>> > feel free to create one using the information I have provided above.
>> >
>> > Thanks,
>> > Marty
>> >
>> >
>> >
>> > On Wed, Jan 29, 2014 at 6:31 PM, Rafael Weingartner <
>> > rafaelweingartner@gmail.com> wrote:
>> >
>> >> Doesn't anyone think that those memory values presented on
>> >> infrastructure>hosts are a little odd?
>> >>
>> >>
>> >> On Tue, Jan 28, 2014 at 2:54 PM, Rafael Weingartner <
>> >> rafaelweingartner@gmail.com> wrote:
>> >>
>> >> > Hi all,
>> >> >
>> >> > I am mailing this to the dev and users list, since I am not sure
>> which
>> >> one
>> >> > would be the best to make such question.
>> >> >
>> >> >
>> >> > So, I deployed Zenoss to monitor my Cloud, and I noticed something
>> weird
>> >> > on a graph that it was generating about the Cloud's memory. The graph
>> >> was
>> >> > showing total memory as 26GB, allocated as 16GB and used as 19MB.
>> >> >
>> >> > It was not making sense, so I was looking around and I noticed that
>> this
>> >> > 19MB is the value that the Cloudstack presents as the memory used on
>> >> > infrastructure>Hosts.
>> >> >
>> >> >
>> >> > I think that this problem is related to this ticket opened last year.
>> >> >
>> >> > https://issues.apache.org/jira/browse/CLOUDSTACK-3048
>> >> > Sadly, it seems that no one has taken a look at it.
>> >> >
>> >> >
>> >> > Does anyone know anything about this problem/bug?
>> >> >
>> >> >
>> >> > --
>> >> > Rafael Weingärtner
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Rafael Weingärtner
>> >>
>> >
>> >
>> >
>>
>
>
>
> --
> Rafael Weingärtner
>



-- 
Rafael Weingärtner

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message