Mailing-List: contact cloudstack-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cloudstack-dev@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of Frank.Zhang@citrix.com
 designates 66.165.176.89 as permitted sender)
From: Frank Zhang <Frank.Zhang@citrix.com>
To: CloudStack DeveloperList <cloudstack-dev@incubator.apache.org>, Nitin
 Mehta <Nitin.Mehta@citrix.com>, Prachi Damle <Prachi.Damle@citrix.com>
CC: Prasanna Santhanam <Prasanna.Santhanam@citrix.com>
Date: Fri, 13 Jul 2012 13:40:24 -0700
Subject: RE: Review Request: CS-15560 : Improve HVM logging of hosts
Thread-Topic: Review Request: CS-15560 : Improve HVM logging of hosts
Thread-Index: Ac1hJXPAlmN3A1XpRi+Gn9a+GbPMCgAD+hJQ
Message-ID: 
 <93099572B72EB341B81A644E134F240B012F6CA38F9F@SJCPMAILBOX01.citrite.net>
References: <20120713065857.729.68744@reviews.apache.org>
 <CC25B601.27D30%chiradeep.vittal@citrix.com>
In-Reply-To: <CC25B601.27D30%chiradeep.vittal@citrix.com>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Taking this chance I have some suggestions to enhance our log strategy.

> 2. The log does not contain any context: if there are a 100 concurrent
> deployment requests active: which one does this apply to? There is the jo=
b-
> id of course, but say the cluster name would be useful for immediate
> grokking 3.
This is what log4j NDC/MDC made for. Cite MDC as an example, you can save
Cluster name, mgmt id, job id or whatever in MDC, then change one line in
Log4j configuration. All log output will be attached with the string you sa=
ved before
just like thread name which is commonly used .
fortunately our thread model is pretty simple,  adding MDC support is not a=
 big work.

>3. It seems to me that just the count of non-HVM hosts is useful: if
> you consider that tens of thousands of vms will be deployed, this same
> verbose log with the exact same info will be printed thousands of times. =
So,
> why not just a counter instead of an expensive list?

I think logging error/warning is better than logging normal stuff verbosely=
. For example,
we log each ping command which leads to a very fat log in condition of even=
 10000 hosts.
And the only thing it tells me is that the periodic ping thread of each Dir=
ectAttachedAgent is alive.

And also we have separate mgmt log in clustered mgmt server environment, it=
 will be better off
setting a log4j socket server to congregate logs to one place. With MDC/NDC=
 support, it's easy to
trace a asycjob from the womb to the tomb in one log.  This way also solves=
 the issue that log exhausts
precious disk of SSVM/CPVM as well.


> -----Original Message-----
> From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com]
> Sent: Friday, July 13, 2012 11:31 AM
> To: CloudStack DeveloperList; Nitin Mehta; Prachi Damle
> Cc: Prasanna Santhanam
> Subject: Re: Review Request: CS-15560 : Improve HVM logging of hosts
>=20
> I think you should re-consider the way this log is handled:
> 1. If the cluster only contains non-HVM hosts, then this is going to be a=
 very
> large log string.
> 2. The log does not contain any context: if there are a 100 concurrent
> deployment requests active: which one does this apply to? There is the jo=
b-
> id of course, but say the cluster name would be useful for immediate
> grokking 3. It seems to me that just the count of non-HVM hosts is useful=
: if
> you consider that tens of thousands of vms will be deployed, this same
> verbose log with the exact same info will be printed thousands of times. =
So,
> why not just a counter instead of an expensive list?
>=20
> On 7/12/12 11:58 PM, "Nitin Mehta" <nitin.mehta@citrix.com> wrote:
>=20
> >
> >-----------------------------------------------------------
> >This is an automatically generated e-mail. To reply, visit:
> >https://reviews.apache.org/r/5927/#review9131
> >-----------------------------------------------------------
> >
> >Ship it!
> >
> >
> >Ship It!
> >
> >- Nitin Mehta
> >
> >
> >On July 13, 2012, 6:46 a.m., Prasanna Santhanam wrote:
> >>
> >> -----------------------------------------------------------
> >> This is an automatically generated e-mail. To reply, visit:
> >> https://reviews.apache.org/r/5927/
> >> -----------------------------------------------------------
> >>
> >> (Updated July 13, 2012, 6:46 a.m.)
> >>
> >>
> >> Review request for cloudstack, Prachi Damle and Nitin Mehta.
> >>
> >>
> >> Description
> >> -------
> >>
> >> CS-15560 : Improve HVM logging of hosts
> >>
> >> When a host is not considered for deployment because it has disabled
> >>HVM, then call that out in the logs for debugging.
> >>
> >>
> >> This addresses bug CS-15560.
> >>
> >>
> >> Diffs
> >> -----
> >>
> >>
> >>server/src/com/cloud/agent/manager/allocator/impl/FirstFitAllocator.ja
> >>va
> >>0f163cc
> >>
> >> Diff: https://reviews.apache.org/r/5927/diff/
> >>
> >>
> >> Testing
> >> -------
> >>
> >> 1) Deployed a Simulator Environment with two Simulator hosts
> >> 2) marked a host as non-HVM and template as HVM enabled
> >>          * update host set capabilities=3DNULL where id=3D1;
> >>          * update vm_template set hvm=3D1 where id=3D10;
> >> 3) restart MS for the VOs to refresh
> >> 4) deploy VM using the template
> >> 5) Verified the non-HVM hosts are called out in the logs
> >>
> >> Logs shown as below:
> >> 2012-07-13 12:07:58,904 DEBUG [allocator.impl.FirstFitAllocator]
> >>(Job-Executor-9:job-30 FirstFitRoutingAllocator) Not considering hosts:
> >>[Host[-1-Routing]]  to deploy template: Tmpl[10-VHD-simulator-Centos
> >>as they are not HVM enabled
> >>
> >>
> >> Thanks,
> >>
> >> Prasanna Santhanam
> >>
> >>
> >