stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nirmal Fernando <nirmal070...@gmail.com>
Subject Re: [Fixed] CEP sends very large values for gradient and second derivative of load average
Date Thu, 06 Nov 2014 07:50:14 GMT
On Thu, Nov 6, 2014 at 8:33 AM, Akila Ravihansa Perera <ravihansa@wso2.com>
wrote:

> Thanks for the explanation, Nirmal. It makes sense now.
>
> Perhaps as an alternative solution, window processor can be set to wait
> until sufficient events have arrived to calculate the gradient?
>

What would be the sufficient value? Theoretically in gradient case it's 2,
in second derivative case it's 3. And we're doing that already.

>
> On Wed, Nov 5, 2014 at 11:48 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com>
> wrote:
>
>> In other words, fix reduces the risks which are coming from bad samples.
>>
>> On Thu, Nov 6, 2014 at 12:08 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com>
>> wrote:
>>
>>>
>>>
>>> On Thu, Nov 6, 2014 at 11:38 AM, Imesh Gunaratne <imesh@apache.org>
>>> wrote:
>>>
>>>> On Thu, Nov 6, 2014 at 9:22 AM, Rajkumar Rajaratnam <rajkumarr@wso2.com
>>>> > wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> No Imesh. There are no difference between *taking the the difference
>>>>> and dividing it by 1000* and *dividing it by 1000 and taking the
>>>>> difference*.
>>>>>
>>>>
>>>> Raj: I agree, in this scenario it makes no difference, however when we
>>>> are applying a fomular the unit conversion should happen prior to applying
>>>> values.
>>>>
>>>
>>> Not exactly Imesh. We can do operations if operands are in the same
>>> units. So we can do subtract operation in milliseconds. And then we can do
>>> division operation to make it in seconds. On the other hand, we can do a
>>> division operation to make each operands to seconds, then since both are
>>> seconds, we can do a subtract operation. So we can apply the formula if
>>> operands are in same unit. I guess this is same for all scenarios.
>>>
>>>
>>>> Still I see a problem with this correction, take the same example:
>>>> Gradient: -999.9999999999998 Last val: 7.000000000000001 First val:
>>>> 12.0 Time Gap: 5 t1: 1415213232152 t2: 1415213232157
>>>>
>>>> *Manual calulation:*
>>>> The value difference: 7 - 12 = -5
>>>> Time difference: 1415213232157 - 1415213232152 = 5 ms => 5/1000 = 0.005
>>>> sec
>>>> *Gradient = - 5/0.005 = -1000*
>>>>
>>>> *According to the current fix:*
>>>> long millisecondsForASecond = 1000;
>>>>         long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1 :
>>>> millisecondsForASecond;
>>>>         double gradient = 0.0;
>>>>         if (tGap > 0) {
>>>>             gradient = ((lastVal - firstVal) * millisecondsForASecond)
>>>> / tGap;
>>>>         }
>>>>
>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7
>>>> Gradient = -5 * 1000 / 1000 = - 5
>>>>
>>>
>>> IMO, this is a sufficient solution.
>>>
>>> Let me explain how I see Nirmal's solution.
>>>
>>> We are taking only first and last values in a window for gradient
>>> calculation. If there are more events in that window, and if that is a good
>>> sample, tGap will be taking some higher values than 1s. This is usual
>>> situation. Nirmal's fix is nothing to do with this scenario.
>>>
>>> If there are few events in that window(say only 2), they can be really
>>> nearer. So there is a possibility that tGap can take a value near to 0. In
>>> that case gradient will be a very big value. Hence, if the tGap is very low
>>> (less than 1 second), then we can assume that the first and last events
>>> actually occurred in 1 second gap. The reason why tGap is less than 0 is,
>>> there are only few events in that window, and that sample is not good
>>> enough. But this a rare situation, it can only happen once in a while.
>>> Nirmal's fix is only dealing with this scenario. Fix imposes a lower limit
>>> in the time interval between two events. This fix is really important to
>>> deal with samples which are not good enough.
>>>
>>> By previous formula, we get -1000. This happened because the sample is
>>> not good enough (may be there are few events). Fix makes it to -5.
>>>
>>> So my understanding is that the solution is good enough for now.
>>>
>>> Thanks.
>>>
>>>>
>>>> Nirmal: Can you please justify this?
>>>>
>>>> Thanks
>>>>
>>>> On Thu, Nov 6, 2014 at 11:31 AM, Nirmal Fernando <
>>>> nirmal070125@gmail.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Thu, Nov 6, 2014 at 6:05 AM, Gayan Gunarathne <gayang@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> AFAIU this is not related with the gradient calculation formula that
>>>>>> we used. But It is related with the how we calculate gradient when the
>>>>>> denominator become a small number.IMO when the divisor (timeInSeconds)
>>>>>> approaches to the small number we do need to normalize the result.
>>>>>>
>>>>>> IMO we do have max and min scale up and down values. So based on that
>>>>>> we can calculate the gradient on that value range.Simply if the denominator
>>>>>> approaches zero result approaches to infinity.
>>>>>>
>>>>>> I think that's what Nirmal doing here by making the lower threshold
>>>>>> as "millisecondsForASecond"
>>>>>>
>>>>>
>>>>> You got it! thanks.
>>>>>
>>>>>>
>>>>>>
>>>>>>         long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1 :
>>>>>> millisecondsForASecond;
>>>>>>         double gradient = 0.0;
>>>>>>         if (tGap > 0) {
>>>>>>             gradient = ((lastVal - firstVal) *
>>>>>> millisecondsForASecond) / tGap;
>>>>>>         }
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Gayan
>>>>>>
>>>>>> On Thu, Nov 6, 2014 at 9:22 AM, Rajkumar Rajaratnam <
>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> On Thu, Nov 6, 2014 at 9:03 AM, Imesh Gunaratne <imesh@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Nirmal,
>>>>>>>>
>>>>>>>> I do not think the milliseconds to seconds convertion is correct
>>>>>>>> here.
>>>>>>>>
>>>>>>>> As I see we are taking the difference of two timestamp values and
>>>>>>>> then dividing it by 1000. The corect way might be to first divide each
>>>>>>>> value by 1000 and then take the difference.
>>>>>>>>
>>>>>>>
>>>>>>> No Imesh. There are no difference between *taking the the
>>>>>>> difference and dividing it by 1000* and *dividing it by 1000 and
>>>>>>> taking the difference*.
>>>>>>>
>>>>>>>
>>>>>>>> On the other hand we might not need to convert these values to
>>>>>>>> seconds since we are taking a time difference and calculating a gradient.
>>>>>>>>
>>>>>>>> I did a quick test with the following sample:
>>>>>>>> Gradient: -999.9999999999998 Last val: 7.000000000000001 First val:
>>>>>>>> 12.0 Time Gap: 5 t1: 1415213232152 t2: 1415213232157
>>>>>>>>
>>>>>>>> *According previous code:*
>>>>>>>> long tGap = t2 - t1;
>>>>>>>>         double gradient = 0.0;
>>>>>>>>         if (tGap > 0) {
>>>>>>>>             gradient = ((lastVal - firstVal) * 1000) / tGap;
>>>>>>>>         }
>>>>>>>>
>>>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7
>>>>>>>> gradient: -1000.0
>>>>>>>>
>>>>>>>> *According to your fix:*
>>>>>>>> long millisecondsForASecond = 1000;
>>>>>>>>         long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1 :
>>>>>>>> millisecondsForASecond;
>>>>>>>>         double gradient = 0.0;
>>>>>>>>         if (tGap > 0) {
>>>>>>>>             gradient = ((lastVal - firstVal) *
>>>>>>>> millisecondsForASecond) / tGap;
>>>>>>>>         }
>>>>>>>>
>>>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7
>>>>>>>> gradient: -5.0
>>>>>>>>
>>>>>>>> *According to an online gradient calculator:*
>>>>>>>> gradient: -1
>>>>>>>>
>>>>>>>> http://www.calculator.net/slope-calculator.html?type=1&x11=1415213232152&y11=12&x12=1415213232157&y12=7&x=27&y=19
>>>>>>>>
>>>>>>>> According to the online gradient calculator (assuming their
>>>>>>>> calculation is correct), the calculation in your fix is not correct. I
>>>>>>>> believe the logic should be simple as follows:
>>>>>>>>
>>>>>>>> long tGap = t2 - t1;
>>>>>>>>         double gradient = 0.0;
>>>>>>>>         if (tGap > 0) {
>>>>>>>>             gradient = ((lastVal - firstVal)) / tGap;
>>>>>>>>         }
>>>>>>>>
>>>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7
>>>>>>>> gradient: -1.0
>>>>>>>>
>>>>>>>
>>>>>>> And AFAIK, standard way to calculate gradient =
>>>>>>> (v1-v2)/timeInSeconds.
>>>>>>>
>>>>>>> Well, we can divide it by what ever time units we wants, but the
>>>>>>> result's unit will be different.
>>>>>>>
>>>>>>> In Nirmal's case, he get will get *stats/seconds*. In your case,
>>>>>>> you will get *stats/milliseconds*
>>>>>>>
>>>>>>> And AS prediction is based on *seconds*, CEP should use *seconds*
>>>>>>> for the calculation.
>>>>>>>
>>>>>>> wdyt?
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Nov 6, 2014 at 1:32 AM, Nirmal Fernando <
>>>>>>>> nirmal070125@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Guys,
>>>>>>>>>
>>>>>>>>> So, I got a chance to reproduce and analyze this issue.
>>>>>>>>>
>>>>>>>>> How we calculate gradient of two events?
>>>>>>>>>
>>>>>>>>> Say the events are; e1(t1,v1) and e2(t2,v2)
>>>>>>>>>
>>>>>>>>> tx - time in *milliseconds* when xth event occurred
>>>>>>>>> vx - value (memory, cpu etc.) that xth event carries
>>>>>>>>>
>>>>>>>>> *time gap = t(2-1) = t2 -t1 milliseconds*
>>>>>>>>> time gap in seconds = t(2-1) = (t2 - t1)/1000 seconds
>>>>>>>>>
>>>>>>>>> Hence,
>>>>>>>>> *Gradient = (v2 - v1)  / t(2-1)  = ( (v2 - v1) * 1000 ) / (t2 -
>>>>>>>>> t1)*
>>>>>>>>>
>>>>>>>>> I've enabled debug logs for CEP extension;
>>>>>>>>>
>>>>>>>>> log4j.logger.org.apache.stratos.cep.extension=DEBUG
>>>>>>>>>
>>>>>>>>> Please find the following 3 logs extracted from the debug logs;
>>>>>>>>>
>>>>>>>>> ===================================================================
>>>>>>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,073] DEBUG
>>>>>>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} -
>>>>>>>>>  Gradient: -0.1996007984031936 Last val: 9.0 First val: 12.0 *Time
>>>>>>>>> Gap: 15030* t1: 1415213202095 t2: 1415213217125 hash: 155426542
>>>>>>>>>
>>>>>>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,073] DEBUG
>>>>>>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} -  Gradient:
>>>>>>>>> -999.9999999999998 Last val: 7.000000000000001 First val: 12.0 *Time
>>>>>>>>> Gap: 5* t1: 1415213232152 t2: 1415213232157 hash: 155426542
>>>>>>>>>
>>>>>>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,074] DEBUG
>>>>>>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} -
>>>>>>>>>  Gradient: -44.34884666437174 Last val: -999.9999999999998 First val:
>>>>>>>>> -0.1996007984031936 *Time Gap: 22544* t1: 1415213209610 t2:
>>>>>>>>> 1415213232154 hash: 155426542
>>>>>>>>> ===================================================================
>>>>>>>>>
>>>>>>>>> So, as you can see the reason behind a large value is when the
>>>>>>>>> time gap between two subjected events is less than 1 second. This could
>>>>>>>>> happen since events are coming from different asynchronous agents and also
>>>>>>>>> when there are less number of events.
>>>>>>>>>
>>>>>>>>> So, the fix I propose is a very simple one and it will not
>>>>>>>>> compromise anything AFAIS.
>>>>>>>>>
>>>>>>>>> Fix is to calculate time gap as follows;
>>>>>>>>>                                                         *____ t2
>>>>>>>>> - t1*
>>>>>>>>>                                                         *| yes?*
>>>>>>>>> *time gap = t(2-1) = t2 -t1 > 1000 ---*
>>>>>>>>>                                                         *|____
>>>>>>>>> 1000*
>>>>>>>>>
>>>>>>>>> I have tested this and works fine.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------- Forwarded message ----------
>>>>>>>>> From: Manula Chathurika Thantriwatte <manulac@wso2.com>
>>>>>>>>> Date: Tue, Oct 28, 2014 at 5:16 AM
>>>>>>>>> Subject: Re: CEP sends very large values for gradient and second
>>>>>>>>> derivative of load average
>>>>>>>>> To: "dev@stratos.apache.org" <dev@stratos.apache.org>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hi Raj,
>>>>>>>>>
>>>>>>>>> Yes it has. But it's very easy to find the values from the agent
>>>>>>>>> with the time stamp. Then we can search that time stamp in the CEP trace
>>>>>>>>> logs and compare the summarized values.
>>>>>>>>>
>>>>>>>>> Thanks !
>>>>>>>>>
>>>>>>>>> On Mon, Oct 27, 2014 at 9:23 PM, Rajkumar Rajaratnam <
>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Oct 27, 2014 at 7:56 PM, Manula Chathurika Thantriwatte <
>>>>>>>>>> manulac@wso2.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Raj,
>>>>>>>>>>>
>>>>>>>>>>> Shall we enable the cartridge agent debug logs. Then we can
>>>>>>>>>>> compare both CEP and agent logs with the time stamp. Then we can narrow
>>>>>>>>>>> down where the actual problem is.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Manula, cep-trace log have the values sent by cartridge agent
>>>>>>>>>> also right?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks !
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Oct 27, 2014 at 5:06 PM, Rajkumar Rajaratnam <
>>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Oct 27, 2014 at 12:53 PM, Manula Chathurika
>>>>>>>>>>>> Thantriwatte <manulac@wso2.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Raj,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Is this happen frequently or intermittently ? Once I go though
>>>>>>>>>>>>> the logs I'm unable to find the high input values for load average. But
>>>>>>>>>>>>> output have high values.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 22:10:38,698 [-] [Siddhi-Scheduler-pool-10-thread-1]  INFO
>>>>>>>>>>>>> EVENT_TRACE_LOGGER TenantId=-1234 : Output Event Adaptor :
>>>>>>>>>>>>> JMSOutputAdaptor, sent
>>>>>>>>>>>>>
>>>>>>>>>>>>> {"org.apache.stratos.messaging.event.health.stat.SecondDerivativeOfLoadAverageEvent":{"message":{"clusterId":"php.php.domain","networkPartitionId":"","value":"2000000.0"}}}
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> This is happening intermittently. Experienced now too. This is
>>>>>>>>>>>> a serious issue. We need to somehow sort this out.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also networkPartitionId is empty. In the current execution
>>>>>>>>>>>>> plans it took networkPartitionId for the calculation. IMO we need to update
>>>>>>>>>>>>> the CEP execution plans for 4.1.0.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:49 AM, Manula Chathurika
>>>>>>>>>>>>> Thantriwatte <manulac@wso2.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks Raj. I'll go through them.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:33 AM, Rajkumar Rajaratnam <
>>>>>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Manula,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You can see those values in the same log I attached in the
>>>>>>>>>>>>>>> previous mail.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:29 AM, Rajkumar Rajaratnam <
>>>>>>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Chamila,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I couldn't find any large values sent by agent. There are
>>>>>>>>>>>>>>>> traces for CEP sending large values. Find the cep trace log here [1]
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1.
>>>>>>>>>>>>>>>> https://drive.google.com/file/d/0B1haIleqJMHIS3FFVFpfa0JlbVE/view?usp=sharing
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:23 AM, Manula Chathurika
>>>>>>>>>>>>>>>> Thantriwatte <manulac@wso2.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Shall we enable the cartridge agent debug logs and see
>>>>>>>>>>>>>>>>> what are the load average values sent from the agent. If those values are
>>>>>>>>>>>>>>>>> normal then we can narrow down to the CEP.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Sat, Oct 25, 2014 at 10:43 PM, Chamila De Alwis <
>>>>>>>>>>>>>>>>> chamilad@wso2.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Raj,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Can you inspect the CEP trace logs for the same time
>>>>>>>>>>>>>>>>>> period? It will have the events published from the agent.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>> Chamila de Alwis
>>>>>>>>>>>>>>>>>> Software Engineer | WSO2 | +94772207163
>>>>>>>>>>>>>>>>>> Blog: code.chamiladealwis.com
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Sat, Oct 25, 2014 at 10:39 PM, Rajkumar Rajaratnam <
>>>>>>>>>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I am doing a test round with M3 with python agent. And
>>>>>>>>>>>>>>>>>>> faced this very strange scenario, $Subject.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,693] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,693] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] -2000.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 1.9090909
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.718182
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Second
>>>>>>>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster] php.php.domain [value]
>>>>>>>>>>>>>>>>>>> 200000.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Second
>>>>>>>>>>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Second
>>>>>>>>>>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 3.3636363
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.11670045
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.7
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Second
>>>>>>>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.7
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 5.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 2000.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.7
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,699] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Second
>>>>>>>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,699] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Second
>>>>>>>>>>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain [value] 2000000.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 13500.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 50.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.738462
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 2.3333333
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 31.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 39.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 39.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,705] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,706] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] -11000.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,706] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,708] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,708] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,709] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.8
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,709] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,811] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>>>>>>>>>>>>>>>>>>>  PodActivationWatcher running : Running
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,942] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>>>>>>>>>>>>>>>>>>>  PodActivationWatcher running : Running
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,945] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>>>>>>>>>>>>>>>>>>>  PodActivationWatcher running : Running
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,712] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} -
>>>>>>>>>>>>>>>>>>>  KubernetesServiceClusterMonitor is running..
>>>>>>>>>>>>>>>>>>> KubernetesServiceClusterMonitor [ kubernetesHostClusterId=KubGrp1,
>>>>>>>>>>>>>>>>>>> clusterId=php.php.domain, serviceId=php]
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,713] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} -
>>>>>>>>>>>>>>>>>>>  Running min check for [kub-cluster] : KubGrp1 [cluster] : php.php.domain
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,713]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  Running minimum rule:
>>>>>>>>>>>>>>>>>>> [kub-cluster] KubGrp1 [cluster] php.php.domain
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,714]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [min-check]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] nonTerminated : 3
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,714]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [min-check]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] minReplicas : 3
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  Running obsolete containers
>>>>>>>>>>>>>>>>>>> rule [kub-cluster] : KubGrp1 [cluster] : php.php.domain
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [obsolete-check] [cluster]
>>>>>>>>>>>>>>>>>>> : php.php.domain [Replicas] obsoleteReplicas : 0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.AutoscalerRuleEvaluator} -  Minimum
>>>>>>>>>>>>>>>>>>> check executed for :
>>>>>>>>>>>>>>>>>>> org.apache.stratos.autoscaler.KubernetesClusterContext@21d906d0
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} -
>>>>>>>>>>>>>>>>>>>  flag of rifReset : true flag of memoryConsumptionReset : true flag of
>>>>>>>>>>>>>>>>>>> loadAverageReset : true
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} -
>>>>>>>>>>>>>>>>>>>  Running scale check for [kub-cluster] : KubGrp1 [cluster] : php.php.domain
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  Running scaling rule
>>>>>>>>>>>>>>>>>>> [kub-cluster] : KubGrp1 [cluster] : php.php.domain
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Predicting the
>>>>>>>>>>>>>>>>>>> value, [average]: 0.0 , [gradient]: 0.0 , [second derivative]: 0.0 , [time
>>>>>>>>>>>>>>>>>>> intervals]: 1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Predicting the
>>>>>>>>>>>>>>>>>>> value, [average]: 28.8 , [gradient]: 0.0 , [second derivative]: 0.0 , [time
>>>>>>>>>>>>>>>>>>> intervals]: 1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Predicting the
>>>>>>>>>>>>>>>>>>> value, [average]: 39.0 , [gradient]: -11000.0 , [second derivative]:
>>>>>>>>>>>>>>>>>>> 2000000.0 , [time intervals]: 1
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] minReplicas : 3
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] maxReplicas : 10
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] nonTerminated : 3
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] activeReplicas : 3
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain [RequestInFlight] predicted value : 0.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain [RequestInFlight] upper limit : 80.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain [RequestInFlight] lower limit : 5.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain [MemoryConsumption] predicted value : 28.799999237060547
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain [MemoryConsumption] upper limit : 80.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain [MemoryConsumption] lower limit : 15.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain [LoadAverage] predicted value : 989039.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain [LoadAverage] upper limit : 180.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain [LoadAverage] lower limit : 20.0
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain scale-up action : true
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain scale-down action : false
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain [LoadAverage] predicted replicas : 16484
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain predicted replicas > max replicas :
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling] Decided to
>>>>>>>>>>>>>>>>>>> scale-up : [cluster] : php.php.domain
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling-up]  [cluster] :
>>>>>>>>>>>>>>>>>>> php.php.domain valid number of replicas to expand : 10
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720]  INFO
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient}
>>>>>>>>>>>>>>>>>>> -  Updating kubernetes replication controller via cloud controller:
>>>>>>>>>>>>>>>>>>> [cluster] php.php.domain [replicas] 10
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,734] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} -
>>>>>>>>>>>>>>>>>>>  CloudControllerServiceImpl:updateContainers for cluster : php.php.domain
>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,758] DEBUG
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} -
>>>>>>>>>>>>>>>>>>>  Cloud Controller is delegating request to update a replication controller
>>>>>>>>>>>>>>>>>>> php.php.domain to Kubernetes layer.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I couldn't investigate the agent side, since containers
>>>>>>>>>>>>>>>>>>> are destroyed.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Any idea?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> email : manulac@wso2.com / manula@apache.org
>>>>>>>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> email : manulac@wso2.com / manula@apache.org
>>>>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>>>
>>>>>>>>>>>>> email : manulac@wso2.com / manula@apache.org
>>>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Regards,
>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>> Software Engineer
>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>
>>>>>>>>>>> email : manulac@wso2.com / manula@apache.org
>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Regards,
>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>> Software Engineer
>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>
>>>>>>>>> email : manulac@wso2.com / manula@apache.org
>>>>>>>>> phone : +94 772492511
>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Best Regards,
>>>>>>>>> Nirmal
>>>>>>>>>
>>>>>>>>> Nirmal Fernando.
>>>>>>>>> PPMC Member & Committer of Apache Stratos,
>>>>>>>>> Senior Software Engineer, WSO2 Inc.
>>>>>>>>>
>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Imesh Gunaratne
>>>>>>>>
>>>>>>>> Technical Lead, WSO2
>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Raj
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Gayan Gunarathne
>>>>>> Technical Lead
>>>>>> WSO2 Inc. (http://wso2.com)
>>>>>> email  : gayang@wso2.com  | mobile : +94 766819985
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>> Nirmal
>>>>>
>>>>> Nirmal Fernando.
>>>>> PPMC Member & Committer of Apache Stratos,
>>>>> Senior Software Engineer, WSO2 Inc.
>>>>>
>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> Raj
>>>
>>
>>
>>
>> --
>> Raj
>>
>
>
>
> --
> Akila Ravihansa Perera
> Software Engineer, WSO2
>
> Blog: http://ravihansa3000.blogspot.com
>



-- 
Best Regards,
Nirmal

Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.

Blog: http://nirmalfdo.blogspot.com/

Mime
View raw message