stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lakmal Warusawithana <lak...@wso2.com>
Subject Re: [Fixed] CEP sends very large values for gradient and second derivative of load average
Date Tue, 11 Nov 2014 09:36:09 GMT
On Sat, Nov 8, 2014 at 9:42 AM, Lahiru Sandaruwan <lahirus@wso2.com> wrote:

>
>
> On Sat, Nov 8, 2014 at 12:49 AM, Nirmal Fernando <nirmal070125@gmail.com>
> wrote:
>
>> Hi Lahiru,
>>
>> All these are improvements and should have discussed in a new thread IMO.
>> (these are buried now)
>>
>
> +1. I will send a complete proposal to a new thread.
>
>
Do we have new thread?

IMO, we should go for the following fix, otherwise we can't defend on the
logic behind this.

   - CEP should calculate first, second derivatives against individual
   member, not the network cluster.
   - AutoScaler should calculate prediction values for individual member.
   Then AutoScaler should calculate the average of prediction values against
   network partition wise.
   - Based on this, we should do the scale up or down.


@Nirmal, yes this will hit some load specially on CEP, but my POV this is
the correct solution, and we should go for that.


> Thanks.
>
>>
>> What we currently have is linear gradient, which is the simplest form
>> (first step!)
>>
>> On Thu, Nov 6, 2014 at 2:45 PM, Lahiru Sandaruwan <lahirus@wso2.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> Very interesting discussion and thanks Nirmal for initiating this!
>>>
>>> I also did some research on this.
>>> I think best approach to find gradient and second derivative is to
>>> "curve fitting". Refer "Locally weighted linear regression" section at [1].
>>> We should be calculating gradient and second derivative by differentiating
>>> the equation of fitted line.
>>>
>>> We would need a second degree polynomial fitter for this, where we can
>>> use Apache commons math library for this. Refer the sample at [2], we need
>>> to run this with degree 2 instead of 3.
>>>
>>> E.g.
>>> So if get degree 2 polynomial fitter, we will have an equation like
>>> below where value(v) is our statistic value, time(t) is the time of event,
>>> gradient(g), and second derivative(sd).
>>>
>>> Equation we get from received events,
>>> v = a*t*t + b*t + c
>>>
>>> g = 2*a*t
>>>
>>
>> g= 2*a*t + b
>>
>>
>>> sd = 2*a
>>>
>>> We can get *g* by applying the median of the time stamp sample.
>>>
>>
>> If you use median of the timestamp sample, g would grow with time?!
>>
>>
>>>
>>> I have run a sample successfully. I can complete the work on CEP
>>> extension if you guys agree.
>>>
>>> In addition to this, we should also restrict the sample size as Akila
>>> suggested. Ideally we should receive 6 events per minute for memberwise
>>> stats and "6 * no. of instances" for clusterwise stats.
>>>
>>
>> From where did you get '6' ?
>>
>>
>>> We should put a constraint with this much of percentage(say 80%) should
>>> be there to calculate *g *and* sd.*
>>>
>>
>> This much of a percentage from what value?
>>
>>>
>>> Also we should not calculated gradient of stats sent from different
>>> members in clusterwise scenario as Imesh mentioned. That causes this type
>>> of small time gap issues.
>>>
>>> Thanks.
>>>
>>> [1] http://cs229.stanford.edu/notes/cs229-notes1.pdf
>>> [2] http://commons.apache.org/proper/commons-math/userguide/fitting.html
>>>
>>> On Thu, Nov 6, 2014 at 6:41 PM, Imesh Gunaratne <imesh@apache.org>
>>> wrote:
>>>
>>>> Raj: Thanks for the clarification, it was my misunderstanding.
>>>>
>>>> I did further analysis on this issue, seems like the root cause of this
>>>> problem resides somewhere else.
>>>>
>>>> Currently we have set the cartridge agent's statistics publishing
>>>> interval to 15 seconds. If so how do we receive series of events within a
>>>> second? Seems like we are considering statistics events coming from
>>>> different members for calculating the gradient. This might not be correct,
>>>> if we were to draw a curve of a data set and consider the gradient of those
>>>> values, those values may have come from the same source. Otherwise the
>>>> gradient value we are calculating might not be correct.
>>>>
>>>> Thanks
>>>>
>>>> On Thu, Nov 6, 2014 at 2:05 PM, Udara Liyanage <udara@wso2.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Imagine we put the values in a graph (value against time). The
>>>>> gradient is the slope of the line. When we draw the line via two very near
>>>>> dots, we get a high slope/gradient even for a low increase of the values.
>>>>> This happens when we have only events which are occurred in very near time
>>>>> intervals.
>>>>>
>>>>> On the other hands, when events are distributed among long time period
>>>>> (are more counts in sample) we don't get this high gradient since the line
>>>>> goes via higher dots count where we get a reasonable value for gradient.
>>>>>
>>>>> As I understood the scenario, it is reasonable to make time gap equals
>>>>> to millisInSeconds in case time gap is very small.
>>>>>
>>>>> Touched, not typed. Erroneous words are a feature, not a typo.
>>>>> I'd rather keep it simple Akila (this would complex the scenario of
>>>>> second derivative too) :-) Isn't it?
>>>>>
>>>>> On Thu, Nov 6, 2014 at 9:03 AM, Akila Ravihansa Perera <
>>>>> ravihansa@wso2.com> wrote:
>>>>>
>>>>>> Until t2 - t1 > 1
>>>>>>
>>>>>> On Thu, Nov 6, 2014 at 12:50 AM, Nirmal Fernando <
>>>>>> nirmal070125@gmail.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Nov 6, 2014 at 8:33 AM, Akila Ravihansa Perera <
>>>>>>> ravihansa@wso2.com> wrote:
>>>>>>>
>>>>>>>> Thanks for the explanation, Nirmal. It makes sense now.
>>>>>>>>
>>>>>>>> Perhaps as an alternative solution, window processor can be set to
>>>>>>>> wait until sufficient events have arrived to calculate the gradient?
>>>>>>>>
>>>>>>>
>>>>>>> What would be the sufficient value? Theoretically in gradient case
>>>>>>> it's 2, in second derivative case it's 3. And we're doing that already.
>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Nov 5, 2014 at 11:48 PM, Rajkumar Rajaratnam <
>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>
>>>>>>>>> In other words, fix reduces the risks which are coming from bad
>>>>>>>>> samples.
>>>>>>>>>
>>>>>>>>> On Thu, Nov 6, 2014 at 12:08 PM, Rajkumar Rajaratnam <
>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Nov 6, 2014 at 11:38 AM, Imesh Gunaratne <
>>>>>>>>>> imesh@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> On Thu, Nov 6, 2014 at 9:22 AM, Rajkumar Rajaratnam <
>>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> No Imesh. There are no difference between *taking the the
>>>>>>>>>>>> difference and dividing it by 1000* and *dividing it by 1000
>>>>>>>>>>>> and taking the difference*.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Raj: I agree, in this scenario it makes no difference, however
>>>>>>>>>>> when we are applying a fomular the unit conversion should happen prior to
>>>>>>>>>>> applying values.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Not exactly Imesh. We can do operations if operands are in the
>>>>>>>>>> same units. So we can do subtract operation in milliseconds. And then we
>>>>>>>>>> can do division operation to make it in seconds. On the other hand, we can
>>>>>>>>>> do a division operation to make each operands to seconds, then since both
>>>>>>>>>> are seconds, we can do a subtract operation. So we can apply the formula if
>>>>>>>>>> operands are in same unit. I guess this is same for all scenarios.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Still I see a problem with this correction, take the same
>>>>>>>>>>> example:
>>>>>>>>>>> Gradient: -999.9999999999998 Last val: 7.000000000000001 First
>>>>>>>>>>> val: 12.0 Time Gap: 5 t1: 1415213232152 t2: 1415213232157
>>>>>>>>>>>
>>>>>>>>>>> *Manual calulation:*
>>>>>>>>>>> The value difference: 7 - 12 = -5
>>>>>>>>>>> Time difference: 1415213232157 - 1415213232152 = 5 ms => 5/1000
>>>>>>>>>>> = 0.005 sec
>>>>>>>>>>> *Gradient = - 5/0.005 = -1000*
>>>>>>>>>>>
>>>>>>>>>>> *According to the current fix:*
>>>>>>>>>>> long millisecondsForASecond = 1000;
>>>>>>>>>>>         long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1 :
>>>>>>>>>>> millisecondsForASecond;
>>>>>>>>>>>         double gradient = 0.0;
>>>>>>>>>>>         if (tGap > 0) {
>>>>>>>>>>>             gradient = ((lastVal - firstVal) *
>>>>>>>>>>> millisecondsForASecond) / tGap;
>>>>>>>>>>>         }
>>>>>>>>>>>
>>>>>>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7
>>>>>>>>>>> Gradient = -5 * 1000 / 1000 = - 5
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> IMO, this is a sufficient solution.
>>>>>>>>>>
>>>>>>>>>> Let me explain how I see Nirmal's solution.
>>>>>>>>>>
>>>>>>>>>> We are taking only first and last values in a window for gradient
>>>>>>>>>> calculation. If there are more events in that window, and if that is a good
>>>>>>>>>> sample, tGap will be taking some higher values than 1s. This is usual
>>>>>>>>>> situation. Nirmal's fix is nothing to do with this scenario.
>>>>>>>>>>
>>>>>>>>>> If there are few events in that window(say only 2), they can be
>>>>>>>>>> really nearer. So there is a possibility that tGap can take a value near to
>>>>>>>>>> 0. In that case gradient will be a very big value. Hence, if the tGap is
>>>>>>>>>> very low (less than 1 second), then we can assume that the first and last
>>>>>>>>>> events actually occurred in 1 second gap. The reason why tGap is less than
>>>>>>>>>> 0 is, there are only few events in that window, and that sample is not good
>>>>>>>>>> enough. But this a rare situation, it can only happen once in a while.
>>>>>>>>>> Nirmal's fix is only dealing with this scenario. Fix imposes a lower limit
>>>>>>>>>> in the time interval between two events. This fix is really important to
>>>>>>>>>> deal with samples which are not good enough.
>>>>>>>>>>
>>>>>>>>>> By previous formula, we get -1000. This happened because the
>>>>>>>>>> sample is not good enough (may be there are few events). Fix makes it to
>>>>>>>>>> -5.
>>>>>>>>>>
>>>>>>>>>> So my understanding is that the solution is good enough for now.
>>>>>>>>>>
>>>>>>>>>> Thanks.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Nirmal: Can you please justify this?
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Nov 6, 2014 at 11:31 AM, Nirmal Fernando <
>>>>>>>>>>> nirmal070125@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Nov 6, 2014 at 6:05 AM, Gayan Gunarathne <
>>>>>>>>>>>> gayang@wso2.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> AFAIU this is not related with the gradient calculation
>>>>>>>>>>>>> formula that we used. But It is related with the how we calculate gradient
>>>>>>>>>>>>> when the denominator become a small number.IMO when the divisor
>>>>>>>>>>>>> (timeInSeconds) approaches to the small number we do need to normalize the
>>>>>>>>>>>>> result.
>>>>>>>>>>>>>
>>>>>>>>>>>>> IMO we do have max and min scale up and down values. So based
>>>>>>>>>>>>> on that we can calculate the gradient on that value range.Simply if the
>>>>>>>>>>>>> denominator approaches zero result approaches to infinity.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think that's what Nirmal doing here by making the lower
>>>>>>>>>>>>> threshold as "millisecondsForASecond"
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> You got it! thanks.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>         long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1
>>>>>>>>>>>>> : millisecondsForASecond;
>>>>>>>>>>>>>         double gradient = 0.0;
>>>>>>>>>>>>>         if (tGap > 0) {
>>>>>>>>>>>>>             gradient = ((lastVal - firstVal) *
>>>>>>>>>>>>> millisecondsForASecond) / tGap;
>>>>>>>>>>>>>         }
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Gayan
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Nov 6, 2014 at 9:22 AM, Rajkumar Rajaratnam <
>>>>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Nov 6, 2014 at 9:03 AM, Imesh Gunaratne <
>>>>>>>>>>>>>> imesh@apache.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Nirmal,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I do not think the milliseconds to seconds convertion is
>>>>>>>>>>>>>>> correct here.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As I see we are taking the difference of two timestamp
>>>>>>>>>>>>>>> values and then dividing it by 1000. The corect way might be to first
>>>>>>>>>>>>>>> divide each value by 1000 and then take the difference.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> No Imesh. There are no difference between *taking the the
>>>>>>>>>>>>>> difference and dividing it by 1000* and *dividing it by 1000
>>>>>>>>>>>>>> and taking the difference*.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On the other hand we might not need to convert these values
>>>>>>>>>>>>>>> to seconds since we are taking a time difference and calculating a gradient.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I did a quick test with the following sample:
>>>>>>>>>>>>>>> Gradient: -999.9999999999998 Last val: 7.000000000000001
>>>>>>>>>>>>>>> First val: 12.0 Time Gap: 5 t1: 1415213232152 t2: 1415213232157
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *According previous code:*
>>>>>>>>>>>>>>> long tGap = t2 - t1;
>>>>>>>>>>>>>>>         double gradient = 0.0;
>>>>>>>>>>>>>>>         if (tGap > 0) {
>>>>>>>>>>>>>>>             gradient = ((lastVal - firstVal) * 1000) / tGap;
>>>>>>>>>>>>>>>         }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7
>>>>>>>>>>>>>>> gradient: -1000.0
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *According to your fix:*
>>>>>>>>>>>>>>> long millisecondsForASecond = 1000;
>>>>>>>>>>>>>>>         long tGap = t2 - t1 > millisecondsForASecond ? t2 -
>>>>>>>>>>>>>>> t1 : millisecondsForASecond;
>>>>>>>>>>>>>>>         double gradient = 0.0;
>>>>>>>>>>>>>>>         if (tGap > 0) {
>>>>>>>>>>>>>>>             gradient = ((lastVal - firstVal) *
>>>>>>>>>>>>>>> millisecondsForASecond) / tGap;
>>>>>>>>>>>>>>>         }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7
>>>>>>>>>>>>>>> gradient: -5.0
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *According to an online gradient calculator:*
>>>>>>>>>>>>>>> gradient: -1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://www.calculator.net/slope-calculator.html?type=1&x11=1415213232152&y11=12&x12=1415213232157&y12=7&x=27&y=19
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> According to the online gradient calculator (assuming their
>>>>>>>>>>>>>>> calculation is correct), the calculation in your fix is not correct. I
>>>>>>>>>>>>>>> believe the logic should be simple as follows:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> long tGap = t2 - t1;
>>>>>>>>>>>>>>>         double gradient = 0.0;
>>>>>>>>>>>>>>>         if (tGap > 0) {
>>>>>>>>>>>>>>>             gradient = ((lastVal - firstVal)) / tGap;
>>>>>>>>>>>>>>>         }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7
>>>>>>>>>>>>>>> gradient: -1.0
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> And AFAIK, standard way to calculate gradient =
>>>>>>>>>>>>>> (v1-v2)/timeInSeconds.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Well, we can divide it by what ever time units we wants, but
>>>>>>>>>>>>>> the result's unit will be different.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In Nirmal's case, he get will get *stats/seconds*. In your
>>>>>>>>>>>>>> case, you will get *stats/milliseconds*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> And AS prediction is based on *seconds*, CEP should use
>>>>>>>>>>>>>> *seconds* for the calculation.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> wdyt?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Nov 6, 2014 at 1:32 AM, Nirmal Fernando <
>>>>>>>>>>>>>>> nirmal070125@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Guys,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> So, I got a chance to reproduce and analyze this issue.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> How we calculate gradient of two events?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Say the events are; e1(t1,v1) and e2(t2,v2)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> tx - time in *milliseconds* when xth event occurred
>>>>>>>>>>>>>>>> vx - value (memory, cpu etc.) that xth event carries
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *time gap = t(2-1) = t2 -t1 milliseconds*
>>>>>>>>>>>>>>>> time gap in seconds = t(2-1) = (t2 - t1)/1000 seconds
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hence,
>>>>>>>>>>>>>>>> *Gradient = (v2 - v1)  / t(2-1)  = ( (v2 - v1) * 1000 ) /
>>>>>>>>>>>>>>>> (t2 - t1)*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I've enabled debug logs for CEP extension;
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> log4j.logger.org.apache.stratos.cep.extension=DEBUG
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please find the following 3 logs extracted from the debug
>>>>>>>>>>>>>>>> logs;
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ===================================================================
>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,073] DEBUG
>>>>>>>>>>>>>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} -
>>>>>>>>>>>>>>>>  Gradient: -0.1996007984031936 Last val: 9.0 First val: 12.0 *Time
>>>>>>>>>>>>>>>> Gap: 15030* t1: 1415213202095 t2: 1415213217125 hash:
>>>>>>>>>>>>>>>> 155426542
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,073] DEBUG
>>>>>>>>>>>>>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} -  Gradient:
>>>>>>>>>>>>>>>> -999.9999999999998 Last val: 7.000000000000001 First val:
>>>>>>>>>>>>>>>> 12.0 *Time Gap: 5* t1: 1415213232152 t2: 1415213232157
>>>>>>>>>>>>>>>> hash: 155426542
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,074] DEBUG
>>>>>>>>>>>>>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} -
>>>>>>>>>>>>>>>>  Gradient: -44.34884666437174 Last val: -999.9999999999998 First val:
>>>>>>>>>>>>>>>> -0.1996007984031936 *Time Gap: 22544* t1: 1415213209610
>>>>>>>>>>>>>>>> t2: 1415213232154 hash: 155426542
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ===================================================================
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> So, as you can see the reason behind a large value is when
>>>>>>>>>>>>>>>> the time gap between two subjected events is less than 1 second. This could
>>>>>>>>>>>>>>>> happen since events are coming from different asynchronous agents and also
>>>>>>>>>>>>>>>> when there are less number of events.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> So, the fix I propose is a very simple one and it will not
>>>>>>>>>>>>>>>> compromise anything AFAIS.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Fix is to calculate time gap as follows;
>>>>>>>>>>>>>>>>                                                         *____
>>>>>>>>>>>>>>>> t2 - t1*
>>>>>>>>>>>>>>>>                                                         *|
>>>>>>>>>>>>>>>> yes?*
>>>>>>>>>>>>>>>> *time gap = t(2-1) = t2 -t1 > 1000 ---*
>>>>>>>>>>>>>>>>                                                         *|____
>>>>>>>>>>>>>>>> 1000*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have tested this and works fine.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ---------- Forwarded message ----------
>>>>>>>>>>>>>>>> From: Manula Chathurika Thantriwatte <manulac@wso2.com>
>>>>>>>>>>>>>>>> Date: Tue, Oct 28, 2014 at 5:16 AM
>>>>>>>>>>>>>>>> Subject: Re: CEP sends very large values for gradient and
>>>>>>>>>>>>>>>> second derivative of load average
>>>>>>>>>>>>>>>> To: "dev@stratos.apache.org" <dev@stratos.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Raj,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yes it has. But it's very easy to find the values from the
>>>>>>>>>>>>>>>> agent with the time stamp. Then we can search that time stamp in the CEP
>>>>>>>>>>>>>>>> trace logs and compare the summarized values.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Oct 27, 2014 at 9:23 PM, Rajkumar Rajaratnam <
>>>>>>>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Oct 27, 2014 at 7:56 PM, Manula Chathurika
>>>>>>>>>>>>>>>>> Thantriwatte <manulac@wso2.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Raj,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Shall we enable the cartridge agent debug logs. Then we
>>>>>>>>>>>>>>>>>> can compare both CEP and agent logs with the time stamp. Then we can narrow
>>>>>>>>>>>>>>>>>> down where the actual problem is.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Manula, cep-trace log have the values sent by cartridge
>>>>>>>>>>>>>>>>> agent also right?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Oct 27, 2014 at 5:06 PM, Rajkumar Rajaratnam <
>>>>>>>>>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Oct 27, 2014 at 12:53 PM, Manula Chathurika
>>>>>>>>>>>>>>>>>>> Thantriwatte <manulac@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi Raj,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Is this happen frequently or intermittently ? Once I go
>>>>>>>>>>>>>>>>>>>> though the logs I'm unable to find the high input values for load average.
>>>>>>>>>>>>>>>>>>>> But output have high values.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 22:10:38,698 [-] [Siddhi-Scheduler-pool-10-thread-1]
>>>>>>>>>>>>>>>>>>>>  INFO EVENT_TRACE_LOGGER TenantId=-1234 : Output Event Adaptor :
>>>>>>>>>>>>>>>>>>>> JMSOutputAdaptor, sent
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> {"org.apache.stratos.messaging.event.health.stat.SecondDerivativeOfLoadAverageEvent":{"message":{"clusterId":"php.php.domain","networkPartitionId":"","value":"2000000.0"}}}
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> This is happening intermittently. Experienced now too.
>>>>>>>>>>>>>>>>>>> This is a serious issue. We need to somehow sort this out.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Also networkPartitionId is empty. In the current
>>>>>>>>>>>>>>>>>>>> execution plans it took networkPartitionId for the calculation. IMO we need
>>>>>>>>>>>>>>>>>>>> to update the CEP execution plans for 4.1.0.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:49 AM, Manula Chathurika
>>>>>>>>>>>>>>>>>>>> Thantriwatte <manulac@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks Raj. I'll go through them.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:33 AM, Rajkumar Rajaratnam <
>>>>>>>>>>>>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi Manula,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> You can see those values in the same log I attached
>>>>>>>>>>>>>>>>>>>>>> in the previous mail.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:29 AM, Rajkumar Rajaratnam
>>>>>>>>>>>>>>>>>>>>>> <rajkumarr@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi Chamila,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I couldn't find any large values sent by agent.
>>>>>>>>>>>>>>>>>>>>>>> There are traces for CEP sending large values. Find the cep trace log here
>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> 1.
>>>>>>>>>>>>>>>>>>>>>>> https://drive.google.com/file/d/0B1haIleqJMHIS3FFVFpfa0JlbVE/view?usp=sharing
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:23 AM, Manula Chathurika
>>>>>>>>>>>>>>>>>>>>>>> Thantriwatte <manulac@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Shall we enable the cartridge agent debug logs and
>>>>>>>>>>>>>>>>>>>>>>>> see what are the load average values sent from the agent. If those values
>>>>>>>>>>>>>>>>>>>>>>>> are normal then we can narrow down to the CEP.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Sat, Oct 25, 2014 at 10:43 PM, Chamila De Alwis
>>>>>>>>>>>>>>>>>>>>>>>> <chamilad@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Hi Raj,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Can you inspect the CEP trace logs for the same
>>>>>>>>>>>>>>>>>>>>>>>>> time period? It will have the events published from the agent.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>>>> Chamila de Alwis
>>>>>>>>>>>>>>>>>>>>>>>>> Software Engineer | WSO2 | +94772207163
>>>>>>>>>>>>>>>>>>>>>>>>> Blog: code.chamiladealwis.com
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Sat, Oct 25, 2014 at 10:39 PM, Rajkumar
>>>>>>>>>>>>>>>>>>>>>>>>> Rajaratnam <rajkumarr@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I am doing a test round with M3 with python
>>>>>>>>>>>>>>>>>>>>>>>>>> agent. And faced this very strange scenario, $Subject.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,693]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,693]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Grad of load avg event: [cluster] php.php.domain [value] -2000.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Grad of load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Avg load avg event: [cluster] php.php.domain [value] 1.9090909
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Avg Memory Consumption event: [cluster] php.php.domain [value] 28.718182
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Second Derivation of Memory Consumption event: [cluster] php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>>>> [value] 200000.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Grad of load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Second Derivation of load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Second Derivation of load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Avg load avg event: [cluster] php.php.domain [value] 3.3636363
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Grad of Memory Consumption event: [cluster] php.php.domain [value]
>>>>>>>>>>>>>>>>>>>>>>>>>> 0.11670045
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Avg Memory Consumption event: [cluster] php.php.domain [value] 28.7
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Second Derivation of Memory Consumption event: [cluster] php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>>>> [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Avg Memory Consumption event: [cluster] php.php.domain [value] 28.7
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Avg load avg event: [cluster] php.php.domain [value] 5.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Grad of load avg event: [cluster] php.php.domain [value] 2000.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Avg Memory Consumption event: [cluster] php.php.domain [value] 28.7
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Grad of Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,699]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Second Derivation of Memory Consumption event: [cluster] php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>>>> [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,699]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Second Derivation of load avg event: [cluster] php.php.domain [value]
>>>>>>>>>>>>>>>>>>>>>>>>>> 2000000.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Grad of load avg event: [cluster] php.php.domain [value] 13500.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Grad of Memory Consumption event: [cluster] php.php.domain [value] 50.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Avg Memory Consumption event: [cluster] php.php.domain [value] 28.738462
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Avg load avg event: [cluster] php.php.domain [value] 2.3333333
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Grad of load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Grad of Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Avg load avg event: [cluster] php.php.domain [value] 31.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Grad of Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Avg load avg event: [cluster] php.php.domain [value] 39.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Avg load avg event: [cluster] php.php.domain [value] 39.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,705]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,706]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Grad of load avg event: [cluster] php.php.domain [value] -11000.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,706]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load
>>>>>>>>>>>>>>>>>>>>>>>>>> average stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,708]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Grad of Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,708]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,709]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Avg Memory Consumption event: [cluster] php.php.domain [value] 28.8
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,709]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,811]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  PodActivationWatcher running : Running
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,942]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  PodActivationWatcher running : Running
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,945]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  PodActivationWatcher running : Running
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,712]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG
>>>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  KubernetesServiceClusterMonitor is running..
>>>>>>>>>>>>>>>>>>>>>>>>>> KubernetesServiceClusterMonitor [ kubernetesHostClusterId=KubGrp1,
>>>>>>>>>>>>>>>>>>>>>>>>>> clusterId=php.php.domain, serviceId=php]
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,713]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG
>>>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Running min check for [kub-cluster] : KubGrp1 [cluster] : php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,713]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  Running minimum rule:
>>>>>>>>>>>>>>>>>>>>>>>>>> [kub-cluster] KubGrp1 [cluster] php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,714]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [min-check]
>>>>>>>>>>>>>>>>>>>>>>>>>>  [cluster] : php.php.domain [Replicas] nonTerminated : 3
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,714]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [min-check]
>>>>>>>>>>>>>>>>>>>>>>>>>>  [cluster] : php.php.domain [Replicas] minReplicas : 3
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  Running obsolete
>>>>>>>>>>>>>>>>>>>>>>>>>> containers rule [kub-cluster] : KubGrp1 [cluster] : php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [obsolete-check]
>>>>>>>>>>>>>>>>>>>>>>>>>> [cluster] : php.php.domain [Replicas] obsoleteReplicas : 0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.rule.AutoscalerRuleEvaluator} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Minimum check executed for :
>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.stratos.autoscaler.KubernetesClusterContext@21d906d0
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG
>>>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  flag of rifReset : true flag of memoryConsumptionReset : true flag of
>>>>>>>>>>>>>>>>>>>>>>>>>> loadAverageReset : true
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG
>>>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>>>  Running scale check for [kub-cluster] : KubGrp1 [cluster] : php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  Running scaling rule
>>>>>>>>>>>>>>>>>>>>>>>>>> [kub-cluster] : KubGrp1 [cluster] : php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Predicting
>>>>>>>>>>>>>>>>>>>>>>>>>> the value, [average]: 0.0 , [gradient]: 0.0 , [second derivative]: 0.0 ,
>>>>>>>>>>>>>>>>>>>>>>>>>> [time intervals]: 1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Predicting
>>>>>>>>>>>>>>>>>>>>>>>>>> the value, [average]: 28.8 , [gradient]: 0.0 , [second derivative]: 0.0 ,
>>>>>>>>>>>>>>>>>>>>>>>>>> [time intervals]: 1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Predicting
>>>>>>>>>>>>>>>>>>>>>>>>>> the value, [average]: 39.0 , [gradient]: -11000.0 , [second derivative]:
>>>>>>>>>>>>>>>>>>>>>>>>>> 2000000.0 , [time intervals]: 1
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain [Replicas] minReplicas : 3
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain [Replicas] maxReplicas : 10
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain [Replicas] nonTerminated : 3
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain [Replicas] activeReplicas : 3
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain [RequestInFlight] predicted value : 0.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain [RequestInFlight] upper limit : 80.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain [RequestInFlight] lower limit : 5.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain [MemoryConsumption] predicted value : 28.799999237060547
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain [MemoryConsumption] upper limit : 80.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain [MemoryConsumption] lower limit : 15.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain [LoadAverage] predicted value : 989039.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain [LoadAverage] upper limit : 180.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain [LoadAverage] lower limit : 20.0
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain scale-up action : true
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain scale-down action : false
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain [LoadAverage] predicted replicas : 16484
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster]
>>>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain predicted replicas > max replicas :
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling] Decided to
>>>>>>>>>>>>>>>>>>>>>>>>>> scale-up : [cluster] : php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling-up]
>>>>>>>>>>>>>>>>>>>>>>>>>>  [cluster] : php.php.domain valid number of replicas to expand : 10
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720]
>>>>>>>>>>>>>>>>>>>>>>>>>>  INFO
>>>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient}
>>>>>>>>>>>>>>>>>>>>>>>>>> -  Updating kubernetes replication controller via cloud controller:
>>>>>>>>>>>>>>>>>>>>>>>>>> [cluster] php.php.domain [replicas] 10
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,734]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl}
>>>>>>>>>>>>>>>>>>>>>>>>>> -  CloudControllerServiceImpl:updateContainers for cluster : php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,758]
>>>>>>>>>>>>>>>>>>>>>>>>>> DEBUG {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl}
>>>>>>>>>>>>>>>>>>>>>>>>>> -  Cloud Controller is delegating request to update a replication
>>>>>>>>>>>>>>>>>>>>>>>>>> controller php.php.domain to Kubernetes layer.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I couldn't investigate the agent side, since
>>>>>>>>>>>>>>>>>>>>>>>>>> containers are destroyed.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Any idea?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>>>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>>>>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> email : manulac@wso2.com / manula@apache.org
>>>>>>>>>>>>>>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> email : manulac@wso2.com / manula@apache.org
>>>>>>>>>>>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> email : manulac@wso2.com / manula@apache.org
>>>>>>>>>>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> email : manulac@wso2.com / manula@apache.org
>>>>>>>>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> email : manulac@wso2.com / manula@apache.org
>>>>>>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>> Nirmal
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Nirmal Fernando.
>>>>>>>>>>>>>>>> PPMC Member & Committer of Apache Stratos,
>>>>>>>>>>>>>>>> Senior Software Engineer, WSO2 Inc.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Raj
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>>
>>>>>>>>>>>>> Gayan Gunarathne
>>>>>>>>>>>>> Technical Lead
>>>>>>>>>>>>> WSO2 Inc. (http://wso2.com)
>>>>>>>>>>>>> email  : gayang@wso2.com  | mobile : +94 766819985
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>> Nirmal
>>>>>>>>>>>>
>>>>>>>>>>>> Nirmal Fernando.
>>>>>>>>>>>> PPMC Member & Committer of Apache Stratos,
>>>>>>>>>>>> Senior Software Engineer, WSO2 Inc.
>>>>>>>>>>>>
>>>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>
>>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Raj
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Raj
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Akila Ravihansa Perera
>>>>>>>> Software Engineer, WSO2
>>>>>>>>
>>>>>>>> Blog: http://ravihansa3000.blogspot.com
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards,
>>>>>>> Nirmal
>>>>>>>
>>>>>>> Nirmal Fernando.
>>>>>>> PPMC Member & Committer of Apache Stratos,
>>>>>>> Senior Software Engineer, WSO2 Inc.
>>>>>>>
>>>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Akila Ravihansa Perera
>>>>>> Software Engineer, WSO2
>>>>>>
>>>>>> Blog: http://ravihansa3000.blogspot.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>> Nirmal
>>>>>
>>>>> Nirmal Fernando.
>>>>> PPMC Member & Committer of Apache Stratos,
>>>>> Senior Software Engineer, WSO2 Inc.
>>>>>
>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Lahiru Sandaruwan
>>> Committer and PMC member, Apache Stratos,
>>> Senior Software Engineer,
>>> WSO2 Inc., http://wso2.com
>>> lean.enterprise.middleware
>>>
>>> email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>
>>>
>>
>>
>> --
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>> Blog: http://nirmalfdo.blogspot.com/
>>
>
>
>
> --
> --
> Lahiru Sandaruwan
> Committer and PMC member, Apache Stratos,
> Senior Software Engineer,
> WSO2 Inc., http://wso2.com
> lean.enterprise.middleware
>
> email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>
>


-- 
Lakmal Warusawithana
Vice President, Apache Stratos
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692
Blog : http://lakmalsview.blogspot.com/

Mime
View raw message