stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lahiru Sandaruwan <lahi...@wso2.com>
Subject Re: [Fixed] CEP sends very large values for gradient and second derivative of load average
Date Thu, 06 Nov 2014 16:28:15 GMT
On Thu, Nov 6, 2014 at 7:15 PM, Lahiru Sandaruwan <lahirus@wso2.com> wrote:

> Hi all,
>
> Very interesting discussion and thanks Nirmal for initiating this!
>
> I also did some research on this.
> I think best approach to find gradient and second derivative is to "curve
> fitting". Refer "Locally weighted linear regression" section at [1]. We
> should be calculating gradient and second derivative by differentiating the
> equation of fitted line.
>
> We would need a second degree polynomial fitter for this, where we can use
> Apache commons math library for this. Refer the sample at [2], we need to
> run this with degree 2 instead of 3.
>
> E.g.
> So if get degree 2 polynomial fitter, we will have an equation like below
> where value(v) is our statistic value, time(t) is the time of event,
> gradient(g), and second derivative(sd).
>
> Equation we get from received events,
> v = a*t*t + b*t + c
>
> g = 2*a*t
> sd = 2*a
>
> We can get *g* by applying the median of the time stamp sample.
>
> I have run a sample successfully. I can complete the work on CEP extension
> if you guys agree.
>
> In addition to this, we should also restrict the sample size as Akila
> suggested. Ideally we should receive 6 events per minute for memberwise
> stats and "6 * no. of instances" for clusterwise stats. We should put a
> constraint with this much of percentage(say 80%) should be there to
> calculate *g *and* sd.*
>
> Also we should not calculated gradient of stats sent from different
> members in clusterwise scenario as Imesh mentioned. That causes this type
> of small time gap issues.
>

I have thought of a solution for this. Also most importantly the solution i
propose suits very well for the prediction requirement at Autoscaler, than
what we have now. Please go through and give feedback.

*CEP's task*

   - Find memberwise curves that fits events received in 1 minute(or
   required) window at CEP
   - We send the parameters of fitted line(a, b, and c in above equation)
   with the timestamp of last event(T) in the window, to Autoscaler

*Autoscaler's task*

   - Autoscaler use *v = a*t*t + b*t + c *function to predict the value in
   any timestamp from the last timestamp

E.g. Say we need to find the value(v) after 1 minute(assuming we carried
all the calculations in milliseconds),

v = a * (T+60000) * (T+60000) + b * (T+60000) + c

   - So we have memberwise predictions and we can find clusterwise
   prediction by averaging all the memberwise values.


Please send your thoughts.

Thanks.



> Thanks.
>
> [1] http://cs229.stanford.edu/notes/cs229-notes1.pdf
> [2] http://commons.apache.org/proper/commons-math/userguide/fitting.html
>
> On Thu, Nov 6, 2014 at 6:41 PM, Imesh Gunaratne <imesh@apache.org> wrote:
>
>> Raj: Thanks for the clarification, it was my misunderstanding.
>>
>> I did further analysis on this issue, seems like the root cause of this
>> problem resides somewhere else.
>>
>> Currently we have set the cartridge agent's statistics publishing
>> interval to 15 seconds. If so how do we receive series of events within a
>> second? Seems like we are considering statistics events coming from
>> different members for calculating the gradient. This might not be correct,
>> if we were to draw a curve of a data set and consider the gradient of those
>> values, those values may have come from the same source. Otherwise the
>> gradient value we are calculating might not be correct.
>>
>> Thanks
>>
>> On Thu, Nov 6, 2014 at 2:05 PM, Udara Liyanage <udara@wso2.com> wrote:
>>
>>> Hi,
>>>
>>> Imagine we put the values in a graph (value against time). The gradient
>>> is the slope of the line. When we draw the line via two very near dots, we
>>> get a high slope/gradient even for a low increase of the values. This
>>> happens when we have only events which are occurred in very near time
>>> intervals.
>>>
>>> On the other hands, when events are distributed among long time period
>>> (are more counts in sample) we don't get this high gradient since the line
>>> goes via higher dots count where we get a reasonable value for gradient.
>>>
>>> As I understood the scenario, it is reasonable to make time gap equals
>>> to millisInSeconds in case time gap is very small.
>>>
>>> Touched, not typed. Erroneous words are a feature, not a typo.
>>> I'd rather keep it simple Akila (this would complex the scenario of
>>> second derivative too) :-) Isn't it?
>>>
>>> On Thu, Nov 6, 2014 at 9:03 AM, Akila Ravihansa Perera <
>>> ravihansa@wso2.com> wrote:
>>>
>>>> Until t2 - t1 > 1
>>>>
>>>> On Thu, Nov 6, 2014 at 12:50 AM, Nirmal Fernando <
>>>> nirmal070125@gmail.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Thu, Nov 6, 2014 at 8:33 AM, Akila Ravihansa Perera <
>>>>> ravihansa@wso2.com> wrote:
>>>>>
>>>>>> Thanks for the explanation, Nirmal. It makes sense now.
>>>>>>
>>>>>> Perhaps as an alternative solution, window processor can be set to
>>>>>> wait until sufficient events have arrived to calculate the gradient?
>>>>>>
>>>>>
>>>>> What would be the sufficient value? Theoretically in gradient case
>>>>> it's 2, in second derivative case it's 3. And we're doing that already.
>>>>>
>>>>>>
>>>>>> On Wed, Nov 5, 2014 at 11:48 PM, Rajkumar Rajaratnam <
>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>
>>>>>>> In other words, fix reduces the risks which are coming from bad
>>>>>>> samples.
>>>>>>>
>>>>>>> On Thu, Nov 6, 2014 at 12:08 PM, Rajkumar Rajaratnam <
>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Nov 6, 2014 at 11:38 AM, Imesh Gunaratne <imesh@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> On Thu, Nov 6, 2014 at 9:22 AM, Rajkumar Rajaratnam <
>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> No Imesh. There are no difference between *taking the the
>>>>>>>>>> difference and dividing it by 1000* and *dividing it by 1000 and
>>>>>>>>>> taking the difference*.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Raj: I agree, in this scenario it makes no difference, however
>>>>>>>>> when we are applying a fomular the unit conversion should happen prior to
>>>>>>>>> applying values.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Not exactly Imesh. We can do operations if operands are in the same
>>>>>>>> units. So we can do subtract operation in milliseconds. And then we can do
>>>>>>>> division operation to make it in seconds. On the other hand, we can do a
>>>>>>>> division operation to make each operands to seconds, then since both are
>>>>>>>> seconds, we can do a subtract operation. So we can apply the formula if
>>>>>>>> operands are in same unit. I guess this is same for all scenarios.
>>>>>>>>
>>>>>>>>
>>>>>>>>> Still I see a problem with this correction, take the same example:
>>>>>>>>> Gradient: -999.9999999999998 Last val: 7.000000000000001 First
>>>>>>>>> val: 12.0 Time Gap: 5 t1: 1415213232152 t2: 1415213232157
>>>>>>>>>
>>>>>>>>> *Manual calulation:*
>>>>>>>>> The value difference: 7 - 12 = -5
>>>>>>>>> Time difference: 1415213232157 - 1415213232152 = 5 ms => 5/1000 =
>>>>>>>>> 0.005 sec
>>>>>>>>> *Gradient = - 5/0.005 = -1000*
>>>>>>>>>
>>>>>>>>> *According to the current fix:*
>>>>>>>>> long millisecondsForASecond = 1000;
>>>>>>>>>         long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1 :
>>>>>>>>> millisecondsForASecond;
>>>>>>>>>         double gradient = 0.0;
>>>>>>>>>         if (tGap > 0) {
>>>>>>>>>             gradient = ((lastVal - firstVal) *
>>>>>>>>> millisecondsForASecond) / tGap;
>>>>>>>>>         }
>>>>>>>>>
>>>>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7
>>>>>>>>> Gradient = -5 * 1000 / 1000 = - 5
>>>>>>>>>
>>>>>>>>
>>>>>>>> IMO, this is a sufficient solution.
>>>>>>>>
>>>>>>>> Let me explain how I see Nirmal's solution.
>>>>>>>>
>>>>>>>> We are taking only first and last values in a window for gradient
>>>>>>>> calculation. If there are more events in that window, and if that is a good
>>>>>>>> sample, tGap will be taking some higher values than 1s. This is usual
>>>>>>>> situation. Nirmal's fix is nothing to do with this scenario.
>>>>>>>>
>>>>>>>> If there are few events in that window(say only 2), they can be
>>>>>>>> really nearer. So there is a possibility that tGap can take a value near to
>>>>>>>> 0. In that case gradient will be a very big value. Hence, if the tGap is
>>>>>>>> very low (less than 1 second), then we can assume that the first and last
>>>>>>>> events actually occurred in 1 second gap. The reason why tGap is less than
>>>>>>>> 0 is, there are only few events in that window, and that sample is not good
>>>>>>>> enough. But this a rare situation, it can only happen once in a while.
>>>>>>>> Nirmal's fix is only dealing with this scenario. Fix imposes a lower limit
>>>>>>>> in the time interval between two events. This fix is really important to
>>>>>>>> deal with samples which are not good enough.
>>>>>>>>
>>>>>>>> By previous formula, we get -1000. This happened because the sample
>>>>>>>> is not good enough (may be there are few events). Fix makes it to -5.
>>>>>>>>
>>>>>>>> So my understanding is that the solution is good enough for now.
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Nirmal: Can you please justify this?
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>> On Thu, Nov 6, 2014 at 11:31 AM, Nirmal Fernando <
>>>>>>>>> nirmal070125@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Nov 6, 2014 at 6:05 AM, Gayan Gunarathne <gayang@wso2.com
>>>>>>>>>> > wrote:
>>>>>>>>>>
>>>>>>>>>>> AFAIU this is not related with the gradient calculation formula
>>>>>>>>>>> that we used. But It is related with the how we calculate gradient when the
>>>>>>>>>>> denominator become a small number.IMO when the divisor (timeInSeconds)
>>>>>>>>>>> approaches to the small number we do need to normalize the result.
>>>>>>>>>>>
>>>>>>>>>>> IMO we do have max and min scale up and down values. So based on
>>>>>>>>>>> that we can calculate the gradient on that value range.Simply if the
>>>>>>>>>>> denominator approaches zero result approaches to infinity.
>>>>>>>>>>>
>>>>>>>>>>> I think that's what Nirmal doing here by making the lower
>>>>>>>>>>> threshold as "millisecondsForASecond"
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> You got it! thanks.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>         long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1 :
>>>>>>>>>>> millisecondsForASecond;
>>>>>>>>>>>         double gradient = 0.0;
>>>>>>>>>>>         if (tGap > 0) {
>>>>>>>>>>>             gradient = ((lastVal - firstVal) *
>>>>>>>>>>> millisecondsForASecond) / tGap;
>>>>>>>>>>>         }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Gayan
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Nov 6, 2014 at 9:22 AM, Rajkumar Rajaratnam <
>>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Nov 6, 2014 at 9:03 AM, Imesh Gunaratne <
>>>>>>>>>>>> imesh@apache.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Nirmal,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I do not think the milliseconds to seconds convertion is
>>>>>>>>>>>>> correct here.
>>>>>>>>>>>>>
>>>>>>>>>>>>> As I see we are taking the difference of two timestamp values
>>>>>>>>>>>>> and then dividing it by 1000. The corect way might be to first divide each
>>>>>>>>>>>>> value by 1000 and then take the difference.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> No Imesh. There are no difference between *taking the the
>>>>>>>>>>>> difference and dividing it by 1000* and *dividing it by 1000
>>>>>>>>>>>> and taking the difference*.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> On the other hand we might not need to convert these values to
>>>>>>>>>>>>> seconds since we are taking a time difference and calculating a gradient.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I did a quick test with the following sample:
>>>>>>>>>>>>> Gradient: -999.9999999999998 Last val: 7.000000000000001 First
>>>>>>>>>>>>> val: 12.0 Time Gap: 5 t1: 1415213232152 t2: 1415213232157
>>>>>>>>>>>>>
>>>>>>>>>>>>> *According previous code:*
>>>>>>>>>>>>> long tGap = t2 - t1;
>>>>>>>>>>>>>         double gradient = 0.0;
>>>>>>>>>>>>>         if (tGap > 0) {
>>>>>>>>>>>>>             gradient = ((lastVal - firstVal) * 1000) / tGap;
>>>>>>>>>>>>>         }
>>>>>>>>>>>>>
>>>>>>>>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7
>>>>>>>>>>>>> gradient: -1000.0
>>>>>>>>>>>>>
>>>>>>>>>>>>> *According to your fix:*
>>>>>>>>>>>>> long millisecondsForASecond = 1000;
>>>>>>>>>>>>>         long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1
>>>>>>>>>>>>> : millisecondsForASecond;
>>>>>>>>>>>>>         double gradient = 0.0;
>>>>>>>>>>>>>         if (tGap > 0) {
>>>>>>>>>>>>>             gradient = ((lastVal - firstVal) *
>>>>>>>>>>>>> millisecondsForASecond) / tGap;
>>>>>>>>>>>>>         }
>>>>>>>>>>>>>
>>>>>>>>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7
>>>>>>>>>>>>> gradient: -5.0
>>>>>>>>>>>>>
>>>>>>>>>>>>> *According to an online gradient calculator:*
>>>>>>>>>>>>> gradient: -1
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://www.calculator.net/slope-calculator.html?type=1&x11=1415213232152&y11=12&x12=1415213232157&y12=7&x=27&y=19
>>>>>>>>>>>>>
>>>>>>>>>>>>> According to the online gradient calculator (assuming their
>>>>>>>>>>>>> calculation is correct), the calculation in your fix is not correct. I
>>>>>>>>>>>>> believe the logic should be simple as follows:
>>>>>>>>>>>>>
>>>>>>>>>>>>> long tGap = t2 - t1;
>>>>>>>>>>>>>         double gradient = 0.0;
>>>>>>>>>>>>>         if (tGap > 0) {
>>>>>>>>>>>>>             gradient = ((lastVal - firstVal)) / tGap;
>>>>>>>>>>>>>         }
>>>>>>>>>>>>>
>>>>>>>>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7
>>>>>>>>>>>>> gradient: -1.0
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> And AFAIK, standard way to calculate gradient =
>>>>>>>>>>>> (v1-v2)/timeInSeconds.
>>>>>>>>>>>>
>>>>>>>>>>>> Well, we can divide it by what ever time units we wants, but
>>>>>>>>>>>> the result's unit will be different.
>>>>>>>>>>>>
>>>>>>>>>>>> In Nirmal's case, he get will get *stats/seconds*. In your
>>>>>>>>>>>> case, you will get *stats/milliseconds*
>>>>>>>>>>>>
>>>>>>>>>>>> And AS prediction is based on *seconds*, CEP should use
>>>>>>>>>>>> *seconds* for the calculation.
>>>>>>>>>>>>
>>>>>>>>>>>> wdyt?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Nov 6, 2014 at 1:32 AM, Nirmal Fernando <
>>>>>>>>>>>>> nirmal070125@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Guys,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So, I got a chance to reproduce and analyze this issue.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> How we calculate gradient of two events?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Say the events are; e1(t1,v1) and e2(t2,v2)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> tx - time in *milliseconds* when xth event occurred
>>>>>>>>>>>>>> vx - value (memory, cpu etc.) that xth event carries
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *time gap = t(2-1) = t2 -t1 milliseconds*
>>>>>>>>>>>>>> time gap in seconds = t(2-1) = (t2 - t1)/1000 seconds
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hence,
>>>>>>>>>>>>>> *Gradient = (v2 - v1)  / t(2-1)  = ( (v2 - v1) * 1000 ) / (t2
>>>>>>>>>>>>>> - t1)*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I've enabled debug logs for CEP extension;
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> log4j.logger.org.apache.stratos.cep.extension=DEBUG
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please find the following 3 logs extracted from the debug
>>>>>>>>>>>>>> logs;
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ===================================================================
>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,073] DEBUG
>>>>>>>>>>>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} -
>>>>>>>>>>>>>>  Gradient: -0.1996007984031936 Last val: 9.0 First val: 12.0 *Time
>>>>>>>>>>>>>> Gap: 15030* t1: 1415213202095 t2: 1415213217125 hash:
>>>>>>>>>>>>>> 155426542
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,073] DEBUG
>>>>>>>>>>>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} -  Gradient:
>>>>>>>>>>>>>> -999.9999999999998 Last val: 7.000000000000001 First val:
>>>>>>>>>>>>>> 12.0 *Time Gap: 5* t1: 1415213232152 t2: 1415213232157 hash:
>>>>>>>>>>>>>> 155426542
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,074] DEBUG
>>>>>>>>>>>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} -
>>>>>>>>>>>>>>  Gradient: -44.34884666437174 Last val: -999.9999999999998 First val:
>>>>>>>>>>>>>> -0.1996007984031936 *Time Gap: 22544* t1: 1415213209610 t2:
>>>>>>>>>>>>>> 1415213232154 hash: 155426542
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ===================================================================
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So, as you can see the reason behind a large value is when
>>>>>>>>>>>>>> the time gap between two subjected events is less than 1 second. This could
>>>>>>>>>>>>>> happen since events are coming from different asynchronous agents and also
>>>>>>>>>>>>>> when there are less number of events.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So, the fix I propose is a very simple one and it will not
>>>>>>>>>>>>>> compromise anything AFAIS.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Fix is to calculate time gap as follows;
>>>>>>>>>>>>>>                                                         *____
>>>>>>>>>>>>>> t2 - t1*
>>>>>>>>>>>>>>                                                         *|
>>>>>>>>>>>>>> yes?*
>>>>>>>>>>>>>> *time gap = t(2-1) = t2 -t1 > 1000 ---*
>>>>>>>>>>>>>>                                                         *|____
>>>>>>>>>>>>>> 1000*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have tested this and works fine.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ---------- Forwarded message ----------
>>>>>>>>>>>>>> From: Manula Chathurika Thantriwatte <manulac@wso2.com>
>>>>>>>>>>>>>> Date: Tue, Oct 28, 2014 at 5:16 AM
>>>>>>>>>>>>>> Subject: Re: CEP sends very large values for gradient and
>>>>>>>>>>>>>> second derivative of load average
>>>>>>>>>>>>>> To: "dev@stratos.apache.org" <dev@stratos.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Raj,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yes it has. But it's very easy to find the values from the
>>>>>>>>>>>>>> agent with the time stamp. Then we can search that time stamp in the CEP
>>>>>>>>>>>>>> trace logs and compare the summarized values.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Oct 27, 2014 at 9:23 PM, Rajkumar Rajaratnam <
>>>>>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Oct 27, 2014 at 7:56 PM, Manula Chathurika
>>>>>>>>>>>>>>> Thantriwatte <manulac@wso2.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Raj,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Shall we enable the cartridge agent debug logs. Then we can
>>>>>>>>>>>>>>>> compare both CEP and agent logs with the time stamp. Then we can narrow
>>>>>>>>>>>>>>>> down where the actual problem is.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Manula, cep-trace log have the values sent by cartridge
>>>>>>>>>>>>>>> agent also right?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Oct 27, 2014 at 5:06 PM, Rajkumar Rajaratnam <
>>>>>>>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Oct 27, 2014 at 12:53 PM, Manula Chathurika
>>>>>>>>>>>>>>>>> Thantriwatte <manulac@wso2.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Raj,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Is this happen frequently or intermittently ? Once I go
>>>>>>>>>>>>>>>>>> though the logs I'm unable to find the high input values for load average.
>>>>>>>>>>>>>>>>>> But output have high values.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 22:10:38,698 [-] [Siddhi-Scheduler-pool-10-thread-1]
>>>>>>>>>>>>>>>>>>  INFO EVENT_TRACE_LOGGER TenantId=-1234 : Output Event Adaptor :
>>>>>>>>>>>>>>>>>> JMSOutputAdaptor, sent
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> {"org.apache.stratos.messaging.event.health.stat.SecondDerivativeOfLoadAverageEvent":{"message":{"clusterId":"php.php.domain","networkPartitionId":"","value":"2000000.0"}}}
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> This is happening intermittently. Experienced now too.
>>>>>>>>>>>>>>>>> This is a serious issue. We need to somehow sort this out.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Also networkPartitionId is empty. In the current
>>>>>>>>>>>>>>>>>> execution plans it took networkPartitionId for the calculation. IMO we need
>>>>>>>>>>>>>>>>>> to update the CEP execution plans for 4.1.0.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:49 AM, Manula Chathurika
>>>>>>>>>>>>>>>>>> Thantriwatte <manulac@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks Raj. I'll go through them.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:33 AM, Rajkumar Rajaratnam <
>>>>>>>>>>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi Manula,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> You can see those values in the same log I attached in
>>>>>>>>>>>>>>>>>>>> the previous mail.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:29 AM, Rajkumar Rajaratnam <
>>>>>>>>>>>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi Chamila,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I couldn't find any large values sent by agent. There
>>>>>>>>>>>>>>>>>>>>> are traces for CEP sending large values. Find the cep trace log here [1]
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 1.
>>>>>>>>>>>>>>>>>>>>> https://drive.google.com/file/d/0B1haIleqJMHIS3FFVFpfa0JlbVE/view?usp=sharing
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:23 AM, Manula Chathurika
>>>>>>>>>>>>>>>>>>>>> Thantriwatte <manulac@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Shall we enable the cartridge agent debug logs and
>>>>>>>>>>>>>>>>>>>>>> see what are the load average values sent from the agent. If those values
>>>>>>>>>>>>>>>>>>>>>> are normal then we can narrow down to the CEP.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Sat, Oct 25, 2014 at 10:43 PM, Chamila De Alwis <
>>>>>>>>>>>>>>>>>>>>>> chamilad@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi Raj,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Can you inspect the CEP trace logs for the same time
>>>>>>>>>>>>>>>>>>>>>>> period? It will have the events published from the agent.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>> Chamila de Alwis
>>>>>>>>>>>>>>>>>>>>>>> Software Engineer | WSO2 | +94772207163
>>>>>>>>>>>>>>>>>>>>>>> Blog: code.chamiladealwis.com
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Sat, Oct 25, 2014 at 10:39 PM, Rajkumar
>>>>>>>>>>>>>>>>>>>>>>> Rajaratnam <rajkumarr@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I am doing a test round with M3 with python agent.
>>>>>>>>>>>>>>>>>>>>>>>> And faced this very strange scenario, $Subject.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,693] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,693] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] -2000.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 1.9090909
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.718182
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Second
>>>>>>>>>>>>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster] php.php.domain [value]
>>>>>>>>>>>>>>>>>>>>>>>> 200000.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Second
>>>>>>>>>>>>>>>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Second
>>>>>>>>>>>>>>>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 3.3636363
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.11670045
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.7
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Second
>>>>>>>>>>>>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.7
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 5.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 2000.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.7
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,699] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Second
>>>>>>>>>>>>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,699] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Second
>>>>>>>>>>>>>>>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain [value] 2000000.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 13500.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 50.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.738462
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 2.3333333
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 31.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 39.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 39.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,705] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,706] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] -11000.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,706] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,708] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,708] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,709] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.8
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,709] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,811] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>>>>>>>>>>>>>>>>>>>>>>>>  PodActivationWatcher running : Running
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,942] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>>>>>>>>>>>>>>>>>>>>>>>>  PodActivationWatcher running : Running
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,945] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>>>>>>>>>>>>>>>>>>>>>>>>  PodActivationWatcher running : Running
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,712] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>  KubernetesServiceClusterMonitor is running..
>>>>>>>>>>>>>>>>>>>>>>>> KubernetesServiceClusterMonitor [ kubernetesHostClusterId=KubGrp1,
>>>>>>>>>>>>>>>>>>>>>>>> clusterId=php.php.domain, serviceId=php]
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,713] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>  Running min check for [kub-cluster] : KubGrp1 [cluster] : php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,713]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  Running minimum rule:
>>>>>>>>>>>>>>>>>>>>>>>> [kub-cluster] KubGrp1 [cluster] php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,714]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [min-check]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] nonTerminated : 3
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,714]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [min-check]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] minReplicas : 3
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  Running obsolete containers
>>>>>>>>>>>>>>>>>>>>>>>> rule [kub-cluster] : KubGrp1 [cluster] : php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [obsolete-check] [cluster]
>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain [Replicas] obsoleteReplicas : 0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.AutoscalerRuleEvaluator} -  Minimum
>>>>>>>>>>>>>>>>>>>>>>>> check executed for :
>>>>>>>>>>>>>>>>>>>>>>>> org.apache.stratos.autoscaler.KubernetesClusterContext@21d906d0
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>  flag of rifReset : true flag of memoryConsumptionReset : true flag of
>>>>>>>>>>>>>>>>>>>>>>>> loadAverageReset : true
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} -
>>>>>>>>>>>>>>>>>>>>>>>>  Running scale check for [kub-cluster] : KubGrp1 [cluster] : php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  Running scaling rule
>>>>>>>>>>>>>>>>>>>>>>>> [kub-cluster] : KubGrp1 [cluster] : php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Predicting the
>>>>>>>>>>>>>>>>>>>>>>>> value, [average]: 0.0 , [gradient]: 0.0 , [second derivative]: 0.0 , [time
>>>>>>>>>>>>>>>>>>>>>>>> intervals]: 1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Predicting the
>>>>>>>>>>>>>>>>>>>>>>>> value, [average]: 28.8 , [gradient]: 0.0 , [second derivative]: 0.0 , [time
>>>>>>>>>>>>>>>>>>>>>>>> intervals]: 1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Predicting the
>>>>>>>>>>>>>>>>>>>>>>>> value, [average]: 39.0 , [gradient]: -11000.0 , [second derivative]:
>>>>>>>>>>>>>>>>>>>>>>>> 2000000.0 , [time intervals]: 1
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] minReplicas : 3
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] maxReplicas : 10
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] nonTerminated : 3
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] activeReplicas : 3
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [RequestInFlight] predicted value : 0.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [RequestInFlight] upper limit : 80.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [RequestInFlight] lower limit : 5.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [MemoryConsumption] predicted value : 28.799999237060547
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [MemoryConsumption] upper limit : 80.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [MemoryConsumption] lower limit : 15.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [LoadAverage] predicted value : 989039.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [LoadAverage] upper limit : 180.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [LoadAverage] lower limit : 20.0
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain scale-up action : true
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain scale-down action : false
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [LoadAverage] predicted replicas : 16484
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain predicted replicas > max replicas :
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling] Decided to
>>>>>>>>>>>>>>>>>>>>>>>> scale-up : [cluster] : php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling-up]  [cluster] :
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain valid number of replicas to expand : 10
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720]  INFO
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient}
>>>>>>>>>>>>>>>>>>>>>>>> -  Updating kubernetes replication controller via cloud controller:
>>>>>>>>>>>>>>>>>>>>>>>> [cluster] php.php.domain [replicas] 10
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,734] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} -
>>>>>>>>>>>>>>>>>>>>>>>>  CloudControllerServiceImpl:updateContainers for cluster : php.php.domain
>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,758] DEBUG
>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} -
>>>>>>>>>>>>>>>>>>>>>>>>  Cloud Controller is delegating request to update a replication controller
>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain to Kubernetes layer.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I couldn't investigate the agent side, since
>>>>>>>>>>>>>>>>>>>>>>>> containers are destroyed.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Any idea?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> email : manulac@wso2.com / manula@apache.org
>>>>>>>>>>>>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> email : manulac@wso2.com / manula@apache.org
>>>>>>>>>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> email : manulac@wso2.com / manula@apache.org
>>>>>>>>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> email : manulac@wso2.com / manula@apache.org
>>>>>>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> email : manulac@wso2.com / manula@apache.org
>>>>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>> Nirmal
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Nirmal Fernando.
>>>>>>>>>>>>>> PPMC Member & Committer of Apache Stratos,
>>>>>>>>>>>>>> Senior Software Engineer, WSO2 Inc.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>
>>>>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Raj
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>>
>>>>>>>>>>> Gayan Gunarathne
>>>>>>>>>>> Technical Lead
>>>>>>>>>>> WSO2 Inc. (http://wso2.com)
>>>>>>>>>>> email  : gayang@wso2.com  | mobile : +94 766819985
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Best Regards,
>>>>>>>>>> Nirmal
>>>>>>>>>>
>>>>>>>>>> Nirmal Fernando.
>>>>>>>>>> PPMC Member & Committer of Apache Stratos,
>>>>>>>>>> Senior Software Engineer, WSO2 Inc.
>>>>>>>>>>
>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Imesh Gunaratne
>>>>>>>>>
>>>>>>>>> Technical Lead, WSO2
>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Raj
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Raj
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Akila Ravihansa Perera
>>>>>> Software Engineer, WSO2
>>>>>>
>>>>>> Blog: http://ravihansa3000.blogspot.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>> Nirmal
>>>>>
>>>>> Nirmal Fernando.
>>>>> PPMC Member & Committer of Apache Stratos,
>>>>> Senior Software Engineer, WSO2 Inc.
>>>>>
>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Akila Ravihansa Perera
>>>> Software Engineer, WSO2
>>>>
>>>> Blog: http://ravihansa3000.blogspot.com
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Nirmal
>>>
>>> Nirmal Fernando.
>>> PPMC Member & Committer of Apache Stratos,
>>> Senior Software Engineer, WSO2 Inc.
>>>
>>> Blog: http://nirmalfdo.blogspot.com/
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
>
> --
> --
> Lahiru Sandaruwan
> Committer and PMC member, Apache Stratos,
> Senior Software Engineer,
> WSO2 Inc., http://wso2.com
> lean.enterprise.middleware
>
> email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>
>


-- 
--
Lahiru Sandaruwan
Committer and PMC member, Apache Stratos,
Senior Software Engineer,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware

email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146

Mime
View raw message