stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manula Chathurika Thantriwatte <manu...@wso2.com>
Subject Re: CEP sends very large values for gradient and second derivative of load average
Date Tue, 28 Oct 2014 04:16:12 GMT
Hi Raj,

Yes it has. But it's very easy to find the values from the agent with the
time stamp. Then we can search that time stamp in the CEP trace logs and
compare the summarized values.

Thanks !

On Mon, Oct 27, 2014 at 9:23 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com>
wrote:

>
>
> On Mon, Oct 27, 2014 at 7:56 PM, Manula Chathurika Thantriwatte <
> manulac@wso2.com> wrote:
>
>> Hi Raj,
>>
>> Shall we enable the cartridge agent debug logs. Then we can compare both
>> CEP and agent logs with the time stamp. Then we can narrow down where the
>> actual problem is.
>>
>
> Manula, cep-trace log have the values sent by cartridge agent also right?
>
>
>>
>> Thanks !
>>
>> On Mon, Oct 27, 2014 at 5:06 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com>
>> wrote:
>>
>>>
>>>
>>> On Mon, Oct 27, 2014 at 12:53 PM, Manula Chathurika Thantriwatte <
>>> manulac@wso2.com> wrote:
>>>
>>>> Hi Raj,
>>>>
>>>> Is this happen frequently or intermittently ? Once I go though the logs
>>>> I'm unable to find the high input values for load average. But output have
>>>> high values.
>>>>
>>>> 22:10:38,698 [-] [Siddhi-Scheduler-pool-10-thread-1]  INFO
>>>> EVENT_TRACE_LOGGER TenantId=-1234 : Output Event Adaptor :
>>>> JMSOutputAdaptor, sent
>>>>
>>>> {"org.apache.stratos.messaging.event.health.stat.SecondDerivativeOfLoadAverageEvent":{"message":{"clusterId":"php.php.domain","networkPartitionId":"","value":"2000000.0"}}}
>>>>
>>>
>>> This is happening intermittently. Experienced now too. This is a serious
>>> issue. We need to somehow sort this out.
>>>
>>>
>>>>
>>>> Also networkPartitionId is empty. In the current execution plans it
>>>> took networkPartitionId for the calculation. IMO we need to update the CEP
>>>> execution plans for 4.1.0.
>>>>
>>>> Thanks !
>>>>
>>>>
>>>> On Sun, Oct 26, 2014 at 8:49 AM, Manula Chathurika Thantriwatte <
>>>> manulac@wso2.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Thanks Raj. I'll go through them.
>>>>>
>>>>> Thanks !
>>>>>
>>>>> On Sun, Oct 26, 2014 at 8:33 AM, Rajkumar Rajaratnam <
>>>>> rajkumarr@wso2.com> wrote:
>>>>>
>>>>>> Hi Manula,
>>>>>>
>>>>>> You can see those values in the same log I attached in the previous
>>>>>> mail.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> On Sun, Oct 26, 2014 at 8:29 AM, Rajkumar Rajaratnam <
>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>
>>>>>>> Hi Chamila,
>>>>>>>
>>>>>>> I couldn't find any large values sent by agent. There are traces
for
>>>>>>> CEP sending large values. Find the cep trace log here [1]
>>>>>>>
>>>>>>> 1.
>>>>>>> https://drive.google.com/file/d/0B1haIleqJMHIS3FFVFpfa0JlbVE/view?usp=sharing
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>> On Sun, Oct 26, 2014 at 8:23 AM, Manula Chathurika Thantriwatte
<
>>>>>>> manulac@wso2.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Shall we enable the cartridge agent debug logs and see what
are the
>>>>>>>> load average values sent from the agent. If those values
are normal then we
>>>>>>>> can narrow down to the CEP.
>>>>>>>>
>>>>>>>> Thanks !
>>>>>>>>
>>>>>>>> On Sat, Oct 25, 2014 at 10:43 PM, Chamila De Alwis <
>>>>>>>> chamilad@wso2.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Raj,
>>>>>>>>>
>>>>>>>>> Can you inspect the CEP trace logs for the same time
period? It
>>>>>>>>> will have the events published from the agent.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Chamila de Alwis
>>>>>>>>> Software Engineer | WSO2 | +94772207163
>>>>>>>>> Blog: code.chamiladealwis.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sat, Oct 25, 2014 at 10:39 PM, Rajkumar Rajaratnam
<
>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I am doing a test round with M3 with python agent.
And faced this
>>>>>>>>>> very strange scenario, $Subject.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,693] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,693] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Grad of
>>>>>>>>>> load avg event: [cluster] php.php.domain [value]
-2000.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Grad of
>>>>>>>>>> load avg event: [cluster] php.php.domain [value]
0.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Avg
>>>>>>>>>> load avg event: [cluster] php.php.domain [value]
1.9090909
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Avg
>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain
[value] 28.718182
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Second
>>>>>>>>>> Derivation of Memory Consumption event: [cluster]
php.php.domain [value]
>>>>>>>>>> 200000.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Grad of
>>>>>>>>>> load avg event: [cluster] php.php.domain [value]
0.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Second
>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain
[value] 0.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Second
>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain
[value] 0.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Avg
>>>>>>>>>> load avg event: [cluster] php.php.domain [value]
3.3636363
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Grad of
>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain
[value] 0.11670045
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Memory
>>>>>>>>>> consumption stats are reset, ready to do scale check
[kub cluster] KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Avg
>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain
[value] 28.7
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Memory
>>>>>>>>>> consumption stats are reset, ready to do scale check
[kub cluster] KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Second
>>>>>>>>>> Derivation of Memory Consumption event: [cluster]
php.php.domain [value] 0.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Memory
>>>>>>>>>> consumption stats are reset, ready to do scale check
[kub cluster] KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Avg
>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain
[value] 28.7
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Memory
>>>>>>>>>> consumption stats are reset, ready to do scale check
[kub cluster] KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Avg
>>>>>>>>>> load avg event: [cluster] php.php.domain [value]
5.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Grad of
>>>>>>>>>> load avg event: [cluster] php.php.domain [value]
2000.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Avg
>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain
[value] 28.7
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Memory
>>>>>>>>>> consumption stats are reset, ready to do scale check
[kub cluster] KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Grad of
>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain
[value] 0.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Memory
>>>>>>>>>> consumption stats are reset, ready to do scale check
[kub cluster] KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,699] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Second
>>>>>>>>>> Derivation of Memory Consumption event: [cluster]
php.php.domain [value] 0.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,699] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Memory
>>>>>>>>>> consumption stats are reset, ready to do scale check
[kub cluster] KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Second
>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain
[value] 2000000.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Grad of
>>>>>>>>>> load avg event: [cluster] php.php.domain [value]
13500.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Grad of
>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain
[value] 50.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Memory
>>>>>>>>>> consumption stats are reset, ready to do scale check
[kub cluster] KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Avg
>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain
[value] 28.738462
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Memory
>>>>>>>>>> consumption stats are reset, ready to do scale check
[kub cluster] KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Avg
>>>>>>>>>> load avg event: [cluster] php.php.domain [value]
2.3333333
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Grad of
>>>>>>>>>> load avg event: [cluster] php.php.domain [value]
0.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Grad of
>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain
[value] 0.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Memory
>>>>>>>>>> consumption stats are reset, ready to do scale check
[kub cluster] KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Avg
>>>>>>>>>> load avg event: [cluster] php.php.domain [value]
31.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Grad of
>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain
[value] 0.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Memory
>>>>>>>>>> consumption stats are reset, ready to do scale check
[kub cluster] KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Avg
>>>>>>>>>> load avg event: [cluster] php.php.domain [value]
39.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Avg
>>>>>>>>>> load avg event: [cluster] php.php.domain [value]
39.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,705] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,706] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Grad of
>>>>>>>>>> load avg event: [cluster] php.php.domain [value]
-11000.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,706] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Load average
>>>>>>>>>> stats are reset, ready to do scale check [kub cluster]
KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,708] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Grad of
>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain
[value] 0.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,708] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Memory
>>>>>>>>>> consumption stats are reset, ready to do scale check
[kub cluster] KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,709] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
-  Avg
>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain
[value] 28.8
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,709] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext}
-  Memory
>>>>>>>>>> consumption stats are reset, ready to do scale check
[kub cluster] KubGrp1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,811] DEBUG
>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher}
-
>>>>>>>>>>  PodActivationWatcher running : Running
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,942] DEBUG
>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher}
-
>>>>>>>>>>  PodActivationWatcher running : Running
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,945] DEBUG
>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher}
-
>>>>>>>>>>  PodActivationWatcher running : Running
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,712] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor}
-
>>>>>>>>>>  KubernetesServiceClusterMonitor is running..
>>>>>>>>>> KubernetesServiceClusterMonitor [ kubernetesHostClusterId=KubGrp1,
>>>>>>>>>> clusterId=php.php.domain, serviceId=php]
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,713] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor}
-
>>>>>>>>>>  Running min check for [kub-cluster] : KubGrp1 [cluster]
: php.php.domain
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,713]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  Running
minimum rule:
>>>>>>>>>> [kub-cluster] KubGrp1 [cluster] php.php.domain
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,714]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [min-check]
 [cluster] :
>>>>>>>>>> php.php.domain [Replicas] nonTerminated : 3
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,714]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [min-check]
 [cluster] :
>>>>>>>>>> php.php.domain [Replicas] minReplicas : 3
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  Running
obsolete containers
>>>>>>>>>> rule [kub-cluster] : KubGrp1 [cluster] : php.php.domain
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [obsolete-check]
[cluster]
>>>>>>>>>> : php.php.domain [Replicas] obsoleteReplicas : 0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.AutoscalerRuleEvaluator}
-  Minimum
>>>>>>>>>> check executed for :
>>>>>>>>>> org.apache.stratos.autoscaler.KubernetesClusterContext@21d906d0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor}
-
>>>>>>>>>>  flag of rifReset : true flag of memoryConsumptionReset
: true flag of
>>>>>>>>>> loadAverageReset : true
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor}
-
>>>>>>>>>>  Running scale check for [kub-cluster] : KubGrp1
[cluster] : php.php.domain
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  Running
scaling rule
>>>>>>>>>> [kub-cluster] : KubGrp1 [cluster] : php.php.domain
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator}
-  Predicting the
>>>>>>>>>> value, [average]: 0.0 , [gradient]: 0.0 , [second
derivative]: 0.0 , [time
>>>>>>>>>> intervals]: 1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator}
-  Predicting the
>>>>>>>>>> value, [average]: 28.8 , [gradient]: 0.0 , [second
derivative]: 0.0 , [time
>>>>>>>>>> intervals]: 1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717] DEBUG
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator}
-  Predicting the
>>>>>>>>>> value, [average]: 39.0 , [gradient]: -11000.0 , [second
derivative]:
>>>>>>>>>> 2000000.0 , [time intervals]: 1
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain [Replicas] minReplicas : 3
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain [Replicas] maxReplicas : 10
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain [Replicas] nonTerminated : 3
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain [Replicas] activeReplicas : 3
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain [RequestInFlight] predicted value
: 0.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain [RequestInFlight] upper limit : 80.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain [RequestInFlight] lower limit : 5.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain [MemoryConsumption] predicted value
: 28.799999237060547
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain [MemoryConsumption] upper limit :
80.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain [MemoryConsumption] lower limit :
15.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain [LoadAverage] predicted value : 989039.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain [LoadAverage] upper limit : 180.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain [LoadAverage] lower limit : 20.0
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain scale-up action : true
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain scale-down action : false
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain [LoadAverage] predicted replicas :
16484
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
 [cluster] :
>>>>>>>>>> php.php.domain predicted replicas > max replicas
:
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]
Decided to
>>>>>>>>>> scale-up : [cluster] : php.php.domain
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling-up]
 [cluster] :
>>>>>>>>>> php.php.domain valid number of replicas to expand
: 10
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient}
>>>>>>>>>> -  Updating kubernetes replication controller via
cloud controller:
>>>>>>>>>> [cluster] php.php.domain [replicas] 10
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,734] DEBUG
>>>>>>>>>> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl}
-
>>>>>>>>>>  CloudControllerServiceImpl:updateContainers for
cluster : php.php.domain
>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,758] DEBUG
>>>>>>>>>> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl}
-
>>>>>>>>>>  Cloud Controller is delegating request to update
a replication controller
>>>>>>>>>> php.php.domain to Kubernetes layer.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I couldn't investigate the agent side, since containers
are
>>>>>>>>>> destroyed.
>>>>>>>>>>
>>>>>>>>>> Any idea?
>>>>>>>>>>
>>>>>>>>>> Thanks.
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Regards,
>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>> Software Engineer
>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>> lean . enterprise . middleware
>>>>>>>>
>>>>>>>> email : manulac@wso2.com / manula@apache.org
>>>>>>>> phone : +94 772492511
>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Rajkumar Rajaratnam
>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Rajkumar Rajaratnam
>>>>>> Software Engineer | WSO2, Inc.
>>>>>> Mobile +94777568639 | +94783498120
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Manula Chathurika Thantriwatte
>>>>> Software Engineer
>>>>> WSO2 Inc. : http://wso2.com
>>>>> lean . enterprise . middleware
>>>>>
>>>>> email : manulac@wso2.com / manula@apache.org
>>>>> phone : +94 772492511
>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Manula Chathurika Thantriwatte
>>>> Software Engineer
>>>> WSO2 Inc. : http://wso2.com
>>>> lean . enterprise . middleware
>>>>
>>>> email : manulac@wso2.com / manula@apache.org
>>>> phone : +94 772492511
>>>> blog : http://manulachathurika.blogspot.com/
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Rajkumar Rajaratnam
>>> Software Engineer | WSO2, Inc.
>>> Mobile +94777568639 | +94783498120
>>>
>>
>>
>>
>> --
>> Regards,
>> Manula Chathurika Thantriwatte
>> Software Engineer
>> WSO2 Inc. : http://wso2.com
>> lean . enterprise . middleware
>>
>> email : manulac@wso2.com / manula@apache.org
>> phone : +94 772492511
>> blog : http://manulachathurika.blogspot.com/
>>
>>
>>
>>
>
>
> --
> Rajkumar Rajaratnam
> Software Engineer | WSO2, Inc.
> Mobile +94777568639 | +94783498120
>



-- 
Regards,
Manula Chathurika Thantriwatte
Software Engineer
WSO2 Inc. : http://wso2.com
lean . enterprise . middleware

email : manulac@wso2.com / manula@apache.org
phone : +94 772492511
blog : http://manulachathurika.blogspot.com/

Mime
View raw message