stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manula Chathurika Thantriwatte <manu...@wso2.com>
Subject Re: CEP removed a terminated member from memberTimeStampMap, but complaining it as a faulty member
Date Mon, 13 Oct 2014 09:21:51 GMT
Hi Raj,

It seems like [1] get executed again. memberTimeStampMap is a concurrent
hash map. Seems that CEPTopologyEventReceiver unable to remove the element
in the memberTimeStampMap.

[1]
https://github.com/apache/stratos/blob/master/extensions/cep/stratos-cep-extension/src/main/java/org/apache/stratos/cep/extension/FaultHandlingWindowProcessor.java#L220

Thanks !

On Mon, Oct 13, 2014 at 11:48 AM, Rajkumar Rajaratnam <rajkumarr@wso2.com>
wrote:

> Hi,
>
> CEP extension FaultHandlingWindowProcessor removing a terminated member
> from the map. But after a while, it is complaining it as a faulty member.
> Here is the log. MemberId highlighted.
>
> TID: [0] [STRATOS] [2014-10-13 11:15:22,409] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Predicting the
> value, [average]: 0.0 , [gradient]: 0.0 , [second derivative]: 0.0 , [time
> intervals]: 1
> TID: [0] [STRATOS] [2014-10-13 11:15:22,410] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Predicting the
> value, [average]: 41.63388 , [gradient]: -0.013520682 , [second
> derivative]: 0.0013238431 , [time intervals]: 1
> TID: [0] [STRATOS] [2014-10-13 11:15:22,410] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Predicting the
> value, [average]: 99.9 , [gradient]: -2.0885546 , [second derivative]:
> 0.03460533 , [time intervals]: 1
> TID: [0] [STRATOS] [2014-10-13 11:15:22,411]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [Replicas] minReplicas : 2
> TID: [0] [STRATOS] [2014-10-13 11:15:22,411]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [Replicas] maxReplicas : 20
> TID: [0] [STRATOS] [2014-10-13 11:15:22,411]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [Replicas] nonTerminated : 5
> TID: [0] [STRATOS] [2014-10-13 11:15:22,411]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [Replicas] activeReplicas : 5
> TID: [0] [STRATOS] [2014-10-13 11:15:22,412] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [RequestInFlight] resetted ? : false
> TID: [0] [STRATOS] [2014-10-13 11:15:22,412] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [RequestInFlight] predicted value : 0.0
> TID: [0] [STRATOS] [2014-10-13 11:15:22,412] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [RequestInFlight] upper limit : 80.0
> TID: [0] [STRATOS] [2014-10-13 11:15:22,412] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [RequestInFlight] lower limit : 5.0
> TID: [0] [STRATOS] [2014-10-13 11:15:22,412] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [MemoryConsumption] resetted ? : true
> TID: [0] [STRATOS] [2014-10-13 11:15:22,413]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [MemoryConsumption] predicted value : 41.62102324969601
> TID: [0] [STRATOS] [2014-10-13 11:15:22,413]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [MemoryConsumption] upper limit : 80.0
> TID: [0] [STRATOS] [2014-10-13 11:15:22,413]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [MemoryConsumption] lower limit : 15.0
> TID: [0] [STRATOS] [2014-10-13 11:15:22,415] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [LoadAverage] resetted ? : true
> TID: [0] [STRATOS] [2014-10-13 11:15:22,415]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [LoadAverage] predicted value : 97.82874980941415
> TID: [0] [STRATOS] [2014-10-13 11:15:22,415]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [LoadAverage] upper limit : 180.0
> TID: [0] [STRATOS] [2014-10-13 11:15:22,415]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [LoadAverage] lower limit : 20.0
> TID: [0] [STRATOS] [2014-10-13 11:15:22,416]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain scale-up action : false
> TID: [0] [STRATOS] [2014-10-13 11:15:22,416]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain scale-down action : true
> TID: [0] [STRATOS] [2014-10-13 11:15:22,416]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling] Decided to
> scale-down [cluster] : myphp.php.domain
> TID: [0] [STRATOS] [2014-10-13 11:15:22,417]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling-down]  [cluster] :
> myphp.php.domain shrink the cluster to minReplicas : 2
> TID: [0] [STRATOS] [2014-10-13 11:15:22,417]  INFO
> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient}
> -  Updating kubernetes replication controller via cloud controller:
> [cluster] myphp.php.domain [replicas] 2
> TID: [0] [STRATOS] [2014-10-13 11:15:22,425] DEBUG
> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} -
>  CloudControllerServiceImpl:updateContainers for cluster : myphp.php.domain
> TID: [0] [STRATOS] [2014-10-13 11:15:22,448] DEBUG
> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} -
>  Cloud Controller is delegating request to update a replication controller
> myphp.php.domain to Kubernetes layer.
> TID: [0] [STRATOS] [2014-10-13 11:15:22,506] DEBUG
> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} -
>  Cloud Controller successfully updated the controller myphp.php.domain via
> Kubernetes layer.
> TID: [0] [STRATOS] [2014-10-13 11:15:22,531] DEBUG
> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} -
>  Pods Count: 5 for cluster: myphp.php.domain
> TID: [0] [STRATOS] [2014-10-13 11:15:22,548] DEBUG
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  PodActivationWatcher running : Running
> TID: [0] [STRATOS] [2014-10-13 11:15:22,561] DEBUG
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  PodActivationWatcher running : Running
> TID: [0] [STRATOS] [2014-10-13 11:15:22,563] DEBUG
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  PodActivationWatcher running : Running
> TID: [0] [STRATOS] [2014-10-13 11:15:22,655] DEBUG
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  PodActivationWatcher running : Running
> TID: [0] [STRATOS] [2014-10-13 11:15:22,657] DEBUG
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  PodActivationWatcher running : Running
> TID: [0] [STRATOS] [2014-10-13 11:15:27,538] DEBUG
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  PodActivationWatcher running : Running
> TID: [0] [STRATOS] [2014-10-13 11:15:27,539] ERROR
> {org.apache.stratos.kubernetes.client.KubernetesApiClient} -  Pod
> [b446c0dd-529b-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:27,539] ERROR
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  Container Activation Watcher Failed.. Cause: Pod
> [b446c0dd-529b-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:27,539] DEBUG
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  org.apache.stratos.kubernetes.client.exceptions.KubernetesClientException:
> Pod [b446c0dd-529b-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:27,539] ERROR
> {org.apache.stratos.kubernetes.client.KubernetesApiClient} -  Pod
> [b4468fb6-529b-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:27,540] ERROR
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  Container Activation Watcher Failed.. Cause: Pod
> [b4468fb6-529b-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:27,540] DEBUG
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  org.apache.stratos.kubernetes.client.exceptions.KubernetesClientException:
> Pod [b4468fb6-529b-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:27,628] ERROR
> {org.apache.stratos.kubernetes.client.KubernetesApiClient} -  Pod
> [3ccf4e4e-529a-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:27,628] ERROR
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  Container Activation Watcher Failed.. Cause: Pod
> [3ccf4e4e-529a-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:27,628] DEBUG
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  org.apache.stratos.kubernetes.client.exceptions.KubernetesClientException:
> Pod [3ccf4e4e-529a-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:27,630] DEBUG
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  PodActivationWatcher running : Running
> TID: [0] [STRATOS] [2014-10-13 11:15:32,542] ERROR
> {org.apache.stratos.kubernetes.client.KubernetesApiClient} -  Pod
> [b4468fb6-529b-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,542] ERROR
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  Container Activation Watcher Failed.. Cause: Pod
> [b4468fb6-529b-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,542] DEBUG
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  org.apache.stratos.kubernetes.client.exceptions.KubernetesClientException:
> Pod [b4468fb6-529b-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,549] DEBUG
> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} -
>  Pods Count: 2 for cluster: myphp.php.domain
> TID: [0] [STRATOS] [2014-10-13 11:15:32,549] DEBUG
> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} -
>  Pods created : 2 for cluster : myphp.php.domain
> TID: [0] [STRATOS] [2014-10-13 11:15:32,551] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologyManager} -  Write
> lock acquired
> TID: [0] [STRATOS] [2014-10-13 11:15:32,551] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologyManager} -  Updating
> topology
> TID: [0] [STRATOS] [2014-10-13 11:15:32,553] DEBUG
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  PodActivationWatcher running : Running
> TID: [0] [STRATOS] [2014-10-13 11:15:32,554] ERROR
> {org.apache.stratos.kubernetes.client.KubernetesApiClient} -  Pod
> [b446c0dd-529b-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,554] ERROR
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  Container Activation Watcher Failed.. Cause: Pod
> [b446c0dd-529b-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,555] DEBUG
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  org.apache.stratos.kubernetes.client.exceptions.KubernetesClientException:
> Pod [b446c0dd-529b-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,564] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologyManager} -  Topology
> updated:
> {"serviceMap":{"php":{"serviceName":"php","serviceType":"SingleTenant","clusterIdClusterMap":{"myphp.php.domain":{"serviceName":"php","clusterId":"myphp.php.domain","autoscalePolicyName":"economy","hostNames":["
> myphp.apachestratos.org
> "],"isLbCluster":false,"isKubernetesCluster":true,"memberMap":{"3ccf4e4e-529a-11e4-acbb-08002794b041":{"serviceName":"php","clusterId":"myphp.php.domain","memberId":"3ccf4e4e-529a-11e4-acbb-08002794b041","portMap":{"8280":{"protocol":"http","value":80,"proxy":8280}},"memberPublicIp":"172.17.8.101","status":"Activated","memberIp":"172.17.8.101","properties":{"ALLOCATED_SERVICE_HOST_PORT":"4500"}},"b446c0dd-529b-11e4-acbb-08002794b041":{"serviceName":"php","clusterId":"myphp.php.domain","memberId":"b446c0dd-529b-11e4-acbb-08002794b041","portMap":{"8280":{"protocol":"http","value":80,"proxy":8280}},"memberPublicIp":"172.17.8.101","status":"Activated","memberIp":"172.17.8.101","properties":{}},"b44650cb-529b-11e4-acbb-08002794b041":{"serviceName":"php","clusterId":"myphp.php.domain","memberId":"b44650cb-529b-11e4-acbb-08002794b041","portMap":{"8280":{"protocol":"http","value":80,"proxy":8280}},"memberPublicIp":"172.17.8.101","status":"Activated","memberIp":"172.17.8.101","properties":{}},"3ccfb8bd-529a-11e4-acbb-08002794b041":{"serviceName":"php","clusterId":"myphp.php.domain","memberId":"3ccfb8bd-529a-11e4-acbb-08002794b041","portMap":{"8280":{"protocol":"http","value":80,"proxy":8280}},"memberPublicIp":"172.17.8.101","status":"Activated","memberIp":"172.17.8.101","properties":{"ALLOCATED_SERVICE_HOST_PORT":"4500"}}},"status":"Created","properties":{"payload_parameter.APP_PATH":"/var/www","KUBERNETES_REPLICAS_MAX":"20","KUBERNETES_CLUSTER_ID":"KubGrp1","payload_parameter.LOG_LEVEL":"DEBUG","KUBERNETES_REPLICAS_MIN":"2","payload_parameter.MB_IP":"172.17.42.1","payload_parameter.CEP_IP":"172.17.42.1","payload_parameter.MB_PORT":"1883","payload_parameter.CEP_PORT":"7611"}}},"portMap":{"8280":{"protocol":"http","value":80,"proxy":8280}},"properties":{}}},"initialized":false}
> TID: [0] [STRATOS] [2014-10-13 11:15:32,564] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologyManager} -  Write
> lock released
> TID: [0] [STRATOS] [2014-10-13 11:15:32,564]  INFO
> {org.apache.stratos.cloud.controller.topology.TopologyEventPublisher} -  Publishing
> member terminated event: [service] php [cluster] myphp.php.domain
> [network-partition] null [partition] null [member]
> b4468fb6-529b-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:15:32,575]  INFO
> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
> -  [MemberTerminatedEventListener] Received: class
> org.apache.stratos.messaging.event.topology.MemberTerminatedEvent
> TID: [0] [STRATOS] [2014-10-13 11:15:32,580] DEBUG
> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
> -  Removed the terminated member with id
> b4468fb6-529b-11e4-acbb-08002794b041 from the cluster
> TID: [0] [STRATOS] [2014-10-13 11:15:32,580] DEBUG
> {org.apache.stratos.manager.topology.model.TopologyClusterInformationModel}
> -   Adding cluster [myphp.php.domain]
> TID: [0] [STRATOS] [2014-10-13 11:15:32,580]  INFO
> {org.apache.stratos.messaging.message.processor.topology.MemberTerminatedMessageProcessor}
> -  Member terminated: [service] php [cluster] myphp.php.domain [member]
> b4468fb6-529b-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:15:32,580] DEBUG
> {org.apache.stratos.cloud.controller.runtime.FasterLookUpDataHolder} -
>  MemberContext [id]: b4468fb6-529b-11e4-acbb-08002794b041 removed from
> information model.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,581]  INFO
> {org.apache.stratos.cloud.controller.runtime.FasterLookUpDataHolder} -
>  Scheduled Pod Activation Watcher task canceled.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,581]  WARN
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Member
> is in the wrong list and it is removed from active members list
> TID: [0] [STRATOS] [2014-10-13 11:15:32,582]  INFO
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Member
> stat context has been removed successfully: [member]
> b4468fb6-529b-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:15:32,582]  WARN
> {org.apache.stratos.messaging.message.processor.topology.MemberTerminatedMessageProcessor}
> -  Member already terminated: [service] php [cluster] myphp.php.domain
> [member] b4468fb6-529b-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:15:32,583]  INFO
> {org.apache.stratos.cep.extension.CEPTopologyEventReceiver} -  Member
> [member id] b4468fb6-529b-11e4-acbb-08002794b041 was removed from the time
> stamp map.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,583]  WARN
> {org.apache.stratos.messaging.message.processor.topology.MemberTerminatedMessageProcessor}
> -  Member already terminated: [service] php [cluster] myphp.php.domain
> [member] b4468fb6-529b-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:15:32,593] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologyManager} -  Write
> lock acquired
> TID: [0] [STRATOS] [2014-10-13 11:15:32,593] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologyManager} -  Updating
> topology
> TID: [0] [STRATOS] [2014-10-13 11:15:32,603] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologyManager} -  Topology
> updated:
> {"serviceMap":{"php":{"serviceName":"php","serviceType":"SingleTenant","clusterIdClusterMap":{"myphp.php.domain":{"serviceName":"php","clusterId":"myphp.php.domain","autoscalePolicyName":"economy","hostNames":["
> myphp.apachestratos.org
> "],"isLbCluster":false,"isKubernetesCluster":true,"memberMap":{"3ccf4e4e-529a-11e4-acbb-08002794b041":{"serviceName":"php","clusterId":"myphp.php.domain","memberId":"3ccf4e4e-529a-11e4-acbb-08002794b041","portMap":{"8280":{"protocol":"http","value":80,"proxy":8280}},"memberPublicIp":"172.17.8.101","status":"Activated","memberIp":"172.17.8.101","properties":{"ALLOCATED_SERVICE_HOST_PORT":"4500"}},"b44650cb-529b-11e4-acbb-08002794b041":{"serviceName":"php","clusterId":"myphp.php.domain","memberId":"b44650cb-529b-11e4-acbb-08002794b041","portMap":{"8280":{"protocol":"http","value":80,"proxy":8280}},"memberPublicIp":"172.17.8.101","status":"Activated","memberIp":"172.17.8.101","properties":{}},"3ccfb8bd-529a-11e4-acbb-08002794b041":{"serviceName":"php","clusterId":"myphp.php.domain","memberId":"3ccfb8bd-529a-11e4-acbb-08002794b041","portMap":{"8280":{"protocol":"http","value":80,"proxy":8280}},"memberPublicIp":"172.17.8.101","status":"Activated","memberIp":"172.17.8.101","properties":{"ALLOCATED_SERVICE_HOST_PORT":"4500"}}},"status":"Created","properties":{"payload_parameter.APP_PATH":"/var/www","KUBERNETES_REPLICAS_MAX":"20","KUBERNETES_CLUSTER_ID":"KubGrp1","payload_parameter.LOG_LEVEL":"DEBUG","KUBERNETES_REPLICAS_MIN":"2","payload_parameter.MB_IP":"172.17.42.1","payload_parameter.CEP_IP":"172.17.42.1","payload_parameter.MB_PORT":"1883","payload_parameter.CEP_PORT":"7611"}}},"portMap":{"8280":{"protocol":"http","value":80,"proxy":8280}},"properties":{}}},"initialized":false}
> TID: [0] [STRATOS] [2014-10-13 11:15:32,603] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologyManager} -  Write
> lock released
> TID: [0] [STRATOS] [2014-10-13 11:15:32,603]  INFO
> {org.apache.stratos.cloud.controller.topology.TopologyEventPublisher} -
>  Publishing member terminated event: [service] php [cluster]
> myphp.php.domain [network-partition] null [partition] null [member]
> b446c0dd-529b-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:15:32,607]  INFO
> {org.apache.stratos.cep.extension.CEPTopologyEventReceiver} -  Member
> [member id] b446c0dd-529b-11e4-acbb-08002794b041 was removed from the time
> stamp map.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,608]  INFO
> {org.apache.stratos.messaging.message.processor.topology.MemberTerminatedMessageProcessor}
> -  Member terminated: [service] php [cluster] myphp.php.domain [member]
> b446c0dd-529b-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:15:32,609]  INFO
> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
> -  [MemberTerminatedEventListener] Received: class
> org.apache.stratos.messaging.event.topology.MemberTerminatedEvent
> TID: [0] [STRATOS] [2014-10-13 11:15:32,609] DEBUG
> {org.apache.stratos.manager.topology.model.TopologyClusterInformationModel}
> -   Adding cluster [myphp.php.domain]
> TID: [0] [STRATOS] [2014-10-13 11:15:32,609]  WARN
> {org.apache.stratos.messaging.message.processor.topology.MemberTerminatedMessageProcessor}
> -  Member already terminated: [service] php [cluster] myphp.php.domain
> [member] b446c0dd-529b-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:15:32,609]  WARN
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Member
> is in the wrong list and it is removed from active members list
> TID: [0] [STRATOS] [2014-10-13 11:15:32,609]  INFO
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Member
> stat context has been removed successfully: [member]
> b446c0dd-529b-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:15:32,610] DEBUG
> {org.apache.stratos.cloud.controller.runtime.FasterLookUpDataHolder} -
>  MemberContext [id]: b446c0dd-529b-11e4-acbb-08002794b041 removed from
> information model.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,610]  WARN
> {org.apache.stratos.messaging.message.processor.topology.MemberTerminatedMessageProcessor}
> -  Member already terminated: [service] php [cluster] myphp.php.domain
> [member] b446c0dd-529b-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:15:32,610]  INFO
> {org.apache.stratos.cloud.controller.runtime.FasterLookUpDataHolder} -
>  Scheduled Pod Activation Watcher task canceled.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,623] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologyManager} -  Write
> lock acquired
> TID: [0] [STRATOS] [2014-10-13 11:15:32,624] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologyManager} -  Updating
> topology
> TID: [0] [STRATOS] [2014-10-13 11:15:32,627] ERROR
> {org.apache.stratos.kubernetes.client.KubernetesApiClient} -  Pod
> [3ccf4e4e-529a-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,627] ERROR
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  Container Activation Watcher Failed.. Cause: Pod
> [3ccf4e4e-529a-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,628] DEBUG
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  org.apache.stratos.kubernetes.client.exceptions.KubernetesClientException:
> Pod [3ccf4e4e-529a-11e4-acbb-08002794b041] doesn't exist.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,633] DEBUG
> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} -
>  PodActivationWatcher running : Running
> TID: [0] [STRATOS] [2014-10-13 11:15:32,642] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologyManager} -  Topology
> updated:
> {"serviceMap":{"php":{"serviceName":"php","serviceType":"SingleTenant","clusterIdClusterMap":{"myphp.php.domain":{"serviceName":"php","clusterId":"myphp.php.domain","autoscalePolicyName":"economy","hostNames":["
> myphp.apachestratos.org
> "],"isLbCluster":false,"isKubernetesCluster":true,"memberMap":{"b44650cb-529b-11e4-acbb-08002794b041":{"serviceName":"php","clusterId":"myphp.php.domain","memberId":"b44650cb-529b-11e4-acbb-08002794b041","portMap":{"8280":{"protocol":"http","value":80,"proxy":8280}},"memberPublicIp":"172.17.8.101","status":"Activated","memberIp":"172.17.8.101","properties":{}},"3ccfb8bd-529a-11e4-acbb-08002794b041":{"serviceName":"php","clusterId":"myphp.php.domain","memberId":"3ccfb8bd-529a-11e4-acbb-08002794b041","portMap":{"8280":{"protocol":"http","value":80,"proxy":8280}},"memberPublicIp":"172.17.8.101","status":"Activated","memberIp":"172.17.8.101","properties":{"ALLOCATED_SERVICE_HOST_PORT":"4500"}}},"status":"Created","properties":{"payload_parameter.APP_PATH":"/var/www","KUBERNETES_REPLICAS_MAX":"20","KUBERNETES_CLUSTER_ID":"KubGrp1","payload_parameter.LOG_LEVEL":"DEBUG","KUBERNETES_REPLICAS_MIN":"2","payload_parameter.MB_IP":"172.17.42.1","payload_parameter.CEP_IP":"172.17.42.1","payload_parameter.MB_PORT":"1883","payload_parameter.CEP_PORT":"7611"}}},"portMap":{"8280":{"protocol":"http","value":80,"proxy":8280}},"properties":{}}},"initialized":false}
> TID: [0] [STRATOS] [2014-10-13 11:15:32,642] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologyManager} -  Write
> lock released
> TID: [0] [STRATOS] [2014-10-13 11:15:32,642]  INFO
> {org.apache.stratos.cloud.controller.topology.TopologyEventPublisher} -
>  Publishing member terminated event: [service] php [cluster]
> myphp.php.domain [network-partition] null [partition] null [member]
> 3ccf4e4e-529a-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:15:32,650]  INFO
> {org.apache.stratos.cep.extension.CEPTopologyEventReceiver} -  Member
> [member id] 3ccf4e4e-529a-11e4-acbb-08002794b041 was removed from the time
> stamp map.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,652]  INFO
> {org.apache.stratos.messaging.message.processor.topology.MemberTerminatedMessageProcessor}
> -  Member terminated: [service] php [cluster] myphp.php.domain [member]
> 3ccf4e4e-529a-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:15:32,653]  INFO
> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
> -  [MemberTerminatedEventListener] Received: class
> org.apache.stratos.messaging.event.topology.MemberTerminatedEvent
> TID: [0] [STRATOS] [2014-10-13 11:15:32,653] DEBUG
> {org.apache.stratos.manager.topology.model.TopologyClusterInformationModel}
> -   Adding cluster [myphp.php.domain]
> TID: [0] [STRATOS] [2014-10-13 11:15:32,653]  WARN
> {org.apache.stratos.messaging.message.processor.topology.MemberTerminatedMessageProcessor}
> -  Member already terminated: [service] php [cluster] myphp.php.domain
> [member] 3ccf4e4e-529a-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:15:32,654]  WARN
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Member
> is in the wrong list and it is removed from active members list
> TID: [0] [STRATOS] [2014-10-13 11:15:32,654] DEBUG
> {org.apache.stratos.cloud.controller.runtime.FasterLookUpDataHolder} -
>  MemberContext [id]: 3ccf4e4e-529a-11e4-acbb-08002794b041 removed from
> information model.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,654]  INFO
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Member
> stat context has been removed successfully: [member]
> 3ccf4e4e-529a-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:15:32,655]  WARN
> {org.apache.stratos.messaging.message.processor.topology.MemberTerminatedMessageProcessor}
> -  Member already terminated: [service] php [cluster] myphp.php.domain
> [member] 3ccf4e4e-529a-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:15:32,654]  INFO
> {org.apache.stratos.cloud.controller.runtime.FasterLookUpDataHolder} -
>  Scheduled Pod Activation Watcher task canceled.
> TID: [0] [STRATOS] [2014-10-13 11:15:32,696]  INFO
> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} -
>  Kubernetes entities are successfully starting up. [MemberContext
> [memberId=b4468fb6-529b-11e4-acbb-08002794b041, nodeId=null,
> instanceId=null, clusterId=myphp.php.domain, partition=null,
> cartridgeType=php, privateIpAddress=172.17.8.100,
> publicIpAddress=172.17.8.100, allocatedIpAddress=null,
> initTime=1413178952532, lbClusterId=null, networkPartitionId=null,
> properties=Properties [properties=[Property
> [name=ALLOCATED_SERVICE_HOST_PORT, value=null]]]], MemberContext
> [memberId=b446c0dd-529b-11e4-acbb-08002794b041, nodeId=null,
> instanceId=null, clusterId=myphp.php.domain, partition=null,
> cartridgeType=php, privateIpAddress=172.17.8.101,
> publicIpAddress=172.17.8.101, allocatedIpAddress=null,
> initTime=1413178952532, lbClusterId=null, networkPartitionId=null,
> properties=Properties [properties=[Property
> [name=ALLOCATED_SERVICE_HOST_PORT, value=null]]]], MemberContext
> [memberId=3ccf4e4e-529a-11e4-acbb-08002794b041, nodeId=null,
> instanceId=null, clusterId=myphp.php.domain, partition=null,
> cartridgeType=php, privateIpAddress=172.17.8.101,
> publicIpAddress=172.17.8.101, allocatedIpAddress=null,
> initTime=1413178322623, lbClusterId=null, networkPartitionId=null,
> properties=Properties [properties=[Property
> [name=ALLOCATED_SERVICE_HOST_PORT, value=4500]]]]]
> TID: [0] [STRATOS] [2014-10-13 11:15:32,709] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Scaling down,
> terminated the member with id b4468fb6-529b-11e4-acbb-08002794b041 in
> cluster myphp.php.domain
> TID: [0] [STRATOS] [2014-10-13 11:15:32,709] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Scaling down,
> terminated the member with id b446c0dd-529b-11e4-acbb-08002794b041 in
> cluster myphp.php.domain
> TID: [0] [STRATOS] [2014-10-13 11:15:32,709] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Scaling down,
> terminated the member with id 3ccf4e4e-529a-11e4-acbb-08002794b041 in
> cluster myphp.php.domain
> TID: [0] [STRATOS] [2014-10-13 11:15:32,709] DEBUG
> {org.apache.stratos.autoscaler.rule.AutoscalerRuleEvaluator} -  Scale check
> executed for :
> org.apache.stratos.autoscaler.KubernetesClusterContext@29d32c84
> TID: [0] [STRATOS] [2014-10-13 11:16:01,002] DEBUG
> {org.apache.stratos.manager.publisher.TenantSynzhronizerTask} -  Publishing
> complete tenant event
> TID: [0] [STRATOS] [2014-10-13 11:16:01,006] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologySynchronizerTask} -
>  Executing topology synchronization task
> TID: [0] [STRATOS] [2014-10-13 11:16:01,006] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologyEventPublisher} -
>  Publishing complete topology event
> TID: [0] [STRATOS] [2014-10-13 11:16:22,230] DEBUG
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
> load avg event: [cluster] myphp.php.domain [value] -0.70397747
> TID: [0] [STRATOS] [2014-10-13 11:16:22,232] DEBUG
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
> Memory Consumption event: [cluster] myphp.php.domain [value] -0.46528494
> TID: [0] [STRATOS] [2014-10-13 11:16:22,234] DEBUG
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
> Memory Consumption event: [cluster] myphp.php.domain [value] 27.538944
> TID: [0] [STRATOS] [2014-10-13 11:16:22,236] DEBUG
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Second
> Derivation of load avg event: [cluster] myphp.php.domain [value]
> -0.0012131054
> TID: [0] [STRATOS] [2014-10-13 11:16:22,236] DEBUG
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Second
> Derivation of Memory Consumption event: [cluster] myphp.php.domain [value]
> 0.021650476
> TID: [0] [STRATOS] [2014-10-13 11:16:22,236] DEBUG
> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
> TID: [0] [STRATOS] [2014-10-13 11:16:22,237] DEBUG
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
> load avg event: [cluster] myphp.php.domain [value] 53.7
> TID: [0] [STRATOS] [2014-10-13 11:16:22,237] DEBUG
> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
> stats are reset, ready to do scale check [kub cluster] KubGrp1
> TID: [0] [STRATOS] [2014-10-13 11:16:22,425] DEBUG
> {org.apache.stratos.autoscaler.message.receiver.health.AutoscalerHealthStatEventReceiver}
> -  Member not found in the Topology: [member]
> 3ccf4e4e-529a-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:16:22,426] DEBUG
> {org.apache.stratos.autoscaler.message.receiver.health.AutoscalerHealthStatEventReceiver}
> -  Member not found in the Topology: [member]
> 3ccf4e4e-529a-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:16:22,426] DEBUG
> {org.apache.stratos.autoscaler.message.receiver.health.AutoscalerHealthStatEventReceiver}
> -  Member not found in the Topology: [member]
> 3ccf4e4e-529a-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:16:22,427] DEBUG
> {org.apache.stratos.autoscaler.message.receiver.health.AutoscalerHealthStatEventReceiver}
> -  Member not found in the Topology: [member]
> 3ccf4e4e-529a-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:16:22,427] DEBUG
> {org.apache.stratos.autoscaler.message.receiver.health.AutoscalerHealthStatEventReceiver}
> -  Member not found in the Topology: [member]
> 3ccf4e4e-529a-11e4-acbb-08002794b041
> TID: [0] [STRATOS] [2014-10-13 11:16:52,406] DEBUG
> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} -
>  KubernetesServiceClusterMonitor is running..
> KubernetesServiceClusterMonitor [ kubernetesHostClusterId=KubGrp1,
> clusterId=myphp.php.domain, serviceId=php]
> TID: [0] [STRATOS] [2014-10-13 11:16:52,407] DEBUG
> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} -
>  Running min check for [kub-cluster] : KubGrp1 [cluster] : myphp.php.domain
> TID: [0] [STRATOS] [2014-10-13 11:16:52,408]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  Running minimum rule:
> [kub-cluster] KubGrp1 [cluster] myphp.php.domain
> TID: [0] [STRATOS] [2014-10-13 11:16:52,408]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [min-check]  [cluster] :
> myphp.php.domain [Replicas] nonTerminated : 2
> TID: [0] [STRATOS] [2014-10-13 11:16:52,409]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [min-check]  [cluster] :
> myphp.php.domain [Replicas] minReplicas : 2
> TID: [0] [STRATOS] [2014-10-13 11:16:52,409]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  Running obsolete containers
> rule [kub-cluster] : KubGrp1 [cluster] : myphp.php.domain
> TID: [0] [STRATOS] [2014-10-13 11:16:52,410]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [obsolete-check] [cluster]
> : myphp.php.domain [Replicas] obsoleteReplicas : 0
> TID: [0] [STRATOS] [2014-10-13 11:16:52,411] DEBUG
> {org.apache.stratos.autoscaler.rule.AutoscalerRuleEvaluator} -  Minimum
> check executed for :
> org.apache.stratos.autoscaler.KubernetesClusterContext@29d32c84
> TID: [0] [STRATOS] [2014-10-13 11:16:52,411] DEBUG
> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} -
>  flag of rifReset : false flag of memoryConsumptionReset : true flag of
> loadAverageReset : true
> TID: [0] [STRATOS] [2014-10-13 11:16:52,412] DEBUG
> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} -
>  Running scale check for [kub-cluster] : KubGrp1 [cluster] :
> myphp.php.domain
> TID: [0] [STRATOS] [2014-10-13 11:16:52,412]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  Running scaling rule
> [kub-cluster] : KubGrp1 [cluster] : myphp.php.domain
> TID: [0] [STRATOS] [2014-10-13 11:16:52,414] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Predicting the
> value, [average]: 0.0 , [gradient]: 0.0 , [second derivative]: 0.0 , [time
> intervals]: 1
> TID: [0] [STRATOS] [2014-10-13 11:16:52,415] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Predicting the
> value, [average]: 27.538944 , [gradient]: -0.46528494 , [second
> derivative]: 0.021650476 , [time intervals]: 1
> TID: [0] [STRATOS] [2014-10-13 11:16:52,416] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -  Predicting the
> value, [average]: 53.7 , [gradient]: -0.70397747 , [second derivative]:
> -0.0012131054 , [time intervals]: 1
> TID: [0] [STRATOS] [2014-10-13 11:16:52,417]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [Replicas] minReplicas : 2
> TID: [0] [STRATOS] [2014-10-13 11:16:52,417]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [Replicas] maxReplicas : 20
> TID: [0] [STRATOS] [2014-10-13 11:16:52,417]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [Replicas] nonTerminated : 2
> TID: [0] [STRATOS] [2014-10-13 11:16:52,417]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [Replicas] activeReplicas : 2
> TID: [0] [STRATOS] [2014-10-13 11:16:52,417] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [RequestInFlight] resetted ? : false
> TID: [0] [STRATOS] [2014-10-13 11:16:52,417] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [RequestInFlight] predicted value : 0.0
> TID: [0] [STRATOS] [2014-10-13 11:16:52,418] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [RequestInFlight] upper limit : 80.0
> TID: [0] [STRATOS] [2014-10-13 11:16:52,418] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [RequestInFlight] lower limit : 5.0
> TID: [0] [STRATOS] [2014-10-13 11:16:52,418] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [MemoryConsumption] resetted ? : true
> TID: [0] [STRATOS] [2014-10-13 11:16:52,418]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [MemoryConsumption] predicted value : 27.084485135041177
> TID: [0] [STRATOS] [2014-10-13 11:16:52,418]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [MemoryConsumption] upper limit : 80.0
> TID: [0] [STRATOS] [2014-10-13 11:16:52,419]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [MemoryConsumption] lower limit : 15.0
> TID: [0] [STRATOS] [2014-10-13 11:16:52,419] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [LoadAverage] resetted ? : true
> TID: [0] [STRATOS] [2014-10-13 11:16:52,419]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [LoadAverage] predicted value : 52.9954185327515
> TID: [0] [STRATOS] [2014-10-13 11:16:52,419]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [LoadAverage] upper limit : 180.0
> TID: [0] [STRATOS] [2014-10-13 11:16:52,420]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain [LoadAverage] lower limit : 20.0
> TID: [0] [STRATOS] [2014-10-13 11:16:52,420]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain scale-up action : false
> TID: [0] [STRATOS] [2014-10-13 11:16:52,420]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain scale-down action : true
> TID: [0] [STRATOS] [2014-10-13 11:16:52,421]  INFO
> {org.apache.stratos.autoscaler.rule.RuleLog} -  [scaling]  [cluster] :
> myphp.php.domain Even if all predicted stats < upperLimit, still
> activeReplicas <= minReplicas ; hence not scaling down
> TID: [0] [STRATOS] [2014-10-13 11:16:52,421] DEBUG
> {org.apache.stratos.autoscaler.rule.AutoscalerRuleEvaluator} -  Scale check
> executed for :
> org.apache.stratos.autoscaler.KubernetesClusterContext@29d32c84
> TID: [0] [STRATOS] [2014-10-13 11:17:01,001] DEBUG
> {org.apache.stratos.manager.publisher.TenantSynzhronizerTask} -  Publishing
> complete tenant event
> TID: [0] [STRATOS] [2014-10-13 11:17:01,002] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologySynchronizerTask} -
>  Executing topology synchronization task
> TID: [0] [STRATOS] [2014-10-13 11:17:01,002] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologyEventPublisher} -
>  Publishing complete topology event
> TID: [0] [STRATOS] [2014-10-13 11:17:16,141]  INFO
> {org.apache.stratos.cep.extension.FaultHandlingWindowProcessor} -  Faulty
> member detected [member-id] 3ccf4e4e-529a-11e4-acbb-08002794b041 with [last
> time-stamp] 1413179137678 [time-out] 60000 milliseconds
> TID: [0] [STRATOS] [2014-10-13 11:17:16,141] ERROR
> {org.apache.stratos.cep.extension.FaultHandlingWindowProcessor} -  Failed
> to publish member fault event. Member having [member-id]
> 3ccf4e4e-529a-11e4-acbb-08002794b041 does not exist in topology
> TID: [0] [STRATOS] [2014-10-13 11:17:22,231] DEBUG
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
> load avg event: [cluster] myphp.php.domain [value] 23.75
> TID: [0] [STRATOS] [2014-10-13 11:17:22,232] DEBUG
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
> load avg event: [cluster] myphp.php.domain [value] -0.2682943
> TID: [0] [STRATOS] [2014-10-13 11:17:22,236] DEBUG
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Avg
> Memory Consumption event: [cluster] myphp.php.domain [value] 17.602432
> TID: [0] [STRATOS] [2014-10-13 11:17:22,237] DEBUG
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Second
> Derivation of load avg event: [cluster] myphp.php.domain [value]
> -0.0045045386
> TID: [0] [STRATOS] [2014-10-13 11:17:22,237] DEBUG
> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Load average
> stats are reset, ready to do scale check [kub cluster] KubGrp1
> TID: [0] [STRATOS] [2014-10-13 11:17:22,237] DEBUG
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Grad of
> Memory Consumption event: [cluster] myphp.php.domain [value] 0.0016795773
> TID: [0] [STRATOS] [2014-10-13 11:17:22,238] DEBUG
> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} -  Second
> Derivation of Memory Consumption event: [cluster] myphp.php.domain [value]
> 2.3589536E-4
> TID: [0] [STRATOS] [2014-10-13 11:17:22,238] DEBUG
> {org.apache.stratos.autoscaler.KubernetesClusterContext} -  Memory
> consumption stats are reset, ready to do scale check [kub cluster] KubGrp1
> TID: [0] [STRATOS] [2014-10-13 11:18:01,000] DEBUG
> {org.apache.stratos.manager.publisher.TenantSynzhronizerTask} -  Publishing
> complete tenant event
> TID: [0] [STRATOS] [2014-10-13 11:18:01,001] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologySynchronizerTask} -
>  Executing topology synchronization task
> TID: [0] [STRATOS] [2014-10-13 11:18:01,001] DEBUG
> {org.apache.stratos.cloud.controller.topology.TopologyEventPublisher} -
>  Publishing complete topology event
> TID: [0] [STRATOS] [2014-10-13 11:18:16,142]  INFO
> {org.apache.stratos.cep.extension.FaultHandlingWindowProcessor} -  Faulty
> member detected [member-id] 3ccf4e4e-529a-11e4-acbb-08002794b041 with [last
> time-stamp] 1413179137678 [time-out] 60000 milliseconds
> TID: [0] [STRATOS] [2014-10-13 11:18:16,144] ERROR
> {org.apache.stratos.cep.extension.FaultHandlingWindowProcessor} -  Failed
> to publish member fault event. Member having [member-id]
> 3ccf4e4e-529a-11e4-acbb-08002794b041 does not exist in topology
>
> Thanks.
>
> --
> Rajkumar Rajaratnam
> Software Engineer | WSO2, Inc.
> Mobile +94777568639 | +94783498120
>



-- 
Regards,
Manula Chathurika Thantriwatte
Software Engineer
WSO2 Inc. : http://wso2.com
lean . enterprise . middleware

email : manulac@wso2.com / manula@apache.org
phone : +94 772492511
blog : http://manulachathurika.blogspot.com/

Mime
View raw message