stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajkumar Rajaratnam <rajkum...@wso2.com>
Subject Re: [Discuss] Kubernetes workflow is now working!
Date Thu, 01 Jan 2015 16:44:40 GMT
On Thu, Jan 1, 2015 at 10:11 PM, Imesh Gunaratne <imesh@apache.org> wrote:

> No, the current workflow is as follows:
>
> - AS talks to CC to start an instance
> - CC creates a new member context with a unique member Id, start a new
> thread to execute the actions, returns the call to AS.
> - The above thread runs and try to start the instance, allocate IP
> addresses, assign storage, etc
> - If everything successfully completes, CC will send the Instance Spawened
> event.
> - If one of the actions fails, CC will raise an error, it will not
> terminate the instance since member context has been already returned to AS.
> - AS will terminate the instance on the next obsolete member evaluation
> iteration.
>

Thanks Imesh. So when we are adding the member to the topology?

Thanks.

>
>
>
> On Thu, Jan 1, 2015 at 9:44 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com>
> wrote:
>
>>
>>
>> On Thu, Jan 1, 2015 at 9:42 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com>
>> wrote:
>>
>>> Hi Imesh,
>>>
>>> May be I didn't understand your solution completely, correct me if I am
>>> wrong.
>>>
>>>    - adding the member to the topology as soon as instance is created
>>>    (just spawned)
>>>
>>> This should be "adding the member context to the data holder as soon as
>> instance is created (just spawned). Sorry for the confusion :)
>>
>>>
>>>    - sending instance spawned event if everything else configured
>>>    correctly (IP, Volume etc)
>>>    - terminating the instance and sending instance terminated event if
>>>    IP allocation fails or Volume allocation fails
>>>
>>> Am I correct?
>>>
>>> If correct, I would like to know when we will be adding the member to
>>> the topology? If we adding the member to the topology after sending
>>> instance spawned event, then there is a problem. Even if IP allocation goes
>>> wrong, we will not be able to send the member terminated event, because we
>>> can't send events for the members which are not in the topology.
>>>
>>> That's why, IMO, we should add the member to the topology as soon as we
>>> get the instance start up call and we should send a new event (before
>>> instance spawned event) so that other components will be aware that a
>>> member is added to the topology. This way we can send member terminated
>>> event at the point where IP allocation goes wrong. This will be efficient
>>> and I guess it is not that complicated to implement.
>>>
>>> wdyt?
>>>
>>> Thanks.
>>>
>>> On Thu, Jan 1, 2015 at 9:15 PM, Imesh Gunaratne <imesh@apache.org>
>>> wrote:
>>>
>>>> IMO we shouldn't make things much complicated.
>>>>
>>>> If we can wait until the point where IP addresses are allocated to send
>>>> the Instance Spawned event, may be that's much simpler than the above
>>>> suggestion.
>>>>
>>>> On Thu, Jan 1, 2015 at 8:12 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com
>>>> > wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am quoting what we agreed from some other mail thread.
>>>>>
>>>>> *Correct approach would be to introduce a new topology event to fill
>>>>> the blind area between real instance creation via CC and member spawned
>>>>> event. So, as soon as CC receives a instance start up call, CC would add
>>>>> the member to topology and will send a new event with a new state possibly.*
>>>>>
>>>>> Thanks.
>>>>>
>>>>> On Thu, Jan 1, 2015 at 7:33 PM, Rajkumar Rajaratnam <
>>>>> rajkumarr@wso2.com> wrote:
>>>>>
>>>>>> s/if anything goes like IP allocation/if anything goes *wrong* like
>>>>>> IP allocation
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> On Thu, Jan 1, 2015 at 7:20 PM, Rajkumar Rajaratnam <
>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>
>>>>>>> Hi Imesh,
>>>>>>>
>>>>>>> Yes, we can either do it as you mentioned, or we can send the member
>>>>>>> terminated event so that the other components will be aware of this
>>>>>>> instance termination. I guess the later one would the efficient solution as
>>>>>>> it AS will take 15 min (pending member timeout) to terminate such
>>>>>>> instances. Even in container case, we can do this.
>>>>>>>
>>>>>>> If we know that the instance will never be activated in advance, we
>>>>>>> can terminate the instance in CC and send the member terminated event. We
>>>>>>> don't need to wait 15 min.
>>>>>>>
>>>>>>> I can recall that we agreed to add the member to the topology as
>>>>>>> soon as member is spawned and send the member terminated event if anything
>>>>>>> goes like IP allocation. It is discussed here,
>>>>>>> If floating IP allocation goes wrong, we are not terminating the
>>>>>>> instances at that point?
>>>>>>> Thanks.
>>>>>>>
>>>>>>> On Thu, Jan 1, 2015 at 6:34 PM, Imesh Gunaratne <imesh@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> As I see we have the same logic for VMs which terminate the
>>>>>>>> instance if it could not allocate IP addresses:
>>>>>>>>
>>>>>>>>
>>>>>>>> https://github.com/apache/stratos/blob/master/components/org.apache.stratos.cloud.controller/src/main/java/org/apache/stratos/cloud/controller/iaases/JcloudsIaas.java
>>>>>>>>
>>>>>>>> String msg = "Error occurred while allocating predefined floating
>>>>>>>> ip address: " + preDefinedIp +
>>>>>>>>                                     " / allocated ip:" + ip +
>>>>>>>>                                     " - terminating node:" +
>>>>>>>> memberContext.toString();
>>>>>>>>                             log.error(msg);
>>>>>>>>                             // terminate instance
>>>>>>>>                             destroyNode(nodeMetadata.getId(),
>>>>>>>> memberContext);
>>>>>>>>                             throw new CloudControllerException(msg);
>>>>>>>>
>>>>>>>> We might need to change this logic to log the error message and
>>>>>>>> wait until the Autoscaler terminate the instance.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> On Thu, Jan 1, 2015 at 12:01 PM, Imesh Gunaratne <imesh@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> A good point Raj, yes, it is not consistent to let the cloud
>>>>>>>>> controller handle this logic as it is already done by the Autoscaler. I
>>>>>>>>> will fix this.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>> On Wed, Dec 31, 2014 at 8:51 PM, Rajkumar Rajaratnam <
>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Currently pods' status are monitored by CC. If pods are not
>>>>>>>>>> getting "Running" status within a time period(default to 120s), replication
>>>>>>>>>> controllers and pods are deleted by CC and nothing happens after that.
>>>>>>>>>>
>>>>>>>>>> I faced some problems due to this. My pods and replication
>>>>>>>>>> controller were deleted by CC, because pods didn't come to "Running" status.
>>>>>>>>>>
>>>>>>>>>> ERROR
>>>>>>>>>> {org.apache.stratos.cloud.controller.iaases.kubernetes.KubernetesIaas} -
>>>>>>>>>> Pod status did not change to running within 120 sec, hence removing
>>>>>>>>>> replication controller and pod: [cluster-id] php.php.domain [member-id]
>>>>>>>>>> php.php.domaine9f27b1c-6694-4b70-b939-f80b2ad74168
>>>>>>>>>> [replication-controller-id]
>>>>>>>>>> php-php-domaine9f27b1c-6694-4b70-b939-f80b2ad74168 [pod-id]
>>>>>>>>>> php-php-domaine9f27b1c-6694-4b70-b939-f80b2ad74168
>>>>>>>>>>
>>>>>>>>>> But AS doesn't know about this. Later AS moved these members to
>>>>>>>>>> obsolete list. When AS asks CC to terminate these pods, CC throws exception
>>>>>>>>>> saying member context not found.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *rajkumar@rajkumar-Latitude-E6540:/opt/stratos-packs/apache-stratos-4.1.0-SNAPSHOT/bin$
>>>>>>>>>> [2014-12-31 20:02:16,728]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.context.partition.ClusterLevelPartitionContext}
>>>>>>>>>> -  Pending state of member:
>>>>>>>>>> php.php.domaine9f27b1c-6694-4b70-b939-f80b2ad74168 is expired. Adding as an
>>>>>>>>>> obsoleted member.[2014-12-31 20:03:31,788]  INFO
>>>>>>>>>> {org.apache.stratos.autoscaler.client.CloudControllerClient} -  Terminating
>>>>>>>>>> instance via cloud controller: [member]
>>>>>>>>>> php.php.domaine9f27b1c-6694-4b70-b939-f80b2ad74168[2014-12-31 20:03:31,798]
>>>>>>>>>> ERROR
>>>>>>>>>> {org.apache.stratos.cloud.controller.services.impl.CloudControllerServiceImpl}
>>>>>>>>>> -  Member termination failed, member context not found: [member-id]
>>>>>>>>>> php.php.domaine9f27b1c-6694-4b70-b939-f80b2ad74168[2014-12-31 20:03:31,799]
>>>>>>>>>> ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} -  Member
>>>>>>>>>> termination failed, member context not found: [member-id]
>>>>>>>>>> php.php.domaine9f27b1c-6694-4b70-b939-f80b2ad74168org.apache.stratos.cloud.controller.exception.InvalidMemberException:
>>>>>>>>>> Member termination failed, member context not found: [member-id]
>>>>>>>>>> php.php.domaine9f27b1c-6694-4b70-b939-f80b2ad74168*
>>>>>>>>>>
>>>>>>>>>> Is this a correct flow?
>>>>>>>>>>
>>>>>>>>>> IMO, we don't need to monitor the pods' status at CC side.
>>>>>>>>>> Rather, AS will move these pods to obsolete list once the pending time is
>>>>>>>>>> expired. Then AS will ask CC to delete these pods. This is how VM scenario
>>>>>>>>>> works now. Otherwise, CC and AS will not be in same state. For example, AS
>>>>>>>>>> will ask to terminate a member which CC does not have.
>>>>>>>>>>
>>>>>>>>>> One more thing I noticed that member context is added to CC's
>>>>>>>>>> data holder only after pods are coming to "Running" status. IMO, we need to
>>>>>>>>>> add the member context as soon as pods are created. This ensures that when
>>>>>>>>>> AS asks a member to terminate, CC will always have it.
>>>>>>>>>>
>>>>>>>>>> wdyt?
>>>>>>>>>>
>>>>>>>>>> Thanks.
>>>>>>>>>>
>>>>>>>>>> On Wed, Dec 31, 2014 at 7:47 PM, Rajkumar Rajaratnam <
>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks Dakshika,
>>>>>>>>>>>
>>>>>>>>>>> Yes my stratos-installer was outdated. Its working fine now.
>>>>>>>>>>>
>>>>>>>>>>> Thanks.
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Dec 30, 2014 at 11:35 AM, Dakshika Jayathilaka <
>>>>>>>>>>> dakshika@wso2.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Raj,
>>>>>>>>>>>>
>>>>>>>>>>>> seems you  missed below DB sqls,  reside on *tools/stratos-*
>>>>>>>>>>>> *installer/resources*.
>>>>>>>>>>>>
>>>>>>>>>>>> application-mysql.sql
>>>>>>>>>>>> security-mysql.sql
>>>>>>>>>>>>
>>>>>>>>>>>> execute both and retry..
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you,
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>>
>>>>>>>>>>>> *Dakshika Jayathilaka*
>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>> WSO2, Inc.
>>>>>>>>>>>> lean.enterprise.middleware
>>>>>>>>>>>> 0771100911
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Dec 30, 2014 at 9:07 AM, Rajkumar Rajaratnam <
>>>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am trying our kubernetes. I am getting the following
>>>>>>>>>>>>> exception while deploying an application.
>>>>>>>>>>>>>
>>>>>>>>>>>>> [2014-12-30 09:02:14,414]  INFO
>>>>>>>>>>>>> {org.apache.stratos.autoscaler.api.AutoScalerServiceImpl} -  Adding
>>>>>>>>>>>>> application: [application-id] single-cartridge-app
>>>>>>>>>>>>> [2014-12-30 09:02:14,482]  INFO
>>>>>>>>>>>>> {org.wso2.carbon.core.services.util.CarbonAuthenticationUtil} -
>>>>>>>>>>>>> 'admin@carbon.super [-1234]' logged in at [2014-12-30
>>>>>>>>>>>>> 09:02:14,482+0530]
>>>>>>>>>>>>> [2014-12-30 09:02:14,492] ERROR
>>>>>>>>>>>>> {org.wso2.carbon.identity.oauth.dao.OAuthAppDAO} -  Error when executing
>>>>>>>>>>>>> the SQL : SELECT * FROM IDN_OAUTH_CONSUMER_APPS WHERE USERNAME=? AND
>>>>>>>>>>>>> TENANT_ID=? AND APP_NAME=?
>>>>>>>>>>>>> [2014-12-30 09:02:14,493] ERROR
>>>>>>>>>>>>> {org.wso2.carbon.identity.oauth.dao.OAuthAppDAO} -  Table
>>>>>>>>>>>>> 'userstore.IDN_OAUTH_CONSUMER_APPS' doesn't exist
>>>>>>>>>>>>> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException:
>>>>>>>>>>>>> Table 'userstore.IDN_OAUTH_CONSUMER_APPS' doesn't exist
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>>>>>>>>>>>>>     at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
>>>>>>>>>>>>>     at com.mysql.jdbc.Util.getInstance(Util.java:386)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1054)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4237)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4169)
>>>>>>>>>>>>>     at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2617)
>>>>>>>>>>>>>     at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2778)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2834)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2156)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2313)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.identity.oauth.dao.OAuthAppDAO.isDuplicateApplication(OAuthAppDAO.java:372)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.identity.oauth.dao.OAuthAppDAO.addOAuthApplication(OAuthAppDAO.java:67)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.identity.oauth.OAuthAdminService.registerOAuthApplicationData(OAuthAdminService.java:228)
>>>>>>>>>>>>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>>>>>>>>>>> Method)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>>>>>>>>>     at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>>>>>>>>>>>     at java.lang.Thread.run(Thread.java:662)
>>>>>>>>>>>>> [2014-12-30 09:02:14,496] ERROR
>>>>>>>>>>>>> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} -  Error when
>>>>>>>>>>>>> reading the application information from the persistence store.
>>>>>>>>>>>>> org.wso2.carbon.identity.oauth.IdentityOAuthAdminException:
>>>>>>>>>>>>> Error when reading the application information from the persistence store.
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.identity.oauth.dao.OAuthAppDAO.isDuplicateApplication(OAuthAppDAO.java:383)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.identity.oauth.dao.OAuthAppDAO.addOAuthApplication(OAuthAppDAO.java:67)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.identity.oauth.OAuthAdminService.registerOAuthApplicationData(OAuthAdminService.java:228)
>>>>>>>>>>>>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>>>>>>>>>>> Method)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>>>>>>>>>     at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>>>>>>>>>>>     at java.lang.Thread.run(Thread.java:662)
>>>>>>>>>>>>> [2014-12-30 09:02:14,511] ERROR
>>>>>>>>>>>>> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} -
>>>>>>>>>>>>> org.apache.axis2.AxisFault: Error when reading the application information
>>>>>>>>>>>>> from the persistence store.
>>>>>>>>>>>>> org.apache.stratos.autoscaler.exception.AutoScalerException:
>>>>>>>>>>>>> org.apache.axis2.AxisFault: Error when reading the application information
>>>>>>>>>>>>> from the persistence store.
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.stratos.autoscaler.applications.parser.DefaultApplicationParser.createToken(DefaultApplicationParser.java:714)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.stratos.autoscaler.applications.parser.DefaultApplicationParser.parse(DefaultApplicationParser.java:109)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.stratos.autoscaler.api.AutoScalerServiceImpl.addApplication(AutoScalerServiceImpl.java:173)
>>>>>>>>>>>>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>>>>>>>>>>> Method)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>>>>>>>>>     at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>>>>>>>>>>>     at java.lang.Thread.run(Thread.java:662)
>>>>>>>>>>>>> Caused by: org.apache.axis2.AxisFault: Error when reading the
>>>>>>>>>>>>> application information from the persistence store.
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.identity.oauth.stub.OAuthAdminServiceStub.registerOAuthApplicationData(OAuthAdminServiceStub.java:323)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.stratos.autoscaler.client.oAuthAdminServiceClient.registerOauthApplication(oAuthAdminServiceClient.java:97)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.stratos.autoscaler.applications.parser.DefaultApplicationParser.createToken(DefaultApplicationParser.java:712)
>>>>>>>>>>>>>     ... 46 more
>>>>>>>>>>>>> [2014-12-30 09:02:14,521] ERROR
>>>>>>>>>>>>> {org.apache.stratos.rest.endpoint.handlers.CustomExceptionMapper} -  Error
>>>>>>>>>>>>> while fulfilling the request
>>>>>>>>>>>>> org.apache.stratos.rest.endpoint.exception.RestAPIException
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.stratos.rest.endpoint.api.StratosApiV41Utils.addApplication(StratosApiV41Utils.java:974)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.stratos.rest.endpoint.api.StratosApiV41.addApplication(StratosApiV41.java:354)
>>>>>>>>>>>>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>>>>>>>>>>> Method)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>>>>>>>>>     at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:180)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:194)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:100)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:57)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:93)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:271)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:239)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:223)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:203)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:137)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:159)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:286)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:206)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:262)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>>>>>>>>>>>     at java.lang.Thread.run(Thread.java:662)
>>>>>>>>>>>>> Caused by: org.apache.axis2.AxisFault:
>>>>>>>>>>>>> org.apache.axis2.AxisFault: Error when reading the application information
>>>>>>>>>>>>> from the persistence store.
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.stratos.autoscaler.stub.AutoScalerServiceStub.addApplication(AutoScalerServiceStub.java:3223)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.stratos.manager.client.AutoscalerServiceClient.addApplication(AutoscalerServiceClient.java:122)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>> org.apache.stratos.rest.endpoint.api.StratosApiV41Utils.addApplication(StratosApiV41Utils.java:970)
>>>>>>>>>>>>>     ...
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any idea?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Dec 29, 2014 at 4:06 PM, Imesh Gunaratne <
>>>>>>>>>>>>> imesh@apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I did an improvement in the container startup logic to wait
>>>>>>>>>>>>>> until the pod status changes to Runnning state. Here cloud controller will
>>>>>>>>>>>>>> wait until two minutes to a pod to become active. if not the call will be
>>>>>>>>>>>>>> rolled back. Pod activation timeout can be configured using the system
>>>>>>>>>>>>>> property: stratos.pod.activation.timeout.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Once pod changes to Running state, IP addresses will be
>>>>>>>>>>>>>> updated as follows:
>>>>>>>>>>>>>> - Member's default private IP: Pod's IP address
>>>>>>>>>>>>>> - Member's default public IP: Pod's host IP address
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> A sample log:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [2014-12-29 15:43:02,419]  INFO Starting container:
>>>>>>>>>>>>>> [cartridge-type] php
>>>>>>>>>>>>>> [2014-12-29 15:43:02,454]  INFO Creating replication
>>>>>>>>>>>>>> controller: [replication-controller-id]
>>>>>>>>>>>>>> php-php-domainfc7e626a-e1e0-456d-8528-744f256c1f7a [name]
>>>>>>>>>>>>>> php-php-domainfc7e626a-e1e0-456d-8528-744f256c1f7a [docker-image]
>>>>>>>>>>>>>> stratos/php:4.1.0-alpha [container-ports] [80] [replicas] 1
>>>>>>>>>>>>>> [2014-12-29 15:43:02,559]  INFO Replication controller
>>>>>>>>>>>>>> created successfully: [replication-controller-id]
>>>>>>>>>>>>>> php-php-domainfc7e626a-e1e0-456d-8528-744f256c1f7a [name]
>>>>>>>>>>>>>> php-php-domainfc7e626a-e1e0-456d-8528-744f256c1f7a [docker-image]
>>>>>>>>>>>>>> stratos/php:4.1.0-alpha [container-port] [80] [replicas] 1
>>>>>>>>>>>>>> [2014-12-29 15:43:02,560]  INFO Creating service:
>>>>>>>>>>>>>> [service-id] php-php-domain-http-80 [service-name] php-php-domain-http-80
>>>>>>>>>>>>>> [service-port] 4500 [container-port] 80, [public-ip] 172.17.8.100
>>>>>>>>>>>>>> [2014-12-29 15:43:02,598]  INFO Service created successfully:
>>>>>>>>>>>>>> [service-id] php-php-domain-http-80 [service-name] php-php-domain-http-80
>>>>>>>>>>>>>> [service-port] 4500 [container-port] 80, [public-ip] 172.17.8.100
>>>>>>>>>>>>>> [2014-12-29 15:43:07,412]  *INFO Waiting pod status to be
>>>>>>>>>>>>>> changed to running*: [member-id]
>>>>>>>>>>>>>> php.php.domainfc7e626a-e1e0-456d-8528-744f256c1f7a [pod-id]
>>>>>>>>>>>>>> 440b7235-8f43-11e4-b193-080027f35f1b [current-pod-status] waiting
>>>>>>>>>>>>>> [2014-12-29 15:43:18,459]  *INFO Waiting pod status to be
>>>>>>>>>>>>>> changed to running*: [member-id]
>>>>>>>>>>>>>> php.php.domainfc7e626a-e1e0-456d-8528-744f256c1f7a [pod-id]
>>>>>>>>>>>>>> 440b7235-8f43-11e4-b193-080027f35f1b [current-pod-status] waiting
>>>>>>>>>>>>>> [2014-12-29 15:43:29,509]  *INFO Pod status changed to
>>>>>>>>>>>>>> running*: [member-id]
>>>>>>>>>>>>>> php.php.domainfc7e626a-e1e0-456d-8528-744f256c1f7a [pod-id]
>>>>>>>>>>>>>> 440b7235-8f43-11e4-b193-080027f35f1b
>>>>>>>>>>>>>> [2014-12-29 15:43:33,595]  INFO Instance started
>>>>>>>>>>>>>> successfully: [cartridge-type] php [cluster-id] php.php.domain
>>>>>>>>>>>>>> [instance-id] 440b7235-8f43-11e4-b193-080027f35f1b [default-private-ip]
>>>>>>>>>>>>>> 10.100.37.4 [default-public-ip] 172.17.8.100
>>>>>>>>>>>>>> [2014-12-29 15:43:33,601]  INFO Publishing instance spawned
>>>>>>>>>>>>>> event: [service-name] php [cluster-id] php.php.domain [cluster-instance-id]
>>>>>>>>>>>>>> single-cartridge-app-1 [member-id]
>>>>>>>>>>>>>> php.php.domainfc7e626a-e1e0-456d-8528-744f256c1f7a [instance-id]
>>>>>>>>>>>>>> 440b7235-8f43-11e4-b193-080027f35f1b [network-partition-id] network-p1
>>>>>>>>>>>>>> [partition-id] p1 [lb-cluster-id] null
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sun, Dec 28, 2014 at 7:40 PM, Imesh Gunaratne <
>>>>>>>>>>>>>> imesh@apache.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks Lakmal!
>>>>>>>>>>>>>>> After the application becomes active I can see that
>>>>>>>>>>>>>>> Autoscaler is detecting a fauly member:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [2014-12-28 19:12:14,016]  INFO
>>>>>>>>>>>>>>> {org.apache.stratos.cep.extension.FaultHandlingWindowProcessor} -  Faulty
>>>>>>>>>>>>>>> member detected [member-id]
>>>>>>>>>>>>>>> php.php.domaincc7fb2e4-ea96-4b70-8706-353e4225f170 with [last time-stamp]
>>>>>>>>>>>>>>> 1419774066943 [time-out] 60000 milliseconds
>>>>>>>>>>>>>>> [2014-12-28 19:12:14,025]  INFO
>>>>>>>>>>>>>>> {org.apache.stratos.cep.extension.FaultHandlingWindowProcessor} -
>>>>>>>>>>>>>>>  Publishing member fault event for [member-id]
>>>>>>>>>>>>>>> php.php.domaincc7fb2e4-ea96-4b70-8706-353e4225f170
>>>>>>>>>>>>>>> [2014-12-28 19:12:14,036]  INFO
>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.status.processor.cluster.ClusterStatusInactiveProcessor}
>>>>>>>>>>>>>>> -  Publishing Cluster inactivate event for [application]:
>>>>>>>>>>>>>>> single-cartridge-app [cluster]: php.php.domain
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'm currently investigating the reason for this.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 7:09 PM, Lakmal Warusawithana <
>>>>>>>>>>>>>>> lakmal@wso2.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> thanks Imesh, will give a try.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 6:35 PM, Imesh Gunaratne <
>>>>>>>>>>>>>>>> imesh@apache.org> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> After fixing the above issue, the application is getting
>>>>>>>>>>>>>>>>> active! :-) I used a single cartridge application for this test. Now we can
>>>>>>>>>>>>>>>>> create a more complex application and see how it works.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I have written a small guide on describing all steps for
>>>>>>>>>>>>>>>>> trying out Kubernetes workflow, please try out:
>>>>>>>>>>>>>>>>> https://gist.github.com/imesh/b8f81fac8de39183a504
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 5:25 PM, Imesh Gunaratne <
>>>>>>>>>>>>>>>>> imesh@apache.org> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I think it should be as follows:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Member private IP => container IP
>>>>>>>>>>>>>>>>>> Member default private IP => container IP
>>>>>>>>>>>>>>>>>> Member public IPs = > service proxy IPs
>>>>>>>>>>>>>>>>>> Member default public IP => kubernetes master/host IP
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> If public IP assignment is not enabled service proxies
>>>>>>>>>>>>>>>>>> are not created, I can recall that Lakmal suggested this.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 5:10 PM, Imesh Gunaratne <
>>>>>>>>>>>>>>>>>> imesh@apache.org> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I fixed the above issue by setting the private and
>>>>>>>>>>>>>>>>>>> public IPs to Kubernetes Master IP, we will need to review few things here:
>>>>>>>>>>>>>>>>>>> - Private IP address of a member needs to be the service
>>>>>>>>>>>>>>>>>>> proxy IP address
>>>>>>>>>>>>>>>>>>> - However each service proxy will have a unique IP
>>>>>>>>>>>>>>>>>>> address (each transport of the server)
>>>>>>>>>>>>>>>>>>> - Which means there will be a private IP for each
>>>>>>>>>>>>>>>>>>> transport
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Will discuss this and see how we can resolve this
>>>>>>>>>>>>>>>>>>> problem later on. For the moment setting the Kubernetes Master IP is
>>>>>>>>>>>>>>>>>>> sufficient IMO.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> After fixing the above issue, now I see the below error:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [2014-12-26 17:05:30,547] ERROR
>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.event.receiver.topology.AutoscalerTopologyEventReceiver}
>>>>>>>>>>>>>>>>>>> -  Error processing event null
>>>>>>>>>>>>>>>>>>> java.lang.NullPointerException
>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> org.apache.stratos.autoscaler.monitor.cluster.ClusterMonitor.getClusterInstanceContext(ClusterMonitor.java:1116)
>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> org.apache.stratos.autoscaler.monitor.cluster.ClusterMonitor.handleMemberActivatedEvent(ClusterMonitor.java:875)
>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> org.apache.stratos.autoscaler.event.receiver.topology.AutoscalerTopologyEventReceiver$12.onEvent(AutoscalerTopologyEventReceiver.java:393)
>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> org.apache.stratos.messaging.listener.EventListener.update(EventListener.java:42)
>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> java.util.Observable.notifyObservers(Observable.java:159)
>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 4:56 PM, Rajkumar Rajaratnam <
>>>>>>>>>>>>>>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 4:34 PM, Imesh Gunaratne <
>>>>>>>>>>>>>>>>>>>> imesh@apache.org> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi Lakmal,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I added these to Kubernetes Cluster definition for the
>>>>>>>>>>>>>>>>>>>>> time being since I'm finding/fixing many issues in agent and docker base
>>>>>>>>>>>>>>>>>>>>> image, will discuss and move them to a proper location later on. WDYT?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>       "clusterId": "kubernetes-cluster-1",
>>>>>>>>>>>>>>>>>>>>>       "description": "Kubernetes CoreOS cluster",
>>>>>>>>>>>>>>>>>>>>>       ...
>>>>>>>>>>>>>>>>>>>>>     "property":[
>>>>>>>>>>>>>>>>>>>>>        {
>>>>>>>>>>>>>>>>>>>>>           "name":"payload_parameter.MB_IP",
>>>>>>>>>>>>>>>>>>>>>           "value":"192.168.59.3"
>>>>>>>>>>>>>>>>>>>>>        },
>>>>>>>>>>>>>>>>>>>>>        {
>>>>>>>>>>>>>>>>>>>>>           "name":"payload_parameter.MB_PORT",
>>>>>>>>>>>>>>>>>>>>>           "value":"1883"
>>>>>>>>>>>>>>>>>>>>>        },
>>>>>>>>>>>>>>>>>>>>>        {
>>>>>>>>>>>>>>>>>>>>>           "name":"payload_parameter.CEP_IP",
>>>>>>>>>>>>>>>>>>>>>           "value":"192.168.59.3"
>>>>>>>>>>>>>>>>>>>>>        },
>>>>>>>>>>>>>>>>>>>>>        {
>>>>>>>>>>>>>>>>>>>>>           "name":"payload_parameter.CEP_PORT",
>>>>>>>>>>>>>>>>>>>>>           "value":"7711"
>>>>>>>>>>>>>>>>>>>>>        },
>>>>>>>>>>>>>>>>>>>>>        {
>>>>>>>>>>>>>>>>>>>>>           "name":"payload_parameter.LOG_LEVEL",
>>>>>>>>>>>>>>>>>>>>>           "value":"DEBUG"
>>>>>>>>>>>>>>>>>>>>>        }
>>>>>>>>>>>>>>>>>>>>>     ]
>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> After the above fix I'm now seeing the below error in
>>>>>>>>>>>>>>>>>>>>> cloud controller. Seems like we do not send the ip address in the member
>>>>>>>>>>>>>>>>>>>>> activated event. Need to check agent logic again:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> [2014-12-26 16:28:06,218]  INFO
>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.manager.listener.InstanceStatusListener} -  Instance
>>>>>>>>>>>>>>>>>>>>> status message received
>>>>>>>>>>>>>>>>>>>>> [2014-12-26 16:28:06,218]  INFO
>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.manager.listener.InstanceStatusListener} -  Event class
>>>>>>>>>>>>>>>>>>>>> name:
>>>>>>>>>>>>>>>>>>>>> org.apache.stratos.messaging.event.instance.status.InstanceActivatedEvent
>>>>>>>>>>>>>>>>>>>>> [2014-12-26 16:28:06,219]  INFO
>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} -
>>>>>>>>>>>>>>>>>>>>>  member started event adding status activated
>>>>>>>>>>>>>>>>>>>>> [2014-12-26 16:28:06,225]  INFO
>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>>>>>>>>>>>>>>>>>>>> -  Publishing member activated event: [service-name] php [cluster-id]
>>>>>>>>>>>>>>>>>>>>> php.php.domain [cluster-instance-id] single-cartridge-app-1 [member-id]
>>>>>>>>>>>>>>>>>>>>> php.php.domaind7655e6f-f962-425b-b93b-2bf7333dc37d [network-partition-id]
>>>>>>>>>>>>>>>>>>>>>  [partition-id]
>>>>>>>>>>>>>>>>>>>>> [2014-12-26 16:28:06,244] ERROR
>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.messaging.message.receiver.topology.TopologyEventMessageDelegator}
>>>>>>>>>>>>>>>>>>>>> -  Failed to retrieve topology event message
>>>>>>>>>>>>>>>>>>>>> java.lang.RuntimeException: *No ip address found in
>>>>>>>>>>>>>>>>>>>>> member activated event: *[service] php [cluster]
>>>>>>>>>>>>>>>>>>>>> php.php.domain [member] php.php.domaind7655e6f-f962-425b-b93b-2bf7333dc37d
>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>> org.apache.stratos.messaging.message.processor.topology.MemberActivatedMessageProcessor.doProcess(MemberActivatedMessageProcessor.java:102)
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I guess spawned containers doesn't have IP addresses.
>>>>>>>>>>>>>>>>>>>> How we are getting IP address for a container is like this.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> // PodActivationWatcher.java
>>>>>>>>>>>>>>>>>>>> String hostIP = pod.getCurrentState().getHost();
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> This is returning null or empty, so member activated
>>>>>>>>>>>>>>>>>>>> event is not having an IP.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 3:49 PM, Lakmal Warusawithana
>>>>>>>>>>>>>>>>>>>>> <lakmal@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Shall we used metadata service to publish all these
>>>>>>>>>>>>>>>>>>>>>> generic information?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 3:40 PM, Imesh Gunaratne <
>>>>>>>>>>>>>>>>>>>>>> imesh@apache.org> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> As I can see in the agent.conf some configuration
>>>>>>>>>>>>>>>>>>>>>>> parameters are missing:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> These parameters are not sent in Kuberbenetes
>>>>>>>>>>>>>>>>>>>>>>> workflow, we might need to think how to send them. ASFAIK in VM workflow
>>>>>>>>>>>>>>>>>>>>>>> most of these missing parameters are sent by puppet master.
>>>>>>>>>>>>>>>>>>>>>>> ‚Äč
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 3:18 PM, Imesh Gunaratne <
>>>>>>>>>>>>>>>>>>>>>>> imesh@apache.org> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Seems like we have a string formatting issue in
>>>>>>>>>>>>>>>>>>>>>>>> agent:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> http://stackoverflow.com/questions/18053500/typeerror-not-all-arguments-converted-during-string-formatting-python
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 3:11 PM, Imesh Gunaratne <
>>>>>>>>>>>>>>>>>>>>>>>> imesh@apache.org> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I fixed the above issue with INTERNAL parameter
>>>>>>>>>>>>>>>>>>>>>>>>> and now I'm seeing the following error:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> [2014-12-26 09:37:27,321] INFO
>>>>>>>>>>>>>>>>>>>>>>>>> {cartridgeagentconfiguration.py:__init__} - Cartridge agent configuration
>>>>>>>>>>>>>>>>>>>>>>>>> initialized
>>>>>>>>>>>>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>>>>>>>>>>>>   File "/usr/lib/python2.7/logging/__init__.py",
>>>>>>>>>>>>>>>>>>>>>>>>> line 850, in emit
>>>>>>>>>>>>>>>>>>>>>>>>>     msg = self.format(record)
>>>>>>>>>>>>>>>>>>>>>>>>>   File "/usr/lib/python2.7/logging/__init__.py",
>>>>>>>>>>>>>>>>>>>>>>>>> line 723, in format
>>>>>>>>>>>>>>>>>>>>>>>>>     return fmt.format(record)
>>>>>>>>>>>>>>>>>>>>>>>>>   File "/usr/lib/python2.7/logging/__init__.py",
>>>>>>>>>>>>>>>>>>>>>>>>> line 464, in format
>>>>>>>>>>>>>>>>>>>>>>>>>     record.message = record.getMessage()
>>>>>>>>>>>>>>>>>>>>>>>>>   File "/usr/lib/python2.7/logging/__init__.py",
>>>>>>>>>>>>>>>>>>>>>>>>> line 328, in getMessage
>>>>>>>>>>>>>>>>>>>>>>>>>     msg = msg % self.args
>>>>>>>>>>>>>>>>>>>>>>>>> TypeError: not all arguments converted during
>>>>>>>>>>>>>>>>>>>>>>>>> string formatting
>>>>>>>>>>>>>>>>>>>>>>>>> Logged from file agent.py, line 331
>>>>>>>>>>>>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>>>>>>>>>>>>   File "/usr/lib/python2.7/logging/__init__.py",
>>>>>>>>>>>>>>>>>>>>>>>>> line 850, in emit
>>>>>>>>>>>>>>>>>>>>>>>>>     msg = self.format(record)
>>>>>>>>>>>>>>>>>>>>>>>>>   File "/usr/lib/python2.7/logging/__init__.py",
>>>>>>>>>>>>>>>>>>>>>>>>> line 723, in format
>>>>>>>>>>>>>>>>>>>>>>>>>     return fmt.format(record)
>>>>>>>>>>>>>>>>>>>>>>>>>   File "/usr/lib/python2.7/logging/__init__.py",
>>>>>>>>>>>>>>>>>>>>>>>>> line 464, in format
>>>>>>>>>>>>>>>>>>>>>>>>>     record.message = record.getMessage()
>>>>>>>>>>>>>>>>>>>>>>>>>   File "/usr/lib/python2.7/logging/__init__.py",
>>>>>>>>>>>>>>>>>>>>>>>>> line 328, in getMessage
>>>>>>>>>>>>>>>>>>>>>>>>>     msg = msg % self.args
>>>>>>>>>>>>>>>>>>>>>>>>> TypeError: not all arguments converted during
>>>>>>>>>>>>>>>>>>>>>>>>> string formatting
>>>>>>>>>>>>>>>>>>>>>>>>> Logged from file agent.py, line 331
>>>>>>>>>>>>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>>>>>>>>>>>>   File "/usr/lib/python2.7/logging/__init__.py",
>>>>>>>>>>>>>>>>>>>>>>>>> line 850, in emit
>>>>>>>>>>>>>>>>>>>>>>>>>     msg = self.format(record)
>>>>>>>>>>>>>>>>>>>>>>>>>   File "/usr/lib/python2.7/logging/__init__.py",
>>>>>>>>>>>>>>>>>>>>>>>>> line 723, in format
>>>>>>>>>>>>>>>>>>>>>>>>>     return fmt.format(record)
>>>>>>>>>>>>>>>>>>>>>>>>>   File "/usr/lib/python2.7/logging/__init__.py",
>>>>>>>>>>>>>>>>>>>>>>>>> line 464, in format
>>>>>>>>>>>>>>>>>>>>>>>>>     record.message = record.getMessage()
>>>>>>>>>>>>>>>>>>>>>>>>>   File "/usr/lib/python2.7/logging/__init__.py",
>>>>>>>>>>>>>>>>>>>>>>>>> line 328, in getMessage
>>>>>>>>>>>>>>>>>>>>>>>>>     msg = msg % self.args
>>>>>>>>>>>>>>>>>>>>>>>>> TypeError: not all arguments converted during
>>>>>>>>>>>>>>>>>>>>>>>>> string formatting
>>>>>>>>>>>>>>>>>>>>>>>>> Logged from file agent.py, line 331
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 3:03 PM, Imesh Gunaratne <
>>>>>>>>>>>>>>>>>>>>>>>>> imesh@apache.org> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> As I found there was an issue in agent code, it
>>>>>>>>>>>>>>>>>>>>>>>>>> was using PROVIDER constant to fetch the INTERNAL parameter. I cannot see
>>>>>>>>>>>>>>>>>>>>>>>>>> PROVIDER parameter being using in the agent.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 2:32 PM, Gayan Gunarathne
>>>>>>>>>>>>>>>>>>>>>>>>>> <gayang@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Imesh,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Attachment is the sample launch_params that I
>>>>>>>>>>>>>>>>>>>>>>>>>>> have tested.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>> Gayan
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 2:22 PM, Gayan
>>>>>>>>>>>>>>>>>>>>>>>>>>> Gunarathne <gayang@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Imesh,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is that launch_params contains the PROVIDER
>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameter? I guess this comes when the provider is not available in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> payload. Seems like we are showing wrong error message.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Gayan
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 2:05 PM, Imesh
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Gunaratne <imesh@apache.org> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks Gayan, yes it was not there in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> launch_params, I just fixed it and seeing the below error:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2014-12-26 08:31:17,311] INFO
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> {cartridgeagentconfiguration.py:__init__} -  INTERNAL payload parameter is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not found
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2014-12-26 08:31:17,311] INFO
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> {cartridgeagentconfiguration.py:__init__} - Cartridge agent configuration
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initialized
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>   File
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "/usr/lib/python2.7/logging/__init__.py", line 850, in emit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>     msg = self.format(record)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>   File
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "/usr/lib/python2.7/logging/__init__.py", line 723, in format
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>     return fmt.format(record)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>   File
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "/usr/lib/python2.7/logging/__init__.py", line 464, in format
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>     record.message = record.getMessage()
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>   File
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "/usr/lib/python2.7/logging/__init__.py", line 328, in getMessage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>     msg = msg % self.args
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> TypeError: not all arguments converted during
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> string formatting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 1:49 PM, Gayan
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Gunarathne <gayang@wso2.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Imesh,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is happen when CLUSTER_INSTANCE_ID not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> contain as the payload parameter of the cartridge agent.I think we can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> first check the agent payload and see CLUSTER_INSTANCE_ID
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is there in the agent payload.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Gayan
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 1:39 PM, Imesh
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Gunaratne <imesh@apache.org> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I fixed the above issue in cartridge agent
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and re-created the base image, now I'm seeing another error:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> root@e82ebdc6-8cd5-11e4-85d5-080027f35f1b:~#
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cat /tmp/agent.screen.log
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>   File "agent.py", line 32, in <module>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>     from modules.extensions import
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> defaultextensionhandler
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>   File
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/defaultextensionhandler.py",
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line 22, in <module>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>     from ..util import extensionutils,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cartridgeagentutils
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>   File
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/util/extensionutils.py",
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line 29, in <module>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>     cartridge_agent_config =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cartridgeagentconfiguration.CartridgeAgentConfiguration()
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>   File
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/config/cartridgeagentconfiguration.py",
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line 349, in __init__
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>     CartridgeAgentConfiguration.instance =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CartridgeAgentConfiguration.__CartridgeAgentConfiguration()
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>   File
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/config/cartridgeagentconfiguration.py",
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line 226, in __init__
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>     raise RuntimeError(ex)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RuntimeError: Cannot find the value of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> required parameter: 'CLUSTER_INSTANCE_ID'
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I can see CLUSTER_INSTANCE_ID being passed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to the container, may be there is something wrong in the agent code, will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> investigate.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 26, 2014 at 2:40 AM, Imesh
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Gunaratne <imesh@apache.org> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Devs,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I fixed some of the issues in Kubernetes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iaas and tested the Kubernetes workflow with sample php docker image. Now
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> replication controllers, services and containers get created without a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem. We can also access the php server in the container.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However I noticed that agent does not send
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the instance started event to CEP. Following error was found in agent
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> screen log:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>   File "agent.py", line 21, in <module>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>     from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cartridgeagent.cartridgeagent.modules import eventsubscriber
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ImportError: No module named
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cartridgeagent.cartridgeagent.modules
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> May be agent was not updated in the php
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> docker image. Will check this tomorrow.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Gayan Gunarathne
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Technical Lead
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> WSO2 Inc. (http://wso2.com)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> email  : gayang@wso2.com  | mobile : +94
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 766819985
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Gayan Gunarathne
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Technical Lead
>>>>>>>>>>>>>>>>>>>>>>>>>>>> WSO2 Inc. (http://wso2.com)
>>>>>>>>>>>>>>>>>>>>>>>>>>>> email  : gayang@wso2.com  | mobile : +94
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 766819985
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Gayan Gunarathne
>>>>>>>>>>>>>>>>>>>>>>>>>>> Technical Lead
>>>>>>>>>>>>>>>>>>>>>>>>>>> WSO2 Inc. (http://wso2.com)
>>>>>>>>>>>>>>>>>>>>>>>>>>> email  : gayang@wso2.com  | mobile : +94
>>>>>>>>>>>>>>>>>>>>>>>>>>> 766819985
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>>>>>>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>>>>>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>>>>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>>>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> Lakmal Warusawithana
>>>>>>>>>>>>>>>>>>>>>> Vice President, Apache Stratos
>>>>>>>>>>>>>>>>>>>>>> Director - Cloud Architecture; WSO2 Inc.
>>>>>>>>>>>>>>>>>>>>>> Mobile : +94714289692
>>>>>>>>>>>>>>>>>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>>>>>>>>> Software Engineer, WSO2
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Mobile : +94777568639
>>>>>>>>>>>>>>>>>>>> Blog : rajkumarr.com
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Lakmal Warusawithana
>>>>>>>>>>>>>>>> Vice President, Apache Stratos
>>>>>>>>>>>>>>>> Director - Cloud Architecture; WSO2 Inc.
>>>>>>>>>>>>>>>> Mobile : +94714289692
>>>>>>>>>>>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>>>> Software Engineer, WSO2
>>>>>>>>>>>>>
>>>>>>>>>>>>> Mobile : +94777568639
>>>>>>>>>>>>> Blog : rajkumarr.com
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>> Software Engineer, WSO2
>>>>>>>>>>>
>>>>>>>>>>> Mobile : +94777568639
>>>>>>>>>>> Blog : rajkumarr.com
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>> Software Engineer, WSO2
>>>>>>>>>>
>>>>>>>>>> Mobile : +94777568639
>>>>>>>>>> Blog : rajkumarr.com
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Imesh Gunaratne
>>>>>>>>>
>>>>>>>>> Technical Lead, WSO2
>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Imesh Gunaratne
>>>>>>>>
>>>>>>>> Technical Lead, WSO2
>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Rajkumar Rajaratnam
>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>> Software Engineer, WSO2
>>>>>>>
>>>>>>> Mobile : +94777568639
>>>>>>> Blog : rajkumarr.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Rajkumar Rajaratnam
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>> Software Engineer, WSO2
>>>>>>
>>>>>> Mobile : +94777568639
>>>>>> Blog : rajkumarr.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Rajkumar Rajaratnam
>>>>> Committer & PMC Member, Apache Stratos
>>>>> Software Engineer, WSO2
>>>>>
>>>>> Mobile : +94777568639
>>>>> Blog : rajkumarr.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> Rajkumar Rajaratnam
>>> Committer & PMC Member, Apache Stratos
>>> Software Engineer, WSO2
>>>
>>> Mobile : +94777568639
>>> Blog : rajkumarr.com
>>>
>>
>>
>>
>> --
>> Rajkumar Rajaratnam
>> Committer & PMC Member, Apache Stratos
>> Software Engineer, WSO2
>>
>> Mobile : +94777568639
>> Blog : rajkumarr.com
>>
>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>



-- 
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2

Mobile : +94777568639
Blog : rajkumarr.com

Mime
View raw message