No, the current workflow is as follows:

- AS talks to CC to start an instance
- CC creates a new member context with a unique member Id, start a new thread to execute the actions, returns the call to AS.
- The above thread runs and try to start the instance, allocate IP addresses, assign storage, etc
- If everything successfully completes, CC will send the Instance Spawened event.
- If one of the actions fails, CC will raise an error, it will not terminate the instance since member context has been already returned to AS.
- AS will terminate the instance on the next obsolete member evaluation iteration.



On Thu, Jan 1, 2015 at 9:44 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com> wrote:


On Thu, Jan 1, 2015 at 9:42 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com> wrote:
Hi Imesh,

May be I didn't understand your solution completely, correct me if I am wrong.
  • adding the member to the topology as soon as instance is created (just spawned)
This should be "adding the member context to the data holder as soon as instance is created (just spawned). Sorry for the confusion :)
  • sending instance spawned event if everything else configured correctly (IP, Volume etc)
  • terminating the instance and sending instance terminated event if IP allocation fails or Volume allocation fails
Am I correct?

If correct, I would like to know when we will be adding the member to the topology? If we adding the member to the topology after sending instance spawned event, then there is a problem. Even if IP allocation goes wrong, we will not be able to send the member terminated event, because we can't send events for the members which are not in the topology.

That's why, IMO, we should add the member to the topology as soon as we get the instance start up call and we should send a new event (before instance spawned event) so that other components will be aware that a member is added to the topology. This way we can send member terminated event at the point where IP allocation goes wrong. This will be efficient and I guess it is not that complicated to implement.

wdyt?

Thanks.

On Thu, Jan 1, 2015 at 9:15 PM, Imesh Gunaratne <imesh@apache.org> wrote:
IMO we shouldn't make things much complicated. 

If we can wait until the point where IP addresses are allocated to send the Instance Spawned event, may be that's much simpler than the above suggestion.

On Thu, Jan 1, 2015 at 8:12 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com> wrote:
Hi,

I am quoting what we agreed from some other mail thread.

Correct approach would be to introduce a new topology event to fill the blind area between real instance creation via CC and member spawned event. So, as soon as CC receives a instance start up call, CC would add the member to topology and will send a new event with a new state possibly.

Thanks.

On Thu, Jan 1, 2015 at 7:33 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com> wrote:
s/if anything goes like IP allocation/if anything goes wrong like IP allocation

Thanks.

On Thu, Jan 1, 2015 at 7:20 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com> wrote:
Hi Imesh,

Yes, we can either do it as you mentioned, or we can send the member terminated event so that the other components will be aware of this instance termination. I guess the later one would the efficient solution as it AS will take 15 min (pending member timeout) to terminate such instances. Even in container case, we can do this.

If we know that the instance will never be activated in advance, we can terminate the instance in CC and send the member terminated event. We don't need to wait 15 min.

I can recall that we agreed to add the member to the topology as soon as member is spawned and send the member terminated event if anything goes like IP allocation. It is discussed here,

If floating IP allocation goes wrong, we are not terminating the instances at that point?

Thanks.

On Thu, Jan 1, 2015 at 6:34 PM, Imesh Gunaratne <imesh@apache.org> wrote:
As I see we have the same logic for VMs which terminate the instance if it could not allocate IP addresses:


String msg = "Error occurred while allocating predefined floating ip address: " + preDefinedIp +
                                    " / allocated ip:" + ip +
                                    " - terminating node:" + memberContext.toString();
                            log.error(msg);
                            // terminate instance
                            destroyNode(nodeMetadata.getId(), memberContext);
                            throw new CloudControllerException(msg);

We might need to change this logic to log the error message and wait until the Autoscaler terminate the instance.

Thanks 

On Thu, Jan 1, 2015 at 12:01 PM, Imesh Gunaratne <imesh@apache.org> wrote:
A good point Raj, yes, it is not consistent to let the cloud controller handle this logic as it is already done by the Autoscaler. I will fix this.

Thanks

On Wed, Dec 31, 2014 at 8:51 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com> wrote:
Hi,

Currently pods' status are monitored by CC. If pods are not getting "Running" status within a time period(default to 120s), replication controllers and pods are deleted by CC and nothing happens after that.

I faced some problems due to this. My pods and replication controller were deleted by CC, because pods didn't come to "Running" status.

ERROR {org.apache.stratos.cloud.controller.iaases.kubernetes.KubernetesIaas} -  Pod status did not change to running within 120 sec, hence removing replication controller and pod: [cluster-id] php.php.domain [member-id] php.php.domaine9f27b1c-6694-4b70-b939-f80b2ad74168 [replication-controller-id] php-php-domaine9f27b1c-6694-4b70-b939-f80b2ad74168 [pod-id] php-php-domaine9f27b1c-6694-4b70-b939-f80b2ad74168

But AS doesn't know about this. Later AS moved these members to obsolete list. When AS asks CC to terminate these pods, CC throws exception saying member context not found.

rajkumar@rajkumar-Latitude-E6540:/opt/stratos-packs/apache-stratos-4.1.0-SNAPSHOT/bin$ [2014-12-31 20:02:16,728]  INFO {org.apache.stratos.autoscaler.context.partition.ClusterLevelPartitionContext} -  Pending state of member: php.php.domaine9f27b1c-6694-4b70-b939-f80b2ad74168 is expired. Adding as an obsoleted member.
[2014-12-31 20:03:31,788]  INFO {org.apache.stratos.autoscaler.client.CloudControllerClient} -  Terminating instance via cloud controller: [member] php.php.domaine9f27b1c-6694-4b70-b939-f80b2ad74168
[2014-12-31 20:03:31,798] ERROR {org.apache.stratos.cloud.controller.services.impl.CloudControllerServiceImpl} -  Member termination failed, member context not found: [member-id] php.php.domaine9f27b1c-6694-4b70-b939-f80b2ad74168
[2014-12-31 20:03:31,799] ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} -  Member termination failed, member context not found: [member-id] php.php.domaine9f27b1c-6694-4b70-b939-f80b2ad74168
org.apache.stratos.cloud.controller.exception.InvalidMemberException: Member termination failed, member context not found: [member-id] php.php.domaine9f27b1c-6694-4b70-b939-f80b2ad74168


Is this a correct flow?

IMO, we don't need to monitor the pods' status at CC side. Rather, AS will move these pods to obsolete list once the pending time is expired. Then AS will ask CC to delete these pods. This is how VM scenario works now. Otherwise, CC and AS will not be in same state. For example, AS will ask to terminate a member which CC does not have.

One more thing I noticed that member context is added to CC's data holder only after pods are coming to "Running" status. IMO, we need to add the member context as soon as pods are created. This ensures that when AS asks a member to terminate, CC will always have it.

wdyt?

Thanks.

On Wed, Dec 31, 2014 at 7:47 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com> wrote:
Thanks Dakshika,

Yes my stratos-installer was outdated. Its working fine now.

Thanks.

On Tue, Dec 30, 2014 at 11:35 AM, Dakshika Jayathilaka <dakshika@wso2.com> wrote:
Hi Raj,

seems you  missed below DB sqls,  reside on tools/stratos-installer/resources.

application-mysql.sql
security-mysql.sql

execute both and retry.. 

Thank you,

Regards,

Dakshika Jayathilaka
Software Engineer
WSO2, Inc.
lean.enterprise.middleware

On Tue, Dec 30, 2014 at 9:07 AM, Rajkumar Rajaratnam <rajkumarr@wso2.com> wrote:
Hi,

I am trying our kubernetes. I am getting the following exception while deploying an application.

[2014-12-30 09:02:14,414]  INFO {org.apache.stratos.autoscaler.api.AutoScalerServiceImpl} -  Adding application: [application-id] single-cartridge-app
[2014-12-30 09:02:14,482]  INFO {org.wso2.carbon.core.services.util.CarbonAuthenticationUtil} -  'admin@carbon.super [-1234]' logged in at [2014-12-30 09:02:14,482+0530]
[2014-12-30 09:02:14,492] ERROR {org.wso2.carbon.identity.oauth.dao.OAuthAppDAO} -  Error when executing the SQL : SELECT * FROM IDN_OAUTH_CONSUMER_APPS WHERE USERNAME=? AND TENANT_ID=? AND APP_NAME=?
[2014-12-30 09:02:14,493] ERROR {org.wso2.carbon.identity.oauth.dao.OAuthAppDAO} -  Table 'userstore.IDN_OAUTH_CONSUMER_APPS' doesn't exist
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'userstore.IDN_OAUTH_CONSUMER_APPS' doesn't exist
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
    at com.mysql.jdbc.Util.getInstance(Util.java:386)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1054)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4237)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4169)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2617)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2778)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2834)
    at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2156)
    at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2313)
    at org.wso2.carbon.identity.oauth.dao.OAuthAppDAO.isDuplicateApplication(OAuthAppDAO.java:372)
    at org.wso2.carbon.identity.oauth.dao.OAuthAppDAO.addOAuthApplication(OAuthAppDAO.java:67)
    at org.wso2.carbon.identity.oauth.OAuthAdminService.registerOAuthApplicationData(OAuthAdminService.java:228)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212)
    at org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
    at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
    at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
    at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
    at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
    at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
    at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
    at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
    at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
    at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:662)
[2014-12-30 09:02:14,496] ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} -  Error when reading the application information from the persistence store.
org.wso2.carbon.identity.oauth.IdentityOAuthAdminException: Error when reading the application information from the persistence store.
    at org.wso2.carbon.identity.oauth.dao.OAuthAppDAO.isDuplicateApplication(OAuthAppDAO.java:383)
    at org.wso2.carbon.identity.oauth.dao.OAuthAppDAO.addOAuthApplication(OAuthAppDAO.java:67)
    at org.wso2.carbon.identity.oauth.OAuthAdminService.registerOAuthApplicationData(OAuthAdminService.java:228)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212)
    at org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
    at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
    at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
    at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
    at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
    at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
    at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
    at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
    at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
    at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:662)
[2014-12-30 09:02:14,511] ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} -  org.apache.axis2.AxisFault: Error when reading the application information from the persistence store.
org.apache.stratos.autoscaler.exception.AutoScalerException: org.apache.axis2.AxisFault: Error when reading the application information from the persistence store.
    at org.apache.stratos.autoscaler.applications.parser.DefaultApplicationParser.createToken(DefaultApplicationParser.java:714)
    at org.apache.stratos.autoscaler.applications.parser.DefaultApplicationParser.parse(DefaultApplicationParser.java:109)
    at org.apache.stratos.autoscaler.api.AutoScalerServiceImpl.addApplication(AutoScalerServiceImpl.java:173)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212)
    at org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
    at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
    at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
    at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
    at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
    at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
    at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
    at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
    at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
    at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.axis2.AxisFault: Error when reading the application information from the persistence store.
    at org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
    at org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
    at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
    at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
    at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
    at org.wso2.carbon.identity.oauth.stub.OAuthAdminServiceStub.registerOAuthApplicationData(OAuthAdminServiceStub.java:323)
    at org.apache.stratos.autoscaler.client.oAuthAdminServiceClient.registerOauthApplication(oAuthAdminServiceClient.java:97)
    at org.apache.stratos.autoscaler.applications.parser.DefaultApplicationParser.createToken(DefaultApplicationParser.java:712)
    ... 46 more
[2014-12-30 09:02:14,521] ERROR {org.apache.stratos.rest.endpoint.handlers.CustomExceptionMapper} -  Error while fulfilling the request
org.apache.stratos.rest.endpoint.exception.RestAPIException
    at org.apache.stratos.rest.endpoint.api.StratosApiV41Utils.addApplication(StratosApiV41Utils.java:974)
    at org.apache.stratos.rest.endpoint.api.StratosApiV41.addApplication(StratosApiV41.java:354)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:180)
    at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
    at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:194)
    at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:100)
    at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:57)
    at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:93)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:271)
    at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
    at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:239)
    at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:223)
    at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:203)
    at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:137)
    at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:159)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:286)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:206)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:262)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
    at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
    at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
    at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
    at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
    at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.axis2.AxisFault: org.apache.axis2.AxisFault: Error when reading the application information from the persistence store.
    at org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
    at org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
    at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
    at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
    at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
    at org.apache.stratos.autoscaler.stub.AutoScalerServiceStub.addApplication(AutoScalerServiceStub.java:3223)
    at org.apache.stratos.manager.client.AutoscalerServiceClient.addApplication(AutoscalerServiceClient.java:122)
    at org.apache.stratos.rest.endpoint.api.StratosApiV41Utils.addApplication(StratosApiV41Utils.java:970)
    ...

Any idea?

Thanks.

On Mon, Dec 29, 2014 at 4:06 PM, Imesh Gunaratne <imesh@apache.org> wrote:
I did an improvement in the container startup logic to wait until the pod status changes to Runnning state. Here cloud controller will wait until two minutes to a pod to become active. if not the call will be rolled back. Pod activation timeout can be configured using the system property: stratos.pod.activation.timeout.

Once pod changes to Running state, IP addresses will be updated as follows:
- Member's default private IP: Pod's IP address
- Member's default public IP: Pod's host IP address

A sample log:

[2014-12-29 15:43:02,419]  INFO Starting container: [cartridge-type] php
[2014-12-29 15:43:02,454]  INFO Creating replication controller: [replication-controller-id] php-php-domainfc7e626a-e1e0-456d-8528-744f256c1f7a [name] php-php-domainfc7e626a-e1e0-456d-8528-744f256c1f7a [docker-image] stratos/php:4.1.0-alpha [container-ports] [80] [replicas] 1
[2014-12-29 15:43:02,559]  INFO Replication controller created successfully: [replication-controller-id] php-php-domainfc7e626a-e1e0-456d-8528-744f256c1f7a [name] php-php-domainfc7e626a-e1e0-456d-8528-744f256c1f7a [docker-image] stratos/php:4.1.0-alpha [container-port] [80] [replicas] 1
[2014-12-29 15:43:02,560]  INFO Creating service: [service-id] php-php-domain-http-80 [service-name] php-php-domain-http-80 [service-port] 4500 [container-port] 80, [public-ip] 172.17.8.100
[2014-12-29 15:43:02,598]  INFO Service created successfully: [service-id] php-php-domain-http-80 [service-name] php-php-domain-http-80 [service-port] 4500 [container-port] 80, [public-ip] 172.17.8.100
[2014-12-29 15:43:07,412]  INFO Waiting pod status to be changed to running: [member-id] php.php.domainfc7e626a-e1e0-456d-8528-744f256c1f7a [pod-id] 440b7235-8f43-11e4-b193-080027f35f1b [current-pod-status] waiting
[2014-12-29 15:43:18,459]  INFO Waiting pod status to be changed to running: [member-id] php.php.domainfc7e626a-e1e0-456d-8528-744f256c1f7a [pod-id] 440b7235-8f43-11e4-b193-080027f35f1b [current-pod-status] waiting
[2014-12-29 15:43:29,509]  INFO Pod status changed to running: [member-id] php.php.domainfc7e626a-e1e0-456d-8528-744f256c1f7a [pod-id] 440b7235-8f43-11e4-b193-080027f35f1b
[2014-12-29 15:43:33,595]  INFO Instance started successfully: [cartridge-type] php [cluster-id] php.php.domain [instance-id] 440b7235-8f43-11e4-b193-080027f35f1b [default-private-ip] 10.100.37.4 [default-public-ip] 172.17.8.100
[2014-12-29 15:43:33,601]  INFO Publishing instance spawned event: [service-name] php [cluster-id] php.php.domain [cluster-instance-id] single-cartridge-app-1 [member-id] php.php.domainfc7e626a-e1e0-456d-8528-744f256c1f7a [instance-id] 440b7235-8f43-11e4-b193-080027f35f1b [network-partition-id] network-p1 [partition-id] p1 [lb-cluster-id] null

Thanks

On Sun, Dec 28, 2014 at 7:40 PM, Imesh Gunaratne <imesh@apache.org> wrote:
Thanks Lakmal! 
After the application becomes active I can see that Autoscaler is detecting a fauly member:

[2014-12-28 19:12:14,016]  INFO {org.apache.stratos.cep.extension.FaultHandlingWindowProcessor} -  Faulty member detected [member-id] php.php.domaincc7fb2e4-ea96-4b70-8706-353e4225f170 with [last time-stamp] 1419774066943 [time-out] 60000 milliseconds
[2014-12-28 19:12:14,025]  INFO {org.apache.stratos.cep.extension.FaultHandlingWindowProcessor} -  Publishing member fault event for [member-id] php.php.domaincc7fb2e4-ea96-4b70-8706-353e4225f170
[2014-12-28 19:12:14,036]  INFO {org.apache.stratos.autoscaler.status.processor.cluster.ClusterStatusInactiveProcessor} -  Publishing Cluster inactivate event for [application]: single-cartridge-app [cluster]: php.php.domain

I'm currently investigating the reason for this.

On Fri, Dec 26, 2014 at 7:09 PM, Lakmal Warusawithana <lakmal@wso2.com> wrote:
thanks Imesh, will give a try.

On Fri, Dec 26, 2014 at 6:35 PM, Imesh Gunaratne <imesh@apache.org> wrote:
Hi All,

After fixing the above issue, the application is getting active! :-) I used a single cartridge application for this test. Now we can create a more complex application and see how it works.

I have written a small guide on describing all steps for trying out Kubernetes workflow, please try out:

Thanks

On Fri, Dec 26, 2014 at 5:25 PM, Imesh Gunaratne <imesh@apache.org> wrote:
I think it should be as follows:

Member private IP => container IP
Member default private IP => container IP
Member public IPs = > service proxy IPs
Member default public IP => kubernetes master/host IP

If public IP assignment is not enabled service proxies are not created, I can recall that Lakmal suggested this.

Thanks

On Fri, Dec 26, 2014 at 5:10 PM, Imesh Gunaratne <imesh@apache.org> wrote:
I fixed the above issue by setting the private and public IPs to Kubernetes Master IP, we will need to review few things here:
- Private IP address of a member needs to be the service proxy IP address
- However each service proxy will have a unique IP address (each transport of the server)
- Which means there will be a private IP for each transport

Will discuss this and see how we can resolve this problem later on. For the moment setting the Kubernetes Master IP is sufficient IMO.

After fixing the above issue, now I see the below error: 

[2014-12-26 17:05:30,547] ERROR {org.apache.stratos.autoscaler.event.receiver.topology.AutoscalerTopologyEventReceiver} -  Error processing event null
java.lang.NullPointerException
at org.apache.stratos.autoscaler.monitor.cluster.ClusterMonitor.getClusterInstanceContext(ClusterMonitor.java:1116)
at org.apache.stratos.autoscaler.monitor.cluster.ClusterMonitor.handleMemberActivatedEvent(ClusterMonitor.java:875)
at org.apache.stratos.autoscaler.event.receiver.topology.AutoscalerTopologyEventReceiver$12.onEvent(AutoscalerTopologyEventReceiver.java:393)
at org.apache.stratos.messaging.listener.EventListener.update(EventListener.java:42)
at java.util.Observable.notifyObservers(Observable.java:159)
Thanks


On Fri, Dec 26, 2014 at 4:56 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com> wrote:


On Fri, Dec 26, 2014 at 4:34 PM, Imesh Gunaratne <imesh@apache.org> wrote:
Hi Lakmal,

I added these to Kubernetes Cluster definition for the time being since I'm finding/fixing many issues in agent and docker base image, will discuss and move them to a proper location later on. WDYT?

{
      "clusterId": "kubernetes-cluster-1",
      "description": "Kubernetes CoreOS cluster",
      ...
    "property":[
       {
          "name":"payload_parameter.MB_IP",
          "value":"192.168.59.3"
       },
       {
          "name":"payload_parameter.MB_PORT",
          "value":"1883"
       },
       {
          "name":"payload_parameter.CEP_IP",
          "value":"192.168.59.3"
       },
       {
          "name":"payload_parameter.CEP_PORT",
          "value":"7711"
       },
       {
          "name":"payload_parameter.LOG_LEVEL",
          "value":"DEBUG"
       }
    ]
}

After the above fix I'm now seeing the below error in cloud controller. Seems like we do not send the ip address in the member activated event. Need to check agent logic again:

[2014-12-26 16:28:06,218]  INFO {org.apache.stratos.manager.listener.InstanceStatusListener} -  Instance status message received
[2014-12-26 16:28:06,218]  INFO {org.apache.stratos.manager.listener.InstanceStatusListener} -  Event class name: org.apache.stratos.messaging.event.instance.status.InstanceActivatedEvent
[2014-12-26 16:28:06,219]  INFO {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} -  member started event adding status activated
[2014-12-26 16:28:06,225]  INFO {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher} -  Publishing member activated event: [service-name] php [cluster-id] php.php.domain [cluster-instance-id] single-cartridge-app-1 [member-id] php.php.domaind7655e6f-f962-425b-b93b-2bf7333dc37d [network-partition-id]  [partition-id]
[2014-12-26 16:28:06,244] ERROR {org.apache.stratos.messaging.message.receiver.topology.TopologyEventMessageDelegator} -  Failed to retrieve topology event message
java.lang.RuntimeException: No ip address found in member activated event: [service] php [cluster] php.php.domain [member] php.php.domaind7655e6f-f962-425b-b93b-2bf7333dc37d
at org.apache.stratos.messaging.message.processor.topology.MemberActivatedMessageProcessor.doProcess(MemberActivatedMessageProcessor.java:102)

I guess spawned containers doesn't have IP addresses. How we are getting IP address for a container is like this.

// PodActivationWatcher.java
String hostIP = pod.getCurrentState().getHost();

This is returning null or empty, so member activated event is not having an IP.

Thanks.

Thanks.

Thanks

On Fri, Dec 26, 2014 at 3:49 PM, Lakmal Warusawithana <lakmal@wso2.com> wrote:
Shall we used metadata service to publish all these generic information?

On Fri, Dec 26, 2014 at 3:40 PM, Imesh Gunaratne <imesh@apache.org> wrote:
As I can see in the agent.conf some configuration parameters are missing:


These parameters are not sent in Kuberbenetes workflow, we might need to think how to send them. ASFAIK in VM workflow most of these missing parameters are sent by puppet master.


On Fri, Dec 26, 2014 at 3:18 PM, Imesh Gunaratne <imesh@apache.org> wrote:

On Fri, Dec 26, 2014 at 3:11 PM, Imesh Gunaratne <imesh@apache.org> wrote:
I fixed the above issue with INTERNAL parameter and now I'm seeing the following error:

[2014-12-26 09:37:27,321] INFO {cartridgeagentconfiguration.py:__init__} - Cartridge agent configuration initialized
Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/__init__.py", line 850, in emit
    msg = self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 723, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 464, in format
    record.message = record.getMessage()
  File "/usr/lib/python2.7/logging/__init__.py", line 328, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Logged from file agent.py, line 331
Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/__init__.py", line 850, in emit
    msg = self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 723, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 464, in format
    record.message = record.getMessage()
  File "/usr/lib/python2.7/logging/__init__.py", line 328, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Logged from file agent.py, line 331
Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/__init__.py", line 850, in emit
    msg = self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 723, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 464, in format
    record.message = record.getMessage()
  File "/usr/lib/python2.7/logging/__init__.py", line 328, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Logged from file agent.py, line 331

On Fri, Dec 26, 2014 at 3:03 PM, Imesh Gunaratne <imesh@apache.org> wrote:
As I found there was an issue in agent code, it was using PROVIDER constant to fetch the INTERNAL parameter. I cannot see PROVIDER parameter being using in the agent.

Thanks

On Fri, Dec 26, 2014 at 2:32 PM, Gayan Gunarathne <gayang@wso2.com> wrote:
Hi Imesh,

Attachment is the sample launch_params that I have tested.

Thanks,
Gayan



On Fri, Dec 26, 2014 at 2:22 PM, Gayan Gunarathne <gayang@wso2.com> wrote:
Hi Imesh,

Is that launch_params contains the PROVIDER parameter? I guess this comes when the provider is not available in the payload. Seems like we are showing wrong error message.

Thanks,
Gayan

On Fri, Dec 26, 2014 at 2:05 PM, Imesh Gunaratne <imesh@apache.org> wrote:
Thanks Gayan, yes it was not there in launch_params, I just fixed it and seeing the below error:

[2014-12-26 08:31:17,311] INFO {cartridgeagentconfiguration.py:__init__} -  INTERNAL payload parameter is not found
[2014-12-26 08:31:17,311] INFO {cartridgeagentconfiguration.py:__init__} - Cartridge agent configuration initialized
Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/__init__.py", line 850, in emit
    msg = self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 723, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 464, in format
    record.message = record.getMessage()
  File "/usr/lib/python2.7/logging/__init__.py", line 328, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting

On Fri, Dec 26, 2014 at 1:49 PM, Gayan Gunarathne <gayang@wso2.com> wrote:
Hi Imesh,

This is happen when CLUSTER_INSTANCE_ID not contain as the payload parameter of the cartridge agent.I think we can first check the agent payload and see CLUSTER_INSTANCE_ID is there in the agent payload.

Thanks,
Gayan

On Fri, Dec 26, 2014 at 1:39 PM, Imesh Gunaratne <imesh@apache.org> wrote:
I fixed the above issue in cartridge agent and re-created the base image, now I'm seeing another error: 

root@e82ebdc6-8cd5-11e4-85d5-080027f35f1b:~# cat /tmp/agent.screen.log
Traceback (most recent call last):
  File "agent.py", line 32, in <module>
    from modules.extensions import defaultextensionhandler
  File "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/defaultextensionhandler.py", line 22, in <module>
    from ..util import extensionutils, cartridgeagentutils
  File "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/util/extensionutils.py", line 29, in <module>
    cartridge_agent_config = cartridgeagentconfiguration.CartridgeAgentConfiguration()
  File "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/config/cartridgeagentconfiguration.py", line 349, in __init__
    CartridgeAgentConfiguration.instance = CartridgeAgentConfiguration.__CartridgeAgentConfiguration()
  File "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/config/cartridgeagentconfiguration.py", line 226, in __init__
    raise RuntimeError(ex)
RuntimeError: Cannot find the value of required parameter: 'CLUSTER_INSTANCE_ID'

I can see CLUSTER_INSTANCE_ID being passed to the container, may be there is something wrong in the agent code, will investigate.

Thanks

On Fri, Dec 26, 2014 at 2:40 AM, Imesh Gunaratne <imesh@apache.org> wrote:
Hi Devs,

I fixed some of the issues in Kubernetes Iaas and tested the Kubernetes workflow with sample php docker image. Now replication controllers, services and containers get created without a problem. We can also access the php server in the container.

However I noticed that agent does not send the instance started event to CEP. Following error was found in agent screen log:

Traceback (most recent call last):
  File "agent.py", line 21, in <module>
    from cartridgeagent.cartridgeagent.modules import eventsubscriber
ImportError: No module named cartridgeagent.cartridgeagent.modules

May be agent was not updated in the php docker image. Will check this tomorrow.

Thanks


--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos



--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos



--

Gayan Gunarathne
Technical Lead
WSO2 Inc. (http://wso2.com)
email  : gayang@wso2.com  | mobile : +94 766819985
 




--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos



--

Gayan Gunarathne
Technical Lead
WSO2 Inc. (http://wso2.com)
email  : gayang@wso2.com  | mobile : +94 766819985
 




--

Gayan Gunarathne
Technical Lead
WSO2 Inc. (http://wso2.com)
email  : gayang@wso2.com  | mobile : +94 766819985
 




--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos



--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos



--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos



--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos



--
Lakmal Warusawithana
Vice President, Apache Stratos
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692



--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos



--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2

Mobile : +94777568639



--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos



--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos



--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos



--
Lakmal Warusawithana
Vice President, Apache Stratos
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692



--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos



--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos



--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2

Mobile : +94777568639




--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2

Mobile : +94777568639



--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2

Mobile : +94777568639



--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos



--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos



--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2

Mobile : +94777568639



--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2

Mobile : +94777568639



--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2

Mobile : +94777568639



--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos



--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2

Mobile : +94777568639



--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2

Mobile : +94777568639



--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos