cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cloud List <cloud-l...@sg.or.id>
Subject ERROR: Java process not running. Try restarting the SSVM.
Date Sat, 18 Jun 2016 02:27:41 GMT
Dear all,

I am running ACS 4.2.0 in our test environment and after restarting the KVM
host, I am not able to get SSVM agent up and running, despite destroying
and recreating it.

The problem is similar to what Mike Tutkowsi reported on below thread, same
symptoms but different cloud.out stack error message:

http://comments.gmane.org/gmane.comp.apache.cloudstack.devel/28827

I can SSH into SSVM but unable to start the Java agent. SSVM health check
result shows two error:
- Unable to connect NFS (wrong IP detected); and
- Unable to start Java.

Unable to connect NFS is because wrong IP is detected for the NFS ip. I can
manually mount the NFS server IP using correct IP.

Result:

====
root@s-121-VM:/var/log/cloud# /usr/local/cloud/systemvm/ssvm-check.sh
================================================
First DNS server is  8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=49 time=4.572 ms
64 bytes from 8.8.8.8: icmp_seq=1 ttl=49 time=4.365 ms
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 4.365/4.468/4.572/0.104 ms
Good: Can ping DNS server
================================================
Good: DNS resolves download.cloud.com
================================================
ERROR: NFS is not currently mounted
Try manually mounting from inside the VM
NFS server is  169.254.0.127
PING 169.254.0.127 (169.254.0.127): 56 data bytes
64 bytes from 169.254.0.127: icmp_seq=0 ttl=64 time=0.106 ms
64 bytes from 169.254.0.127: icmp_seq=1 ttl=64 time=0.040 ms
--- 169.254.0.127 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.040/0.073/0.106/0.033 ms
Good: Can ping NFS server
================================================
Management server is 192.168.0.222. Checking connectivity.
Good: Can connect to management server port 8250
================================================
ERROR: Java process not running.  Try restarting the SSVM.
====

Repeated error shown on /var/log/cloud/cloud.out:

====
+ java -Djavax.net.ssl.trustStore=./certs/realhostip.keystore
-Dlog.home=/var/log/cloud/ -mx-24m -cp
./:./conf:activation-1.1.jar:antisamy-1.4.3.jar:aopalliance-1.0.jar:apache-log4j-extras-1.1.jar:aspectjrt-1.7.1.jar:aspectjweaver-1.7.1.jar:aws-java-sdk-1.3.22.jar:backport-util-concurrent-3.1.jar:batik-css-1.7.jar:batik-ext-1.7.jar:batik-util-1.7.jar:bcprov-jdk16-1.46.jar:bsh-core-2.0b4.jar:c3p0-0.9.1.1.jar:cglib-nodep-2.2.2.jar:cloud-agent-4.2.0.jar:cloud-api-4.2.0.jar:cloud-console-proxy-4.2.0.jar:cloud-core-4.2.0.jar:cloud-engine-api-4.2.0.jar:cloud-engine-components-api-4.2.0.jar:cloud-engine-schema-4.2.0.jar:cloud-framework-events-4.2.0.jar:cloud-framework-ipc-4.2.0.jar:cloud-framework-jobs-4.2.0.jar:cloud-framework-rest-4.2.0.jar:cloud-secondary-storage-4.2.0.jar:cloud-server-4.2.0.jar:cloud-utils-4.2.0.jar:commons-beanutils-core-1.7.0.jar:commons-codec-1.6.jar:commons-collections-3.2.1.jar:commons-configuration-1.8.jar:commons-daemon-1.0.10.jar:commons-dbcp-1.4.jar:commons-discovery-0.5.jar:commons-fileupload-1.2.jar:commons-httpclient-3.1.jar:commons-io-1.4.jar:commons-lang-2.6.jar:commons-logging-1.1.1.jar:commons-pool-1.6.jar:cxf-bundle-jaxrs-2.7.0.jar:dom4j-1.6.1.jar:ehcache-1.5.0.jar:ejb-api-3.0.jar:esapi-2.0.1.jar:geronimo-javamail_1.4_spec-1.7.1.jar:geronimo-servlet_3.0_spec-1.0.jar:gson-1.7.1.jar:guava-11.0.2.jar:httpclient-4.2.1.jar:httpcore-4.2.1.jar:jackson-annotations-2.1.1.jar:jackson-core-2.1.1.jar:jackson-core-asl-1.8.9.jar:jackson-databind-2.1.1.jar:jackson-jaxrs-json-provider-2.1.1.jar:jackson-mapper-asl-1.8.9.jar:jackson-module-jaxb-annotations-2.1.1.jar:jasypt-1.9.0.jar:java-ipv6-0.10.jar:javassist-3.12.1.GA.jar:javax.inject-1.jar:javax.persistence-2.0.0.jar:javax.ws.rs-api-2.0-m10.jar:jaxen-1.1-beta-8.jar:jdom-1.0.jar:jetty-continuation-8.1.7.v20120910.jar:jetty-http-8.1.7.v20120910.jar:jetty-io-8.1.7.v20120910.jar:jetty-security-8.1.7.v20120910.jar:jetty-util-8.1.7.v20120910.jar:jsch-0.1.42.jar:jsr107cache-1.0.jar:jsr305-1.3.9.jar:jstl-1.2.jar:log4j-1.2.16.jar:mail-1.4.jar:nekohtml-1.9.12.jar:quartz-2.1.6.jar:reflections-0.9.8.jar:slf4j-api-1.6.2.jar:spring-aop-3.1.2.RELEASE.jar:spring-asm-3.1.2.RELEASE.jar:spring-beans-3.1.2.RELEASE.jar:spring-context-3.1.2.RELEASE.jar:spring-core-3.1.2.RELEASE.jar:spring-expression-3.1.2.RELEASE.jar:spring-web-3.1.2.RELEASE.jar:stax2-api-3.1.1.jar:trilead-ssh2-build213-svnkit-1.3-patch.jar:woodstox-core-asl-4.1.4.jar:wsdl4j-1.6.2.jar:xalan-2.7.0.jar:xercesImpl-2.6.2.jar:xml-apis-1.0.b2.jar:xml-apis-ext-1.3.04.jar:xmlParserAPIs-2.6.2.jar:xmlschema-core-2.0.3.jar:xom-1.1.jar:xpp3_min-1.1.4c.jar:xstream-1.3.1.jar
com.cloud.agent.AgentShell template=domP type=secstorage host=192.168.0.222
port=8250 name=s-121-VM zone=1 pod=1 guid=s-121-VM
resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
instance=SecStorage sslcopy=true role=templateProcessor mtu=1500
eth2ip=192.168.5.170 eth2mask=255.255.255.0 gateway=192.168.5.1
eth0ip=169.254.0.127 eth0mask=255.255.0.0 eth1ip=192.168.0.154
eth1mask=255.255.255.0 mgmtcidr=192.168.0.0/24 localgw=192.168.0.1
private.network.device=eth1 eth3ip=192.168.0.160 eth3mask=255.255.255.0
storageip=192.168.0.160 storagenetmask=255.255.255.0
storagegateway=192.168.0.1 internaldns1=8.8.8.8 dns1=8.8.8.8 dns2=8.8.4.4
Invalid maximum heap size: -Xmx-24m
Could not create the Java virtual machine.
====

I did some research and found that this could be due to the use of mixed
OSS and non-OSS package, but I installed both the management and agent from
the OSS package.

There's also mentioned that it could be due to systemvm.iso file, I have
tried to re-register the system VM template using cloud-install-sys-tmplt,
delete and restore the systemvm.iso file from
/usr/share/cloudstack-common/vms folder, but none helped.

The strange thing is that it was working before, I only restarted the KVM
host and now the problem happened. I even tried to backup the DB and do a
clean reinstallation of ACS 4.2.0 from scratch, and the problem still
persists!

Any pointer is highly appreciated.

Thank you.

-ip-

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message