Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Message-ID: <55C1C38F.9090901@widodh.nl>
Date: Wed, 05 Aug 2015 10:04:31 +0200
From: Wido den Hollander <wido@widodh.nl>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:31.0) Gecko/20100101 Thunderbird/31.8.0
MIME-Version: 1.0
To: dev@cloudstack.apache.org
Subject: MariaDB crash with loading CloudStack database
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Hi,

I've just spent a few hours debugging why this failed:

$ mvn -P developer -pl developer -Ddeploydb

After about 30 seconds MariaDB (5.5 and 10.0) would crash:

[WARNING]
java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
	at java.lang.Thread.run(Thread.java:745)
Caused by: com.cloud.utils.exception.CloudRuntimeException: Unable to
upgrade the database
	at
com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(DatabaseUpgradeChecker.java:369)
	at
com.cloud.upgrade.DatabaseUpgradeChecker.check(DatabaseUpgradeChecker.java:465)
	at com.cloud.upgrade.DatabaseCreator.main(DatabaseCreator.java:217)
	... 6 more
Caused by: com.cloud.utils.exception.CloudRuntimeException: Unable to
execute upgrade script:
/home/wido/repos/cloudstack/developer/target/db/db/schema-421to430.sql
	at
com.cloud.upgrade.DatabaseUpgradeChecker.runScript(DatabaseUpgradeChecker.java:281)
	at
com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(DatabaseUpgradeChecker.java:338)
	... 8 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
Communications link failure

The last packet successfully received from the server was 27
milliseconds ago.  The last packet sent successfully to the server was 1
milliseconds ago.

After debugging I found that these two queries cause MariaDB to crash:

UPDATE `cloud`.`configuration` SET value = CONCAT("*.",(SELECT
`temptable`.`value` FROM (SELECT * FROM `cloud`.`configuration` WHERE
`name`="consoleproxy.url.domain") AS `temptable` WHERE
`temptable`.`name`="consoleproxy.url.domain")) WHERE
`name`="consoleproxy.url.domain";

UPDATE `cloud`.`configuration` SET `value` = CONCAT("*.",(SELECT
`temptable`.`value` FROM (SELECT * FROM `cloud`.`configuration` WHERE
`name`="secstorage.ssl.cert.domain") AS `temptable` WHERE
`temptable`.`name`="secstorage.ssl.cert.domain")) WHERE
`name`="secstorage.ssl.cert.domain";

After I disabled these queries my data loads fine.


150805  9:49:04 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see http://kb.askmonty.org/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.

Server version: 10.0.20-MariaDB-1~trusty-log


I'm not sure yet what this is, but should anybody run into it, disable
those queries.

I was running MariaDB on Ubuntu 14.04 in this case. (on my desktop).

Wido