cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alena Prokharchyk <Alena.Prokharc...@citrix.com>
Subject Re: Latest Master DB issue
Date Tue, 08 Oct 2013 17:47:28 GMT
Ok, this is what going on - the DB upgrade procedure is different on
developer's setup and when deployed using cloudstack-setup-databases

On developers setup:

1) you deploy the code
2) Deploy the DB using 'mvn -P developer -pl developer -Ddeploydb'. As a
part of this step, the DataBaseUpgradeChecker:

* first deploys the base DB version - 4.0.0
* then checks the current version of the code, and performs the db upgrade
if needed. So on master, version table looks like this after the db is
deployed:

mysql> select * from version;
+----+---------+---------------------+----------+
| id | version | updated             | step     |
+----+---------+---------------------+----------+
|  1 | 4.0.0   | 2013-10-08 10:34:47 | Complete |
|  2 | 4.1.0   | 2013-10-08 17:35:22 | Complete |
|  3 | 4.2.0   | 2013-10-08 17:35:22 | Complete |
|  4 | 4.3.0   | 2013-10-08 17:35:22 | Complete |
+----+---------+---------------------+----------+
4 rows in set (0.00 sec)


3) Start management server.


When deployed from rpm:

1) you deploy the code
2) run cloudstack-setup-databases. As the result of this step, 4.0.0 base
version of the DB is deployed. Thats why you see only 4.0.0 record in the
DB.
3) Start management server. DataBaseUpgradeChecker is being invoked as a
part of it, and performs the db upgrade to the version of the current
code. Only after that all the managers get invoked + system caller context
get initialized.

Looks like the load order for step 3) got broken recently, and system
context gets initialized before the db upgrade is finished. So we either
need to fix the order, or invoke DataBaseUpgradeChecker as a part of
cloudstack-setup-databases so at the point when management server starts
up, the DB already upgraded to the latest version.

-Alena.


On 10/8/13 10:25 AM, "Francois Gaudreault" <fgaudreault@cloudops.com>
wrote:

>Hmm... I just checked the DB version and it's 4.0??? It should be 4.3.0
>no?
>
>mysql> select * from version;
>+----+---------+---------------------+----------+
>| id | version | updated             | step     |
>+----+---------+---------------------+----------+
>|  1 | 4.0.0   | 2013-10-08 10:58:49 | Complete |
>+----+---------+---------------------+----------+
>1 row in set (0.00 sec)
>
>I installed cloudstack-management-4.3.0:
>[root@eng-testing-cstack_master ~]# rpm -qf
>/usr/bin/cloudstack-setup-databases
>cloudstack-management-4.3.0-SNAPSHOT.el6.x86_64
>
>Francois
>
>On 10/8/2013, 1:04 PM, Francois Gaudreault wrote:
>> It's a fresh master RPM install.
>>
>> Francois
>>
>> On 10/8/2013, 12:40 PM, Alena Prokharchyk wrote:
>>> It is not a small issue. is_default filed was added to the table as a
>>> part
>>> of the 41-42 db upgrade. Looks like the code tries to retrieve system
>>> user
>>> before the db upgrade is completed.
>>>
>>> DB upgrade is a major part of system integrity check; no queries to
>>> the DB
>>> should be made before its completed. Francois, did you start seeing
>>>this
>>> problem just recently?
>>>
>>> -Alena.
>>>
>>> On 10/8/13 8:04 AM, "Francois Gaudreault" <fgaudreault@cloudops.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I compiled Master this morning, and there is a small DB issue. One
>>>> field
>>>> is missing in the account table (field default). CS will not start
>>>> because of that.
>>>>
>>>> 2013-10-08 11:01:42,623 FATAL [o.a.c.c.CallContext] (Timer-2:null)
>>>> Exiting the system because we're unable to register the system call
>>>> context.
>>>> com.cloud.utils.exception.CloudRuntimeException: DB Exception on:
>>>> com.mysql.jdbc.JDBC4PreparedStatement@4c1aa2e9: SELECT account.id,
>>>> account.account_name, account.type, account.domain_id, account.state,
>>>> account.removed, account.cleanup_needed, account.network_domain,
>>>> account.uuid, account.default_zone_id, account.default FROM account
>>>> WHERE account.id = 1  AND account.removed IS NULL
>>>>      at
>>>> com.cloud.utils.db.GenericDaoBase.findById(GenericDaoBase.java:986)
>>>>      at
>>>> 
>>>>com.cloud.utils.component.ComponentInstantiationPostProcessor$Intercept
>>>>orD 
>>>>
>>>> ispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>>>>      at 
>>>> com.cloud.utils.db.GenericDaoBase.lockRow(GenericDaoBase.java:963)
>>>>      at
>>>> 
>>>>com.cloud.utils.component.ComponentInstantiationPostProcessor$Intercept
>>>>orD 
>>>>
>>>> ispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>>>>      at
>>>> com.cloud.utils.db.GenericDaoBase.findById(GenericDaoBase.java:926)
>>>>      at
>>>> 
>>>>com.cloud.utils.component.ComponentInstantiationPostProcessor$Intercept
>>>>orD 
>>>>
>>>> ispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>>>>      at
>>>> com.cloud.dao.EntityManagerImpl.findById(EntityManagerImpl.java:45)
>>>>      at
>>>> 
>>>>org.apache.cloudstack.context.CallContext.register(CallContext.java:166
>>>>) 
>>>>
>>>>      at
>>>> 
>>>>org.apache.cloudstack.context.CallContext.registerSystemCallContextOnce
>>>>Onl 
>>>>
>>>> y(CallContext.java:141)
>>>>      at
>>>> 
>>>>org.apache.cloudstack.context.CallContextListener.onEnterContext(CallCo
>>>>nte 
>>>>
>>>> xtListener.java:36)
>>>>      at
>>>> 
>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWi
>>>>thC 
>>>>
>>>> ontext(DefaultManagedContext.java:83)
>>>>      at
>>>> 
>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWit
>>>>hCo 
>>>>
>>>> ntext(DefaultManagedContext.java:53)
>>>>      at
>>>> 
>>>>org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manage
>>>>dCo 
>>>>
>>>> ntextRunnable.java:46)
>>>>      at
>>>> 
>>>>org.apache.cloudstack.managed.context.ManagedContextTimerTask.run(Manag
>>>>edC 
>>>>
>>>> ontextTimerTask.java:27)
>>>>      at java.util.TimerThread.mainLoop(Timer.java:534)
>>>>      at java.util.TimerThread.run(Timer.java:484)
>>>> Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException:
>>>> Unknown column 'account.default' in 'field list'
>>>>      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>>>> Method)
>>>>      at
>>>> 
>>>>sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructor
>>>>Acc 
>>>>
>>>> essorImpl.java:57)
>>>>      at
>>>> 
>>>>sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCon
>>>>str 
>>>>
>>>> uctorAccessorImpl.java:45)
>>>>      at 
>>>>java.lang.reflect.Constructor.newInstance(Constructor.java:532)
>>>>      at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
>>>>      at com.mysql.jdbc.Util.getInstance(Util.java:386)
>>>>      at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1053)
>>>>      at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4074)
>>>>      at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4006)
>>>>      at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2468)
>>>>      at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2629)
>>>>      at 
>>>>com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2719)
>>>>      at
>>>> 
>>>>com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java
>>>>:21 
>>>>
>>>> 55)
>>>>      at
>>>> 
>>>>com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:23
>>>>18) 
>>>>
>>>>      at
>>>> 
>>>>org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Delega
>>>>tin 
>>>>
>>>> gPreparedStatement.java:96)
>>>>      at
>>>> 
>>>>org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Delega
>>>>tin 
>>>>
>>>> gPreparedStatement.java:96)
>>>>      at
>>>> com.cloud.utils.db.GenericDaoBase.findById(GenericDaoBase.java:983)
>>>>      ... 27 more
>>>>
>>>>
>>>> -- 
>>>> Francois Gaudreault
>>>> Architecte de Solution Cloud | Cloud Solutions Architect
>>>> fgaudreault@cloudops.com
>>>> 514-629-6775
>>>> - - -
>>>> CloudOps
>>>> 420 rue Guy
>>>> Montréal QC  H3J 1S6
>>>> www.cloudops.com
>>>> @CloudOps_
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>
>
>-- 
>Francois Gaudreault
>Architecte de Solution Cloud | Cloud Solutions Architect
>fgaudreault@cloudops.com
>514-629-6775
>- - -
>CloudOps
>420 rue Guy
>Montréal QC  H3J 1S6
>www.cloudops.com
>@CloudOps_
>
>



Mime
View raw message