cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlos Reátegui <create...@gmail.com>
Subject Re: 4.4 upgrade issues
Date Thu, 03 Jul 2014 05:39:54 GMT
An update on this.

I finally succeeded in getting my hosts out of alert state by reverting to an earlier version
of the kernel (see other thread).  Unfortunately when it came up I realized I had installed
the 4.3 systemvm template and not the 4.4 one so I just reverted to my 4.1.1 installation
and did the upgrade to 4.3 which appears to work.  I am seeing some errors in the log which
I’ll post to a separate thread.  I’ll wait to test 4.4 when it is released.

With regards to the db.properties file.  I looked into it in more detail and the problem was
that the file had somehow gotten re-ordered in a random fashion which is why I could not make
sense out of the diff during the install.  Not sure how that happened but my concern mentioned
below is probably a non-issue.

regards,
-Carlos


On Jun 30, 2014, at 5:11 PM, Carlos Reátegui <creategui@gmail.com> wrote:

> I set encryption to none in db.properties and updated the passwords in host_details to
unencrypted versions so I could make progress. 
> 
> I don’t know what exactly the problem was but this is probably something that needs
better testing.  I’m pretty sure I had all the encryption stuff correct in the db.properties
file but could not get it to work.
> 
> It would be nice if there was a specialized merging utility for the db.properties given
the change in the organization of the file.  I am guessing if the file had not been reorganized
it would have been more obvious how to merge the 2 and I may have avoided this issue. 
> 
> Now my hosts come up in an alert state and I’m not sure where to go from here.  Please
note I am not using bridge mode because I wanted to to a 4 nic bridge which bridge does not
allow (only 2 nics).  This was working fine in 4.1 so hopefully this is not a requirement
for 4.4.  I am not using security groups which was my understanding is what requires bridge
networking:
> 
> The error in the log is this:
> 2014-06-30 14:06:50,073 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-1:ctx-35941dc7)
Failed to configure brige firewall
> 2014-06-30 14:06:50,073 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-1:ctx-35941dc7)
Check host 172.30.45.32 for CSP is installed or not and check network mode for bridge
> 2014-06-30 14:06:50,074 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-1:ctx-35941dc7)
Seq 2-6232418934327345153: Response Received: 
> 2014-06-30 14:06:50,075 DEBUG [c.c.a.t.Request] (DirectAgent-1:ctx-35941dc7) Seq 2-6232418934327345153:
Processing:  { Ans: , MgmtId: 233845174730253, via: 2, Ver: v1, Flags: 110, [{"com.cloud.agent.api
> .SetupAnswer":{"_reconnect":true,"result":false,"details":"Failed to configure brige
firewall","wait":0}}] }
> 2014-06-30 14:06:50,075 DEBUG [c.c.a.t.Request] (AgentTaskPool-2:ctx-b360d1bb) Seq 2-6232418934327345153:
Received:  { Ans: , MgmtId: 233845174730253, via: 2, Ver: v1, Flags: 110, { SetupAnswer }
}
> 2014-06-30 14:06:50,076 DEBUG [c.c.a.m.AgentAttache] (DirectAgent-1:ctx-35941dc7) Seq
2-6232418934327345153: No more commands found
> 2014-06-30 14:06:50,076 WARN  [c.c.h.x.d.XcpServerDiscoverer] (AgentTaskPool-2:ctx-b360d1bb)
Unable to setup agent 2 due to Failed to configure brige firewall
> 2014-06-30 14:06:50,079 INFO  [c.c.u.e.CSExceptionErrorCode] (AgentTaskPool-2:ctx-b360d1bb)
Could not find exception: com.cloud.exception.ConnectionException in error code list for exceptions
> 2014-06-30 14:06:50,079 WARN  [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Monitor XcpServerDiscoverer says there is an error in the connect process for 2 due to Reinitialize
agent after se
> tup.
> 2014-06-30 14:06:50,079 INFO  [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Host 2 is disconnecting with event AgentDisconnected
> 2014-06-30 14:06:50,085 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
The next status of agent 2is Alert, current status is Connecting
> 2014-06-30 14:06:50,085 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Deregistering link for 2 with state Alert
> 2014-06-30 14:06:50,085 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Remove Agent : 2
> 2014-06-30 14:06:50,085 DEBUG [c.c.a.m.DirectAgentAttache] (AgentTaskPool-2:ctx-b360d1bb)
Processing disconnect 2(srvengxen02)
> 2014-06-30 14:06:50,085 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer
> 2014-06-30 14:06:50,085 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.hypervisor.hyperv.discoverer.HypervServerDiscoverer
> 2014-06-30 14:06:50,085 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: org.apache.cloudstack.engine.orchestration.NetworkOrchestrator
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.network.security.SecurityGroupListener
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.vm.ClusteredVirtualMachineManagerImpl
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.storage.secondary.SecondaryStorageListener
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.deploy.DeploymentPlanningManagerImpl
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.storage.listener.StoragePoolMonitor
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.storage.download.DownloadListener
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.network.SshKeysDistriMonitor
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.network.router.VirtualNetworkApplianceManagerImpl
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.consoleproxy.ConsoleProxyListener
> 2014-06-30 14:06:50,091 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.network.SshKeysDistriMonitor
> 2014-06-30 14:06:50,091 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl
> 2014-06-30 14:06:50,091 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.storage.LocalStoragePoolListener
> 2014-06-30 14:06:50,091 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.storage.upload.UploadListener
> 2014-06-30 14:06:50,093 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.capacity.StorageCapacityListener
> 2014-06-30 14:06:50,093 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.capacity.ComputeCapacityListener
> 2014-06-30 14:06:50,093 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.network.NetworkUsageManagerImpl$DirectNetworkStatsListener
> 2014-06-30 14:06:50,093 DEBUG [c.c.n.NetworkUsageManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Disconnected called on 2 with status Alert
> 2014-06-30 14:06:50,093 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Sending Disconnect to listener: com.cloud.agent.manager.AgentManagerImpl$BehindOnPingListener
> 2014-06-30 14:06:50,093 DEBUG [c.c.h.Status] (AgentTaskPool-2:ctx-b360d1bb) Transition:[Resource
state = Enabled, Agent event = AgentDisconnected, Host id = 2, name = srvengxen02]
> 2014-06-30 14:06:50,102 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Notifying other nodes of to disconnect
> 2014-06-30 14:06:50,108 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-2:ctx-59808af2)
Copying /usr/share/cloudstack-management/webapps/client/WEB-INF/classes/scripts/vm/hypervisor/xenserver/xenserver60/../../../../network/domr//router_proxy.sh
to /opt/cloud/bin on 172.30.45.32 with permission 0755
> 2014-06-30 14:06:50,108 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-2:ctx-59808af2)
Unable to create destination path: /opt/cloud/bin on 172.30.45.32 but trying anyway
> 2014-06-30 14:06:50,110 WARN  [c.c.r.ResourceManagerImpl] (AgentTaskPool-2:ctx-b360d1bb)
Unable to connect due to 
> com.cloud.exception.ConnectionException: Reinitialize agent after setup.
>        at com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer.processConnect(XcpServerDiscoverer.java:656)
>        at com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:514)
>        at com.cloud.agent.manager.AgentManagerImpl.handleDirectConnectAgent(AgentManagerImpl.java:1427)
>        at com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1765)
>        at com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1891)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>        at java.lang.reflect.Method.invoke(Method.java:606)
>        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> 
> 
> 
> 
> On Jun 30, 2014, at 1:57 PM, Carlos Reátegui <creategui@gmail.com> wrote:
> 
>> Making a little progress but still stuck…
>> 
>> I realized that when I did the upgrade it had asked me if to keep the old dp.properties
or use the new one.  The structure of the file seemed different enough and I did not recall
using anything but the defaults so I went ahead and told it to use the new one.  Seems this
was not the right thing to do.
>> 
>> I have updated the password/ecryption settings to match the old file but it is still
not working.  Now I am getting stuck here:
>> 
>> 2014-06-30 13:50:32,139 DEBUG [c.c.s.d.VMTemplateDaoImpl] (main:null) Found parameter
routing unique name null
>> 2014-06-30 13:50:32,139 DEBUG [c.c.s.d.VMTemplateDaoImpl] (main:null) Use console
proxy template : routing
>> 2014-06-30 13:50:32,143 INFO  [c.c.u.d.GenericDaoBase] (main:null) Cache created:
[ name = HostPodDaoImpl status = STATUS_ALIVE eternal = false overflowToDisk = false maxEntriesLocalHeap
= 50 maxEntriesLocalDisk = 0 memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 600 timeToIdleSeconds
= 300 persistence = none diskExpiryThreadIntervalSeconds = 120 cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper
 hitCount = 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0 missCountExpired
= 0 maxBytesLocalHeap = 0 overflowToOffHeap = false maxBytesLocalOffHeap = 0 maxBytesLocalDisk
= 0 pinned = false ]
>> 2014-06-30 13:50:32,157 INFO  [c.c.u.d.GenericDaoBase] (main:null) Cache created:
[ name = DedicatedResourceDaoImpl status = STATUS_ALIVE eternal = false overflowToDisk = false
maxEntriesLocalHeap = 30 maxEntriesLocalDisk = 0 memoryStoreEvictionPolicy = LRU timeToLiveSeconds
= 3600 timeToIdleSeconds = 300 persistence = none diskExpiryThreadIntervalSeconds = 120 cacheEventListeners:
net.sf.ehcache.statistics.LiveCacheStatisticsWrapper  hitCount = 0 memoryStoreHitCount = 0
diskStoreHitCount = 0 missCountNotFound = 0 missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap
= false maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = false ]
>> 2014-06-30 13:50:32,168 INFO  [c.c.u.d.GenericDaoBase] (main:null) Cache created:
[ name = HypervisorCapabilitiesDaoImpl status = STATUS_ALIVE eternal = false overflowToDisk
= false maxEntriesLocalHeap = 100 maxEntriesLocalDisk = 0 memoryStoreEvictionPolicy = LRU
timeToLiveSeconds = 600 timeToIdleSeconds = 300 persistence = none diskExpiryThreadIntervalSeconds
= 120 cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper  hitCount
= 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0 missCountExpired =
0 maxBytesLocalHeap = 0 overflowToOffHeap = false maxBytesLocalOffHeap = 0 maxBytesLocalDisk
= 0 pinned = false ]
>> 2014-06-30 13:50:32,175 INFO  [c.c.u.d.GenericDaoBase] (main:null) Cache created:
[ name = UserDaoImpl status = STATUS_ALIVE eternal = false overflowToDisk = false maxEntriesLocalHeap
= 5000 maxEntriesLocalDisk = 0 memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 300 timeToIdleSeconds
= 300 persistence = none diskExpiryThreadIntervalSeconds = 120 cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper
 hitCount = 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0 missCountExpired
= 0 maxBytesLocalHeap = 0 overflowToOffHeap = false maxBytesLocalOffHeap = 0 maxBytesLocalDisk
= 0 pinned = false ]
>> 2014-06-30 13:50:32,180 INFO  [c.c.u.d.GenericDaoBase] (main:null) Cache created:
[ name = ServiceOfferingDaoImpl status = STATUS_ALIVE eternal = false overflowToDisk = false
maxEntriesLocalHeap = 50 maxEntriesLocalDisk = 0 memoryStoreEvictionPolicy = LRU timeToLiveSeconds
= 600 timeToIdleSeconds = 300 persistence = none diskExpiryThreadIntervalSeconds = 120 cacheEventListeners:
net.sf.ehcache.statistics.LiveCacheStatisticsWrapper  hitCount = 0 memoryStoreHitCount = 0
diskStoreHitCount = 0 missCountNotFound = 0 missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap
= false maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = false ]
>> 2014-06-30 13:50:32,187 INFO  [c.c.u.d.GenericDaoBase] (main:null) Cache created:
[ name = DataCenterDaoImpl status = STATUS_ALIVE eternal = false overflowToDisk = false maxEntriesLocalHeap
= 50 maxEntriesLocalDisk = 0 memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 600 timeToIdleSeconds
= 300 persistence = none diskExpiryThreadIntervalSeconds = 120 cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper
 hitCount = 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0 missCountExpired
= 0 maxBytesLocalHeap = 0 overflowToOffHeap = false maxBytesLocalOffHeap = 0 maxBytesLocalDisk
= 0 pinned = false ]
>> 2014-06-30 13:50:32,188 INFO  [c.c.u.d.GenericDaoBase] (main:null) Cache created:
[ name = Ip Alloc status = STATUS_ALIVE eternal = false overflowToDisk = false maxEntriesLocalHeap
= 50 maxEntriesLocalDisk = 0 memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 600 timeToIdleSeconds
= 300 persistence = none diskExpiryThreadIntervalSeconds = 120 cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper
 hitCount = 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0 missCountExpired
= 0 maxBytesLocalHeap = 0 overflowToOffHeap = false maxBytesLocalOffHeap = 0 maxBytesLocalDisk
= 0 pinned = false ]
>> 2014-06-30 13:50:32,189 INFO  [c.c.u.d.GenericDaoBase] (main:null) Cache created:
[ name = vnet Alloc status = STATUS_ALIVE eternal = false overflowToDisk = false maxEntriesLocalHeap
= 50 maxEntriesLocalDisk = 0 memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 600 timeToIdleSeconds
= 300 persistence = none diskExpiryThreadIntervalSeconds = 120 cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper
 hitCount = 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0 missCountExpired
= 0 maxBytesLocalHeap = 0 overflowToOffHeap = false maxBytesLocalOffHeap = 0 maxBytesLocalDisk
= 0 pinned = false ]
>> 2014-06-30 13:50:32,198 INFO  [c.c.u.d.GenericDaoBase] (main:null) Cache created:
[ name = VlanDaoImpl status = STATUS_ALIVE eternal = false overflowToDisk = false maxEntriesLocalHeap
= 30 maxEntriesLocalDisk = 0 memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 3600 timeToIdleSeconds
= 300 persistence = none diskExpiryThreadIntervalSeconds = 120 cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper
 hitCount = 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0 missCountExpired
= 0 maxBytesLocalHeap = 0 overflowToOffHeap = false maxBytesLocalOffHeap = 0 maxBytesLocalDisk
= 0 pinned = false ]
>> 2014-06-30 13:50:32,232 DEBUG [c.c.u.c.DBEncryptionUtil] (main:null) Error while
decrypting: true
>> 
>> The key is still the default password and I have decrypted all the ENC parameters
from the db.properties file and they seem ok.  What am I missing?
>> 
>> thanks,
>> Carlos
>> 
>> 
>> On Jun 30, 2014, at 1:16 PM, Carlos Reátegui <creategui@gmail.com> wrote:
>> 
>>> I found the comments in: https://issues.apache.org/jira/browse/CLOUDSTACK-3990
useful but how do I find out the database key so that I can set the pw.
>>> 
>>> Also in looking at my previous backups for the host_details table it seems like
the password entry changes on a regular basis.
>>> 
>>> Is there something the keeps updating the db key and re-ecrypts the host passwords?
>>> 
>>> On Jun 30, 2014, at 1:01 PM, Carlos Reátegui <creategui@gmail.com> wrote:
>>> 
>>>> Hi All,
>>>> 
>>>> I am having problems bringing my system back up.  I have not checked the
credentials of my hosts but the upgraded management server is unable to connect to them. 
Where is the password stored?
>>>> 
>>>> thanks.
>>>> Carlos
>>>> 
>>>> 
>>>> 2014-06-30 12:55:59,277 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (ClusteredAgentManager
Timer:ctx-060c8ace) Loading directly connected host 1(srvengxen01)
>>>> 2014-06-30 12:56:04,394 DEBUG [c.c.n.l.LBHealthCheckManagerImpl] (LBHealthCheck-1:ctx-c6869648)
LB HealthCheck Manager is running and getting the updates from LB providers and updating service
status
>>>> 2014-06-30 12:56:04,428 DEBUG [c.c.n.l.LBHealthCheckManagerImpl] (LBHealthCheck-1:ctx-c6869648)
LB HealthCheck Manager is running and getting the updates from LB providers and updating service
status
>>>> 2014-06-30 12:56:06,844 DEBUG [c.c.h.x.r.XenServerConnectionPool] (ClusteredAgentManager
Timer:ctx-060c8ace) Unable to create master connection to host(172.30.45.31) , due to The
credentials given by the user are incorrect, so access has been denied, and you have not been
issued a session handle.
>>>> 2014-06-30 12:56:06,848 DEBUG [c.c.h.Status] (ClusteredAgentManager Timer:ctx-060c8ace)
Transition:[Resource state = Enabled, Agent event = AgentDisconnected, Host id = 1, name =
srvengxen01]
>>>> 2014-06-30 12:56:06,862 WARN  [c.c.a.m.ClusteredAgentManagerImpl] (ClusteredAgentManager
Timer:ctx-060c8ace)  can not load directly connected host 1(srvengxen01) due to 
>>>> com.cloud.utils.exception.CloudRuntimeException: Unable to create master
connection to host(172.30.45.31) , due to The credentials given by the user are incorrect,
so access has been denied, and you have not been issued a session handle.
>>>>     at com.cloud.hypervisor.xen.resource.XenServerConnectionPool.getConnect(XenServerConnectionPool.java:168)
>>>>     at com.cloud.hypervisor.xen.resource.CitrixResourceBase.CheckXenHostInfo(CitrixResourceBase.java:5722)
>>>>     at com.cloud.hypervisor.xen.resource.CitrixResourceBase.configure(CitrixResourceBase.java:5705)
>>>>     at com.cloud.resource.DiscovererBase.reloadResource(DiscovererBase.java:157)
>>>>     at com.cloud.agent.manager.AgentManagerImpl.loadDirectlyConnectedHost(AgentManagerImpl.java:672)
>>>>     at com.cloud.agent.manager.ClusteredAgentManagerImpl.scanDirectAgentToLoad(ClusteredAgentManagerImpl.java:218)
>>>>     at com.cloud.agent.manager.ClusteredAgentManagerImpl.runDirectAgentScanTimerTask(ClusteredAgentManagerImpl.java:184)
>>>>     at com.cloud.agent.manager.ClusteredAgentManagerImpl.access$100(ClusteredAgentManagerImpl.java:98)
>>>>     at com.cloud.agent.manager.ClusteredAgentManagerImpl$DirectAgentScanTimerTask.runInContext(ClusteredAgentManagerImpl.java:234)
>>>>     at org.apache.cloudstack.managed.context.ManagedContextTimerTask$1.runInContext(ManagedContextTimerTask.java:30)
>>>>     at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>>>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>>>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>>>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>>>     at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>>>     at org.apache.cloudstack.managed.context.ManagedContextTimerTask.run(ManagedContextTimerTask.java:27)
>>>>     at java.util.TimerThread.mainLoop(Timer.java:555)
>>>>     at java.util.TimerThread.run(Timer.java:505)
>>>> Caused by: The credentials given by the user are incorrect, so access has
been denied, and you have not been issued a session handle.
>>>>     at com.xensource.xenapi.Types.checkResponse(Types.java:322)
>>>>     at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
>>>>     at com.xensource.xenapi.Session.loginWithPassword(Session.java:537)
>>>>     at com.cloud.hypervisor.xen.resource.XenServerConnectionPool.loginWithPassword(XenServerConnectionPool.java:321)
>>>>     at com.cloud.hypervisor.xen.resource.XenServerConnectionPool.getConnect(XenServerConnectionPool.java:154)
>>>>     ... 17 more
>>>> 2014-06-30 12:56:06,864 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (ClusteredAgentManager
Timer:ctx-060c8ace) Loading directly connected host 2(srvengxen02)
>>>> 2014-06-30 12:56:09,225 DEBUG [c.c.s.StatsCollector] (StatsCollector-1:ctx-8458e286)
HostStatsCollector is running...
>>>> 2014-06-30 12:56:09,226 DEBUG [c.c.s.StatsCollector] (StatsCollector-2:ctx-aa245eed)
VmStatsCollector is running...
>>>> 2014-06-30 12:56:09,227 DEBUG [c.c.s.StatsCollector] (StatsCollector-3:ctx-19894fa1)
StorageCollector is running...
>>>> 2014-06-30 12:56:09,230 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-d66c71fb)
AutoScaling Monitor is running...
>>>> 
>>>> 
>>>> 
>>>> On Jun 30, 2014, at 9:54 AM, Carlos Reátegui <creategui@gmail.com>
wrote:
>>>> 
>>>>> Hi Sudha,
>>>>> Thanks for checking in.  I was out for the weekend and just getting back
to this now.
>>>>> 
>>>>> My main question at this point is if it is ok for me to kill the system
vms with the xe vm-shutdown command since the script provided by cloudstack does not work
with ubuntu.
>>>>> 
>>>>> Also it would be great if someone could have a look at my logs to see
if they look normal. I am seeing a lot of HA-Worker messages but I do not have an HA deployment
(unless this is the thread that keeps the system vas running).
>>>>> 
>>>>> thanks,
>>>>> Carlos
>>>>> 
>>>>> 
>>>>> 
>>>>> On Jun 29, 2014, at 11:51 PM, Sudha Ponnaganti <sudha.ponnaganti@citrix.com>
wrote:
>>>>> 
>>>>>> Hi Carlos,
>>>>>> 
>>>>>> Were you able to resolve the following? Was your upgrade successful?
>>>>>> 
>>>>>> Thanks
>>>>>> /sudha
>>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Carlos Reátegui [mailto:creategui@gmail.com] 
>>>>>> Sent: Friday, June 27, 2014 8:55 PM
>>>>>> To: CloudStack-Users
>>>>>> Cc: dev@cloudstack.apache.org
>>>>>> Subject: 4.4 upgrade issues
>>>>>> 
>>>>>> I am trying out the upgrade instructions from http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.3/rnotes.html#upgrade-from-4-1-x-to-4-3
but going to 4.4 built from source today.
>>>>>> 
>>>>>> My setup: XenServer 6.0.2 Hosts, Management Server on Ubuntu 12.04,
Primary and Secondary on NFS, Basic Network, no security groups
>>>>>> 
>>>>>> -----
>>>>>> Notes on the docs:
>>>>>> 
>>>>>> 8.4 - 8.6: This is only for hosts that use the cloudstack agent.
Does not apply to KVM. In general this whole section does not do a good job of explaining
what is on the MS vs the Hosts.
>>>>>> 
>>>>>> 13: This fails on ubuntu because: cloudstack-sysvmadm sources /etc/rc.d/init.d/functions
which does not exist on ubuntu/debian systems.
>>>>>> 
>>>>>> 14: Copy vhf-util from where? Also the path /usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver
does not exist on the hosts so I am assuming this is on the MS, however the MS already has
it since it is an upgrade and was put there by the original install.  Or is this a new version
that needs to be grabbed from somewhere?
>>>>>> 
>>>>>> Other: earlier versions like 4.1 worked with JDK 1.6 current releases
require 1.7 but the Upgrade doc does not mention that.
>>>>>> 
>>>>>> --
>>>>>> Issues:
>>>>>> 
>>>>>> Saw the following in catalina.out, not sure if it is an issues:
>>>>>> Jun 27, 2014 5:28:42 PM org.apache.catalina.loader.WebappClassLoader
validateJarFile
>>>>>> INFO: validateJarFile(/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/servlet-api-2.5-20081211.jar)
- jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/servlet/Servlet.class
Jun 27, 2014 5:28:42 PM org.apache.catalina.loader.WebappClassLoader validateJarFile
>>>>>> INFO: validateJarFile(/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/tomcat-embed-core-7.0.30.jar)
- jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/servlet/Servlet.class
>>>>>> 
>>>>>> Since the above script in step 13 did not work is it ok to do "xe
vm-shutdown vm=." on each of the system vms?  Will CloudStack notice they are ton and start
new ones?
>>>>>> 
>>>>>> Here are my log files (please note I stopped the service prior to
capturing these logs in case you are wondering):
>>>>>> Management server log: https://www.dropbox.com/s/7xhkutt8e724il1/management-server.log
>>>>>> Catalina log: https://www.dropbox.com/s/f45ypkbazhkogyj/catalina.2014-06-27.log
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 
> 


Mime
View raw message