Mailing-List: contact cloudstack-users-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cloudstack-users@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of alexeyzilber@gmail.com
 designates 209.85.213.175 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <67EF18FDCA335F489B366120481AB6C5EE3AC0D49E@BANPMAILBOX01.citrite.net>
References: 
 <CAGdvdE099FHc+88e=1akWauGFz7ter5xARgaeuP5rzrdPovVTA@mail.gmail.com>
 <67EF18FDCA335F489B366120481AB6C5EE3AC0D49E@BANPMAILBOX01.citrite.net>
From: Alexey Zilber <alexeyzilber@gmail.com>
Date: Sat, 23 Jun 2012 01:14:14 +0800
Message-ID: 
 <CAGdvdE3MhbGyR-9AX7J=n4T38EEYtz5+=M=45-EzPdUgdzuZhQ@mail.gmail.com>
Subject: Re: Cloudstack agent keeps rebooting kvm host..
To: cloudstack-users@incubator.apache.org
Content-Type: multipart/alternative; boundary=20cf302d4e2a251ca004c312c4ba

--20cf302d4e2a251ca004c312c4ba
Content-Type: text/plain; charset=ISO-8859-1

Is there no way to disable HA checks?  I don't understand why it has to do
HA checks when there's only one host in the cluster.   It seems like a bug
to have HA on with a single server.

-Alex

On Sat, Jun 23, 2012 at 12:55 AM, Suresh Sadhu <Suresh.Sadhu@citrix.com>wrote:

> HI Alex,
>
> When heartbeat fails ,host will reboot continuously till the problem
> resolved(heartbeat successful)...
>
> The heartbeat failure might be caused due to fail to  write on mounted
> storage path,
> Did you see any permission denied messages in the logs ..and does your
>  mounted storage paths has rw permissions after this problem. because due
> some corruption in the mounted FS your  mounted file system might become
> read-only. That might cause  heart-beat failure.
>
>
>
> Regards
> Sadhu
>
>
> -----Original Message-----
> From: Alexey Zilber [mailto:alexeyzilber@gmail.com]
> Sent: 22 June 2012 21:52
> To: cloudstack-users@incubator.apache.org
> Subject: Cloudstack agent keeps rebooting kvm host..
>
> Hi,
>
>  The saga continues!  I added a KVM host.  The agent decided it wants to
> constantly reboot the server:
>
> 2012-06-23 00:11:32,083{GMT} INFO  [cloud.agent.Agent] (Agent-Handler-2:)
> Startup Response Received: agent id = 5
> 2012-06-23 00:11:32,083 INFO  [cloud.agent.Agent] (Agent-Handler-2:null)
> Startup Response Received: agent id = 5
> 2012-06-23 00:12:30,187{GMT} WARN  [resource.computing.KVMHAMonitor]
> (Thread-7:) write heartbeat failed: Failed to create
> /mnt/9c2be815-de2b-3c14-84bb-54025d782794/KVMHA//hb-10.1.1.17, retry: 0
> 2012-06-23 00:12:30,187 WARN  [resource.computing.KVMHAMonitor]
> (Thread-7:null) write heartbeat failed: Failed to create
> /mnt/9c2be815-de2b-3c14-84bb-54025d782794/KVMHA//hb-10.1.1.17, retry: 0
> 2012-06-23 00:12:30,209{GMT} WARN  [resource.computing.KVMHAMonitor]
> (Thread-7:) write heartbeat failed: Failed to create
> /mnt/9c2be815-de2b-3c14-84bb-54025d782794/KVMHA//hb-10.1.1.17, retry: 1
> 2012-06-23 00:12:30,209 WARN  [resource.computing.KVMHAMonitor]
> (Thread-7:null) write heartbeat failed: Failed to create
> /mnt/9c2be815-de2b-3c14-84bb-54025d782794/KVMHA//hb-10.1.1.17, retry: 1
> 2012-06-23 00:12:30,232{GMT} WARN  [resource.computing.KVMHAMonitor]
> (Thread-7:) write heartbeat failed: Failed to create
> /mnt/9c2be815-de2b-3c14-84bb-54025d782794/KVMHA//hb-10.1.1.17, retry: 2
> 2012-06-23 00:12:30,232 WARN  [resource.computing.KVMHAMonitor]
> (Thread-7:null) write heartbeat failed: Failed to create
> /mnt/9c2be815-de2b-3c14-84bb-54025d782794/KVMHA//hb-10.1.1.17, retry: 2
> 2012-06-23 00:12:30,254{GMT} WARN  [resource.computing.KVMHAMonitor]
> (Thread-7:) write heartbeat failed: Failed to create
> /mnt/9c2be815-de2b-3c14-84bb-54025d782794/KVMHA//hb-10.1.1.17, retry: 3
> 2012-06-23 00:12:30,254 WARN  [resource.computing.KVMHAMonitor]
> (Thread-7:null) write heartbeat failed: Failed to create
> /mnt/9c2be815-de2b-3c14-84bb-54025d782794/KVMHA//hb-10.1.1.17, retry: 3
> 2012-06-23 00:12:30,275{GMT} WARN  [resource.computing.KVMHAMonitor]
> (Thread-7:) write heartbeat failed: Failed to create
> /mnt/9c2be815-de2b-3c14-84bb-54025d782794/KVMHA//hb-10.1.1.17, retry: 4
> 2012-06-23 00:12:30,275 WARN  [resource.computing.KVMHAMonitor]
> (Thread-7:null) write heartbeat failed: Failed to create
> /mnt/9c2be815-de2b-3c14-84bb-54025d782794/KVMHA//hb-10.1.1.17, retry: 4
> 2012-06-23 00:12:30,275{GMT} WARN  [resource.computing.KVMHAMonitor]
> (Thread-7:) write heartbeat failed: Failed to create
> /mnt/9c2be815-de2b-3c14-84bb-54025d782794/KVMHA//hb-10.1.1.17; reboot the
> host
> 2012-06-23 00:12:30,275 WARN  [resource.computing.KVMHAMonitor]
> (Thread-7:null) write heartbeat failed: Failed to create
> /mnt/9c2be815-de2b-3c14-84bb-54025d782794/KVMHA//hb-10.1.1.17; reboot the
> host
>
> Broadcast message from root@kvm1.xxxxx.xxxx
>        (unknown) at 0:12 ...
>
> The system is going down for reboot NOW!
>
> It looks like the agent was in fact, at least able to create the initial
> directory:
>
> [root@kvm1 ~]# ls -al /mnt/9c2be815-de2b-3c14-84bb-54025d782794
> total 8
> drwxrwxrwx  2 root root 4096 Jun 22 23:58 .
> drwxr-xr-x. 4 root root 4096 Jun 22 23:58 ..
>
> Here's the agent properties file:
>
> #Storage
> #Sat Jun 23 00:11:32 MYT 2012
> guest.network.device=cloudbr0
> workers=5
> private.network.device=cloudbr0
> port=8250
> resource=com.cloud.agent.resource.computing.LibvirtComputingResource
> pod=1
> zone=1
> guid=0f0f4f5c-99d0-3813-a7a6-00248cdfd17e
> cluster=2
> public.network.device=cloudbr0
> local.storage.uuid=fbefb2ea-f3e0-4f02-96cb-1b8abb6e8c54
> host=10.1.1.18
> LibvirtComputingResource.id=5
>
>
> First time I'm seeing this error...  Last time my kvm setup went well, but
> KVM was my first hypervisor, now it's the second.
>
> Thanks!
> Alex
>

--20cf302d4e2a251ca004c312c4ba--