Mailing-List: contact users-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@cloudstack.apache.org
MIME-Version: 1.0
In-Reply-To: <BN6PR02MB33312DC6D924A951AA0D1A99A95F0@BN6PR02MB3331.namprd02.prod.outlook.com>
References: <CAAQ6Ra-SBU6fD=iG=tSfHN1W+jgX-wiEMG++LrS+S1mME7zaJA@mail.gmail.com>
 <BN6PR02MB33313031E0383E2F78EC2F26A95A0@BN6PR02MB3331.namprd02.prod.outlook.com>
 <CA+FPaeawUewOJi2fyLXfG4Wb-hPof0nHoKP2E0oPeRi4aVKSiw@mail.gmail.com>
 <00762B56-6DE6-4BBB-A18E-78E5836CEB34@shapeblue.com> <CAAQ6Ra8vVg365CW3XV=UgonQtM1Tktga-A8HC2wY5gRPzVhwww@mail.gmail.com>
 <BN6PR02MB33312DC6D924A951AA0D1A99A95F0@BN6PR02MB3331.namprd02.prod.outlook.com>
From: Ivan Kudryavtsev <kudryavtsev_ia@bw-sw.com>
Date: Wed, 1 Nov 2017 21:29:15 +0700
Message-ID: <CA+FPaea6-Q3GxUQu+4NJFBpVGtEWeRun7F0CcjRMkdnjxLW1fw@mail.gmail.com>
Subject: Re: Problems with KVM HA & STONITH
To: users@cloudstack.apache.org
Content-Type: multipart/alternative; boundary="001a11429b783e6643055cecb25b"
archived-at: Wed, 01 Nov 2017 14:29:24 -0000

--001a11429b783e6643055cecb25b
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Also you can run ceph if you need HA. I met setup description which uses
compute nodes for ceph cluster nodes simultaneously.

1 =D0=BD=D0=BE=D1=8F=D0=B1. 2017 =D0=B3. 21:11 =D0=BF=D0=BE=D0=BB=D1=8C=D0=
=B7=D0=BE=D0=B2=D0=B0=D1=82=D0=B5=D0=BB=D1=8C "Simon Weller" <sweller@ena.c=
om.invalid>
=D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB:

> James,
>
>
> Try just configuring a single NFS server and see if your setup works. If
> you have 3 NFS shares, across all 3 hosts, i'm wondering whether ACS is
> picking the one you rebooted as the storage for your VMs and when that
> storage goes away (when you bounce the host), all storage for your VMs
> vanishes and ACS tries to reboot your other hosts.
>
>
> Normally in a simple ACS setup, you would have a separate storage server
> that can serve up NFS to all hosts. If a host dies, then a VM would be
> brought up on a spare hosts since all hosts have access to the same stora=
ge.
>
> Your other option is to use local storage, but that won't provide HA.
>
>
> - Si
>
>
> ________________________________
> From: McClune, James <mcclunej@norwalktruckers.net>
> Sent: Monday, October 30, 2017 2:26 PM
> To: users@cloudstack.apache.org
> Subject: Re: Problems with KVM HA & STONITH
>
> Hi Dag,
>
> Thank you for responding back. I am currently running ACS 4.9 on an Ubunt=
u
> 14.04 VM. I have the three nodes, each having about 1TB of primary storag=
e
> (NFS) and 1TB of secondary storage (NFS). I added each NFS share into ACS=
.
> All nodes are in a cluster.
>
> Maybe I'm not understanding the setup or misconfigured something. I'm
> trying to setup an HA environment where if one node goes down, running an
> HA marked VM, the VM will start on another host. When I simulate a networ=
k
> disconnect or reboot of a host, all of the nodes go down (STONITH?).
>
> I am unsure on how to setup an HA environment, if all the nodes in the
> cluster go down. Any help is much appreciated!
>
> Thanks,
> James
>
> On Mon, Oct 30, 2017 at 3:49 AM, Dag Sonstebo <Dag.Sonstebo@shapeblue.com=
>
> wrote:
>
> > Hi James,
> >
> > I think  you possibly have over-configured your KVM hosts. If you use N=
FS
> > (and no clustered file system like CLVM) then there should be no need t=
o
> > configure STONITH. CloudStack takes care of your HA, so this is not
> > something you offload to the KVM host.
> >
> > (As mentioned the only time I have played with STONITH and CloudStack w=
as
> > for CLVM =E2=80=93 and I eventually found it not fit for purpose, too u=
nstable
> and
> > causing too many issues like you describe. Note this was for block
> storage
> > though =E2=80=93 not NFS).
> >
> > Regards,
> > Dag Sonstebo
> > Cloud Architect
> > ShapeBlue
> >
> > On 28/10/2017, 03:40, "Ivan Kudryavtsev" <kudryavtsev_ia@bw-sw.com>
> wrote:
> >
> >     Hi. If the node losts nfs host it reboots (acs agent behaviour). If
> you
> >     really have 3 storages, you'll go clusterwide reboot everytime your
> > host is
> >     down.
> >
> >     28 =D0=BE=D0=BA=D1=82. 2017 =D0=B3. 3:02 =D0=BF=D0=BE=D0=BB=D1=8C=
=D0=B7=D0=BE=D0=B2=D0=B0=D1=82=D0=B5=D0=BB=D1=8C "Simon Weller"
> > <sweller@ena.com.invalid>
> >     =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB:
> >
> >     > Hi James,
> >     >
> >     >
> >     > Can you elaborate a bit further on the storage? You say you're
> > running NFS
> >     > on all 3 nodes, can you explain how it is setup?
> >     >
> >     > Also, what version of ACS are you running?
> >     >
> >     >
> >     > - Si
> >     >
> >     >
> >     >
> >     >
> >     > ________________________________
> >     > From: McClune, James <mcclunej@norwalktruckers.net>
> >     > Sent: Friday, October 27, 2017 2:21 PM
> >     > To: users@cloudstack.apache.org
> >     > Subject: Problems with KVM HA & STONITH
> >     >
> >     > Hello Apache CloudStack Community,
> >     >
> >     > My setup consists of the following:
> >     >
> >     > - Three nodes (NODE1, NODE2, and NODE3)
> >     > NODE1 is running Ubuntu 16.04.3, NODE2 is running Ubuntu 16.04.3,
> > and NODE3
> >     > is running Ubuntu 14.04.5.
> >     > - Management Server (running on separate VM, not in cluster)
> >     >
> >     > The three nodes use KVM as the hypervisor. I also configured
> primary
> > and
> >     > secondary storage on all three of the nodes. I'm using NFS for th=
e
> > primary
> >     > & secondary storage. VM operations work great. Live migration wor=
ks
> > great.
> >     >
> >     > However, when a host goes down, the HA functionality does not wor=
k
> > at all.
> >     > Instead of spinning up the VM on another available host, the down
> > host
> >     > seems to trigger STONITH. When STONITH happens, all hosts in the
> > cluster go
> >     > down. This not only causes no HA, but also downs perfectly good
> > VM's. I
> >     > have read countless articles and documentation related to this
> > issue. I
> >     > still cannot find a viable solution for this issue. I really want
> to
> > use
> >     > Apache CloudStack, but cannot implement this in production when
> > STONITH
> >     > happens.
> >     >
> >     > I think I have something misconfigured. I thought I would reach o=
ut
> > to the
> >     > CloudStack community and ask for some friendly assistance.
> >     >
> >     > If there is anything (system-wise) you request in order to furthe=
r
> >     > troubleshoot this issue, please let me know and I'll send. I
> > appreciate any
> >     > help in this issue!
> >     >
> >     > --
> >     >
> >     > Thanks,
> >     >
> >     > James
> >     >
> >
> >
> >
> > Dag.Sonstebo@shapeblue.com
> > www.shapeblue.com<http://www.shapeblue.com>
> [http://www.shapeblue.com/wp-content/uploads/2017/06/logo.png]<
> http://www.shapeblue.com/>
>
> Shapeblue - The CloudStack Company<http://www.shapeblue.com/>
> www.shapeblue.com
> Rapid deployment framework for Apache CloudStack IaaS Clouds. CSForge is =
a
> framework developed by ShapeBlue to deliver the rapid deployment of a
> standardised ...
>
>
>
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
>
>
> --
>
>
>
> James McClune
>
> Technical Support Specialist
>
> Norwalk City Schools
>
> Phone: 419-660-6590
>
> mcclunej@norwalktruckers.net
>

--001a11429b783e6643055cecb25b--