cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Farrington <rfarring...@remitdata.com>
Subject RE: [Questionable] Re: Timeout with live migration
Date Tue, 13 Oct 2015 01:00:36 GMT
Hypervisor:  XenServer

We are moving a data volume from one storage onto another without shutting down the VM cause
that would just be silly and a triplication of effort with the whole copying to secondary
storage and then back off again. The volume is staying in the same cluster just moving to
a different Primary storage (or SR in the XenServer vernacular) 

If you are familiar with ESX this is a "Storage VMotion" where as in XenServer it is called
"Storage XenMotion". 

________________________________________
From: Rafael Weingärtner [rafaelweingartner@gmail.com]
Sent: Monday, October 12, 2015 7:53 PM
To: users@cloudstack.apache.org
Subject: [Questionable]  Re: Timeout with live migration

what do you mean with livre migrating data volume ?!
I understand a live migration of a VM, but volumes...

do you mean live migrating a VM that has a volume attached?
are you migrating that volume to a different cluster? or just a different
storage in the same cluster?
What hypervisor are you using ?


On Mon, Oct 12, 2015 at 9:47 PM, Ryan Farrington <rfarrington@remitdata.com>
wrote:

> Live migrating a data volume. We are purely on shared storage so no local
> storage is involved.
>
> ________________________________________
> From: Rafael Weingärtner [rafaelweingartner@gmail.com]
> Sent: Monday, October 12, 2015 7:37 PM
> To: users@cloudstack.apache.org
> Subject: [Questionable]  Re: Timeout with live migration
>
> Are you live migrating a VM, or migrating a volume of a stopped VM to a
> different primary storage?
>
> If it is a running VM, is the VM allocated in a shared storage or local
> storage?
>
> On Mon, Oct 12, 2015 at 9:17 PM, Ryan Farrington <
> rfarrington@remitdata.com>
> wrote:
>
> > The slow transfer is related to the storage we are trying to migrate off
> > of.  We are capable of getting about 350mbps off the disks but when we
> are
> > moving volumes that are greater than about 500GB we end up racing the
> clock
> > and hoping that the migration finishes before the job times out.   It
> would
> > be awesome to be able to manage that timeout and I know there are a ton
> of
> > settings I just don't know about and am hoping someone might be able to
> > point me in the right direction.
> >
> >
> > ________________________________________
> > From: Rafael Weingärtner [rafaelweingartner@gmail.com]
> > Sent: Monday, October 12, 2015 6:40 PM
> > To: users@cloudstack.apache.org
> > Subject: [Questionable]  Re: Timeout with live migration
> >
> > I would first check your NICs' speed and load, the amount of RAM
> allocated
> > for the migrating VM and than check the hypervisor log files.
> >
> > On Mon, Oct 12, 2015 at 8:19 PM, Jan-Arve Nygård <
> > jan.arve.nygard@gmail.com>
> > wrote:
> >
> > > What version are you running? Check if the copy.volume.wait setting is
> > set
> > > to 7200 and increase it. If not you could also check
> > > job.cancel.threshold.minutes and job.expire.minutes.
> > >
> > > -Jan-Arve
> > >
> > > 2015-10-13 0:46 GMT+02:00 Ryan Farrington <rfarrington@remitdata.com>:
> > >
> > > > We are experiencing a failure in cloudstack waiting for an async job
> > > > performing a live migration of a volume to finish. I've copied the
> > > relevant
> > > > log entries below.We acknowledge that the migration will take a few
> > hours
> > > > based on the volume of the data and we are looking for a way to
> > increase
> > > > the timeout of 7200 seconds into something we know we can work with.
> > > >
> > > >
> > > > 2015-10-12 00:19:36,043 DEBUG [o.a.c.s.RemoteHostEndPoint]
> > > > (Job-Executor-62:ctx-802065a9 ctx-bb27a168) Failed to send command,
> due
> > > to
> > > > Agent:27, com.cloud.exception.OperationTimedoutException: Commands
> > > > 835325398 to Host 27 timed out after 7200
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Rafael Weingärtner
> >
>
>
>
> --
> Rafael Weingärtner
>



--
Rafael Weingärtner

Mime
View raw message