cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrija Panic <andrija.pa...@gmail.com>
Subject Re: Merging/Removing Secondary Storage NFS
Date Mon, 16 Dec 2013 21:33:53 GMT
Yes that is true, but it was ISOs copy (they are also located under
/templates/ folder, so I though it was really templates that were being
copied, but instead it was ISOs...)
I need to migrate all 3 NFSs to new one, and it seems like I'm going to
change database, which is not really my prefered way...


On 16 December 2013 22:27, Nitin Mehta <Nitin.Mehta@citrix.com> wrote:

> This is definitely not expected. But, in the thread below I see you
> writing that the restarting MS initiated template copy. Is that not the
> case ?
>
> On 16/12/13 11:58 AM, "Andrija Panic" <andrija.panic@gmail.com> wrote:
>
> >Well, only ISOs have been copied to the new NFS storage... no other
> >templates...
> >
> >Is this expected behaviour or should I consider this a bug ?
> >
> >Any clue on troubleshooting this ?
> >
> >
> >On 16 December 2013 17:05, Andrija Panic <andrija.panic@gmail.com> wrote:
> >
> >> I have just restarted the management server, and the templates copy
> >> process has begin to the new NFS server...
> >>
> >> Will update.
> >>
> >>
> >> On 16 December 2013 16:14, Andrija Panic <andrija.panic@gmail.com>
> >>wrote:
> >>
> >>> HI Geoff,
> >>> I have just added new NFS secondary storage to CS.
> >>> Can you please tell, when shall I expect all existing ISOs and
> >>>templates
> >>> to be copied over (start being copied...) - so far, I can observe NO
> >>> network traffic.
> >>> Is this maybe configurable like purge thread, or similar ? I have also
> >>> issues the script for checking the SSVM (
> >>> /usr/local/cloud/systemvm/ssvm-check.sh) and have no problems so far.
> >>>
> >>> More important, I can confirm that I do NOT have checksum inside the
> >>> "template.properties" on public templates, only on system templates...
> >>> And I can confirm that I do not have public templates being replicated
> >>>on
> >>> all NFS storages...that is strange. ISOs do get replicated, but public
> >>> templates does not...
> >>> Is this a bug or what...? I can not observer any errors in management
> >>> log...
> >>>
> >>> Thanks,
> >>> Andrija
> >>>
> >>>
> >>>
> >>> On 13 December 2013 20:10, Andrija Panic <andrija.panic@gmail.com>
> >>>wrote:
> >>>
> >>>> Thank you very much, I will check... that... funny thing, I search for
> >>>> files created in last i.e. 1 hour, and there is only one file created
> >>>>on 1
> >>>> NFS, no files created on other NFSs.
> >>>>
> >>>> Thanks
> >>>>
> >>>> Sent from Google Nexus 4
> >>>> On Dec 13, 2013 6:06 PM, "Geoff Higginbottom" <
> >>>> geoff.higginbottom@shapeblue.com> wrote:
> >>>>
> >>>>> The templates get unique names, however they are in the same folder
> >>>>> structure.
> >>>>>
> >>>>> e.g.
> >>>>>
> >>>>> My S1 has
> >>>>> \\IPADDRESS\lonsec\template\tmpl\1\5
> >>>>>
> >>>>> And my S2 has
> >>>>> \\IPADDDRESS\lonsec2\template\tmpl\1\5
> >>>>>
> >>>>> Note that the '1' is the account ID of the user who created the
> >>>>> template, in this case 'system' as this is a system vm template,
the
> >>>>>'5' is
> >>>>> simply the 5th template created on this system
> >>>>>
> >>>>> The two template.properties files contain the following
> >>>>>
> >>>>> S1 template.properties file
> >>>>> #
> >>>>> #Mon May 20 14:11:06 UTC 2013
> >>>>> filename=d015ebd8-5395-31a5-9648-a179062885d8.vhd
> >>>>> vhd=true
> >>>>> id=5
> >>>>> vhd.filename=d015ebd8-5395-31a5-9648-a179062885d8.vhd
> >>>>> public=true
> >>>>> uniquename=centos56-x86_64-xen
> >>>>> vhd.virtualsize=21474836480
> >>>>> virtualsize=21474836480
> >>>>> checksum=905cec879afd9c9d22ecc8036131a180
> >>>>> hvm=false
> >>>>> description=CentOS 5.6(64-bit) no GUI (XenServer)
> >>>>> vhd.size=1758786048
> >>>>> size=1758786048
> >>>>>
> >>>>> S1 template.properties file
> >>>>> #
> >>>>> #Wed Nov 21 23:14:59 UTC 2012
> >>>>> filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd
> >>>>> vhd=true
> >>>>> id=5
> >>>>> vhd.filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd
> >>>>> public=true
> >>>>> uniquename=centos56-x86_64-xen
> >>>>> vhd.virtualsize=21474836480
> >>>>> virtualsize=21474836480
> >>>>> checksum=905cec879afd9c9d22ecc8036131a180
> >>>>> hvm=false
> >>>>> description=CentOS 5.6(64-bit) no GUI (XenServer)
> >>>>> vhd.size=1758786048
> >>>>> size=1758786048
> >>>>> dsa
> >>>>>
> >>>>> Note that the line
> >>>>> checksum=905cec879afd9c9d22ecc8036131a180
> >>>>> is common to both, you will also find this value in the db in table
> >>>>> cloud.vm_template
> >>>>> This is the link to the template on disk, and the one referenced
in
> >>>>>the
> >>>>> DB
> >>>>>
> >>>>> You can also see from the dates within the two properties files,
that
> >>>>> this template was originally deployed back in may, but when I added
> >>>>>the 2nd
> >>>>> Secondary Storage Node in November, the template was automatically
> >>>>>copied
> >>>>> across.
> >>>>>
> >>>>> Hopefully all this info helps you.
> >>>>>
> >>>>> Regards
> >>>>>
> >>>>> Geoff Higginbottom
> >>>>>
> >>>>> D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581
> >>>>>
> >>>>> geoff.higginbottom@shapeblue.com
> >>>>>
> >>>>> -----Original Message-----
> >>>>> From: Andrija Panic [mailto:andrija.panic@gmail.com]
> >>>>> Sent: 13 December 2013 16:47
> >>>>> To: users@cloudstack.apache.org
> >>>>> Subject: Re: Merging/Removing Secondary Storage NFS
> >>>>>
> >>>>> Hi Jeff, thanks for that detailed explanation, I'm on the KVM by
the
> >>>>> way...
> >>>>> For the *volume* snaphost, I can see that real snapshot exist localy
> >>>>>(I
> >>>>> say real snaphost, not clone), but also the clone is created on
1 of
> >>>>>those
> >>>>> NFS servers...
> >>>>>
> >>>>> Can you somehow instruct me to check if new template I have just
> >>>>> created, is actualy copied on all 3 NFS hosts (all3 NFS are online,
> >>>>>and
> >>>>> working
> >>>>> fine) ?
> >>>>> Template is only 4GB of real size on the host, and it should be
very
> >>>>> quickly copied to where it shoyld be. (my IO and netowrk traffic
are
> >>>>>pretty
> >>>>> low in general).
> >>>>>
> >>>>> Should the templates files be created on all 3 NFS storages, with
the
> >>>>> same file name? Or can I see this somehow from the database...I'm
> >>>>>trying to
> >>>>> decode it by myself, but can't figure this out...
> >>>>>
> >>>>> Thanks again,
> >>>>> Andrija
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 13 December 2013 17:35, Geoff Higginbottom <
> >>>>> geoff.higginbottom@shapeblue.com> wrote:
> >>>>>
> >>>>> > Andrija,
> >>>>> >
> >>>>> > The design is that any Public Template, or ISO gets copied
to every
> >>>>> > secondary storage in the zone.  The copying is undertaken by
the
> >>>>>SSVM
> >>>>> > and will obviously take some time if you have just added a
2nd
> >>>>>Public
> >>>>> > ISO or Template, so your copy process may not have finished.
> >>>>> >
> >>>>> > For 4.2 'Snapshots' are now even more confusing than ever.
 We now
> >>>>> have:
> >>>>> >
> >>>>> > 1. VM Snapshots (ESXi & XenServer Only) these are true
Hypervisor
> >>>>> > Snapshots, enabling quick roll back to the Snapshot, including
> >>>>>Memory
> >>>>> State.
> >>>>> > 2. Volume Snapshots (better thought of as Volume Backups)
> >>>>> >
> >>>>> > VM Snapshots remain on Primary Storage to enable the quick
> >>>>>roll-back
> >>>>> > to Snapshot to work.
> >>>>> > Volume Snapshots (backups) get copied to Secondary Storage.
 If you
> >>>>> > have more than one Secondary Storage available, the first backup
is
> >>>>> > copied to one of the Secondary Storage volumes, chosen at random,
> >>>>>then
> >>>>> > all subsequent backups are always copied to the same Secondary
> >>>>>Storage
> >>>>> > volume
> >>>>> >
> >>>>> > Regards
> >>>>> >
> >>>>> > Geoff Higginbottom
> >>>>> >
> >>>>> > D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581
> >>>>> >
> >>>>> > geoff.higginbottom@shapeblue.com
> >>>>> >
> >>>>> > -----Original Message-----
> >>>>> > From: Andrija Panic [mailto:andrija.panic@gmail.com]
> >>>>> > Sent: 13 December 2013 16:29
> >>>>> > To: users@cloudstack.apache.org
> >>>>> > Subject: Re: Merging/Removing Secondary Storage NFS
> >>>>> >
> >>>>> > HI Geoff,
> >>>>> > But, I have verified that my public templates are storaged
only on
> >>>>>1
> >>>>> > host... (at least based on file name of the template on one
of
> >>>>>those
> >>>>> > host - these file names does NOT exist on other 2 NFS hosts...)
Any
> >>>>> clue ?
> >>>>> >
> >>>>> > Can you also confirm that now in 4.2, since snaphost are really
a
> >>>>> > snapshots, they are not moved to secondary storage, but exist
> >>>>>localy
> >>>>> > on the primary storage next to the original image they were
created
> >>>>> > from (I can see this with lsof and qemu-img info).
> >>>>> >
> >>>>> > Thanks
> >>>>> >
> >>>>> >
> >>>>> > On 13 December 2013 17:21, Geoff Higginbottom <
> >>>>> > geoff.higginbottom@shapeblue.com> wrote:
> >>>>> >
> >>>>> > > You don't need to copy Public Templates from S1 to S2
if they
> >>>>>have
> >>>>> > > both been on line as Public Templates are always copied
to every
> >>>>>sec
> >>>>> > > storage in the zone
> >>>>> > >
> >>>>> > > Regards
> >>>>> > >
> >>>>> > > Geoff Higginbottom
> >>>>> > >
> >>>>> > > D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581
> >>>>> > >
> >>>>> > > geoff.higginbottom@shapeblue.com
> >>>>> > >
> >>>>> > > -----Original Message-----
> >>>>> > > From: Andrija Panic [mailto:andrija.panic@gmail.com]
> >>>>> > > Sent: 13 December 2013 16:11
> >>>>> > > To: users@cloudstack.apache.org
> >>>>> > > Subject: Re: Merging/Removing Secondary Storage NFS
> >>>>> > >
> >>>>> > > Just to correct myself: that guide:  tells nice job for
migrating
> >>>>> > > snapshots.
> >>>>> > >
> >>>>> > > But I'm confused why only private templates should be
migrated,
> >>>>>why
> >>>>> > > not also the public ones?
> >>>>> > >
> >>>>> > > Quote from the page:
> >>>>> > > *"Note*: You must copy only private templates on Secondary
> >>>>>storage
> >>>>> > > host S2 to S1."
> >>>>> > > And the whole guide is about migrating those private templates
> >>>>> > > (these are mostly system VM templates and such)
> >>>>> > >
> >>>>> > > What about public templates? I have predefined some public
> >>>>> > > templates, that as far as I can see, are only storaged
on some of
> >>>>> my 3 NFS hosts...
> >>>>> > > ?
> >>>>> > >
> >>>>> > > Thanks
> >>>>> > >
> >>>>> > >
> >>>>> > >
> >>>>> > > On 13 December 2013 16:51, Andrija Panic
> >>>>><andrija.panic@gmail.com>
> >>>>> > wrote:
> >>>>> > >
> >>>>> > > > HI,
> >>>>> > > >
> >>>>> > > > anyone has good doc/procedure for removing old NFS
(migrate
> >>>>> > > > everything to new NFS server) ?
> >>>>> > > >
> >>>>> > > > I was thinking of editing the database - dump to
sql file,
> >>>>>search,
> >>>>> > > > replace old IP address with new IP address, and rsync
> >>>>>everything
> >>>>> > > > from old NFS to new NFS, keeping the same directory
structure.
> >>>>> > > > Then import the edited dump file.
> >>>>> > > >
> >>>>> > > > I also found some Citrix's guide, but that does not
seem to
> >>>>> > > > migrate everything (only the system VM templates,
as per my
> >>>>> > > > understanding after reading the docs...)
> >>>>> > > > http://support.citrix.com/article/CTX135229
> >>>>> > > >
> >>>>> > > > Thanks.
> >>>>> > > >
> >>>>> > > > --
> >>>>> > > >
> >>>>> > > > Andrija Panić
> >>>>> > > > --------------------------------------
> >>>>> > > >   http://admintweets.com
> >>>>> > > > --------------------------------------
> >>>>> > > >
> >>>>> > >
> >>>>> > >
> >>>>> > >
> >>>>> > > --
> >>>>> > >
> >>>>> > > Andrija Panić
> >>>>> > > --------------------------------------
> >>>>> > >   http://admintweets.com
> >>>>> > > --------------------------------------
> >>>>> > > This email and any attachments to it may be confidential
and are
> >>>>> > > intended solely for the use of the individual to whom
it is
> >>>>> addressed.
> >>>>> > > Any views or opinions expressed are solely those of the
author
> >>>>>and
> >>>>> > > do not necessarily represent those of Shape Blue Ltd or
related
> >>>>> > > companies. If you are not the intended recipient of this
email,
> >>>>>you
> >>>>> > > must neither take any action based upon its contents,
nor copy or
> >>>>> > > show it to anyone. Please contact the sender if you believe
you
> >>>>>have
> >>>>> > > received this email in error. Shape Blue Ltd is a company
> >>>>> > > incorporated in England & Wales. ShapeBlue Services
India LLP is
> >>>>>a
> >>>>> > > company incorporated in India and is operated under license
from
> >>>>> > > Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is
a company
> >>>>> > > incorporated in Brasil and is operated under license from
Shape
> >>>>>Blue
> >>>>> > > Ltd. ShapeBlue is a
> >>>>> > registered trademark.
> >>>>> > >
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> > --
> >>>>> >
> >>>>> > Andrija Panić
> >>>>> > --------------------------------------
> >>>>> >   http://admintweets.com
> >>>>> > --------------------------------------
> >>>>> > This email and any attachments to it may be confidential and
are
> >>>>> > intended solely for the use of the individual to whom it is
> >>>>>addressed.
> >>>>> > Any views or opinions expressed are solely those of the author
and
> >>>>>do
> >>>>> > not necessarily represent those of Shape Blue Ltd or related
> >>>>> > companies. If you are not the intended recipient of this email,
you
> >>>>> > must neither take any action based upon its contents, nor copy
or
> >>>>>show
> >>>>> > it to anyone. Please contact the sender if you believe you
have
> >>>>> > received this email in error. Shape Blue Ltd is a company
> >>>>>incorporated
> >>>>> > in England & Wales. ShapeBlue Services India LLP is a company
> >>>>> > incorporated in India and is operated under license from Shape
Blue
> >>>>> > Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated
> >>>>>in
> >>>>> > Brasil and is operated under license from Shape Blue Ltd. ShapeBlue
> >>>>> is a registered trademark.
> >>>>> >
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>>
> >>>>> Andrija Panić
> >>>>> --------------------------------------
> >>>>>   http://admintweets.com
> >>>>> --------------------------------------
> >>>>> This email and any attachments to it may be confidential and are
> >>>>> intended solely for the use of the individual to whom it is
> >>>>>addressed. Any
> >>>>> views or opinions expressed are solely those of the author and do
not
> >>>>> necessarily represent those of Shape Blue Ltd or related companies.
> >>>>>If you
> >>>>> are not the intended recipient of this email, you must neither take
> >>>>>any
> >>>>> action based upon its contents, nor copy or show it to anyone. Please
> >>>>> contact the sender if you believe you have received this email in
> >>>>>error.
> >>>>> Shape Blue Ltd is a company incorporated in England & Wales.
> >>>>>ShapeBlue
> >>>>> Services India LLP is a company incorporated in India and is
> >>>>>operated under
> >>>>> license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda
is a
> >>>>> company incorporated in Brasil and is operated under license from
> >>>>>Shape
> >>>>> Blue Ltd. ShapeBlue is a registered trademark.
> >>>>>
> >>>>
> >>>
> >>>
> >>> --
> >>>
> >>> Andrija Panić
> >>> --------------------------------------
> >>>   http://admintweets.com
> >>> --------------------------------------
> >>>
> >>
> >>
> >>
> >> --
> >>
> >> Andrija Panić
> >> --------------------------------------
> >>   http://admintweets.com
> >> --------------------------------------
> >>
> >
> >
> >
> >--
> >
> >Andrija Panić
> >--------------------------------------
> >  http://admintweets.com
> >--------------------------------------
>
>


-- 

Andrija Panić
--------------------------------------
  http://admintweets.com
--------------------------------------

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message