cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hieu LE <hieul...@gmail.com>
Subject Re: [DISCUSS] Increasing VM IOPS by separating golden image in high IOPS partition in Xen Server ?
Date Fri, 06 Jun 2014 03:52:42 GMT
Hi Tim,


On Fri, Jun 6, 2014 at 1:39 AM, Tim Mackey <tmackey@gmail.com> wrote:

> Hieu,
>
> If I understand the objective correctly, you are trying to reduce the
> IO associated with a desktop "start of day" boot storm.  In your
> proposal, you're effectively wanting to move the CloudStack secondary
> storage concept to include a locally attached storage device which is
> SSD based.  While that seems viable in concept, in practice with
> XenServer your proposed flow could cause a bunch of issues.  Some of
> the challenges I see include:
>
> - XenServer hosts with multiple independent local storage are very
> rare.  See this KB article covering how to create such storage:
> http://support.citrix.com/article/CTX121313


Yes, I have been following this guide to attach a new SSD storage to Xen
Server host.


>
> - By default local storage is LVM based, but to enable thin
> provisioning you'll want EXT3.  See this blog for how to convert to
> EXT3:
> http://thinworldblog.blogspot.com/2011/08/enabling-thin-provisioning-on-existing.html


Thank you, I did not test with LVM storage repository, so I think I will
give it a try this approach with LVM based repo.


>
> - It seems like you're planning on using Storage XenMotion to move the
> VHD from the golden primary storage to normal primary storage, but
> that's going to move the entire VHD chain and it will do so over the
> network.  Here's a blog article describing a bit about how it works:
> http://blogs.citrix.com/2012/08/24/storage_xenmotion/.  I'm reasonably
> certain the design parameters didn't include local->local without
> network.
>

No, I did not use XenMotion to move VHD from golden PS to normal PS. Just a
simply Linux cp command to avoid moving whole VHD chain. This idea refer to
OpenStack Xapi plugins while importing VHD from staging area to SR.

https://github.com/openstack/nova/blob/master/plugins/xenserver/xenapi/etc/xapi.d/plugins/utils.py


> - If someone wants to take a snapshot of the VM, will that snapshot
> then got to normal secondary storage or back to the golden master?
> - To Punith's point, I *think* VM start will occur post clone, so the
> clone will consume network to occur and then will start on local
> storage.
>
> The big test I'd like to see first would be creating the golden master
> and from it creating a few VMs.  Then once you have those VMs run some
> normal XenServer operations like moving a VM within a pool, moving
> that VM across pools and assigning a home server.  If those pass, then
> things might work out, but if those fail then you'll need to sort
> things out within the XenServer code first. If these basic tests do
> work, then I'd look at the network usage to see if things did indeed
> get better.
>
> -tim
>

I have tested it and have a few runiing VMs from same golden image in
another SR. I will make the test case of live-migrate or migrate between
pool and report to you soon.


>
> On Thu, Jun 5, 2014 at 8:11 AM, Punith S <punith.s@cloudbyte.com> wrote:
> > hi Hieu,
> >
> > after going through your  "Golden Primary Storage" proposal , from my
> > understanding you are creating a SSD golden PS for holding parent
> > VDH(nothing but the template which go copied from secondary storage) and
> a
> > normal primary storage for ROOT volumes(child VHD) for the corresponding
> > vm's.
> >
> > from the following flowchart , i have the following questions,
> >
> > 1. since you are having problem with slow boot time of the vm's, will the
> > booting of the vm's happen in golden PS, ie while cloning ?
> >      if so, the spawning of the vm's will be always fast .
> >
> >     but i see you are starting the vm after moving the cloned vhd to the
> > ROOT PS and pointing the child vhd to its parent vhd on the GOLDEN PS,
> >     hence , there will be a network traffic between these two
> > primary storages, which will obviously slow down the vm's performance
> > forever.
> >
> > 2. what if someone removes the golden primary storage containing the the
> > parent VHD(template) where all the child VDH's in the root primary
> storage
> > are been pointed to ?
> >    if so, all vm's running will be crashed immediately. since its child
> > vhd's parent is removed.
> >
> > thanks
> >
> >
> > On Thu, Jun 5, 2014 at 8:59 AM, Hieu LE <hieulq19@gmail.com> wrote:
> >
> >> Mike, Punith,
> >>
> >> Please review "Golden Primary Storage" proposal. [1]
> >>
> >> Thank you.
> >>
> >> [1]:
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Golden+Primary+Storage
> >>
> >>
> >> On Wed, Jun 4, 2014 at 10:32 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com> wrote:
> >>
> >>> Daan helped out with this. You should be good to go now.
> >>>
> >>>
> >>> On Tue, Jun 3, 2014 at 8:50 PM, Hieu LE <hieulq19@gmail.com> wrote:
> >>>
> >>> > Hi Mike,
> >>> >
> >>> > Could you please give edit/create permission on ASF Jira/Wiki
> >>> confluence ?
> >>> > I can not add a new Wiki page.
> >>> >
> >>> > My Jira ID: hieulq
> >>> > Wiki: hieulq89
> >>> > Review Board: hieulq
> >>> >
> >>> > Thanks !
> >>> >
> >>> >
> >>> > On Wed, Jun 4, 2014 at 9:17 AM, Mike Tutkowski <
> >>> > mike.tutkowski@solidfire.com
> >>> > > wrote:
> >>> >
> >>> > > Hi,
> >>> > >
> >>> > > Yes, please feel free to add a new Wiki page for your design.
> >>> > >
> >>> > > Here is a link to applicable design info:
> >>> > >
> >>> > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Design
> >>> > >
> >>> > > Also, feel free to ask more questions and have me review your
> design.
> >>> > >
> >>> > > Thanks!
> >>> > > Mike
> >>> > >
> >>> > >
> >>> > > On Tue, Jun 3, 2014 at 7:29 PM, Hieu LE <hieulq19@gmail.com>
> wrote:
> >>> > >
> >>> > > > Hi Mike,
> >>> > > >
> >>> > > > You are right, performance will be decreased over time because
> >>> writes
> >>> > > IOPS
> >>> > > > will always end up on slower storage pool.
> >>> > > >
> >>> > > > In our case, we are using CloudStack integrated in VDI solution
> to
> >>> > > provived
> >>> > > > pooled VM type[1]. So may be my approach can bring better
UX for
> >>> user
> >>> > > with
> >>> > > > lower bootime ...
> >>> > > >
> >>> > > > A short change in design are followings
> >>> > > > - VM will be deployed with golden primary storage if primary
> >>> storage is
> >>> > > > marked golden and this VM template is also marked as golden.
> >>> > > > - Choosing the best deploy destionation for both golden primary
> >>> storage
> >>> > > and
> >>> > > > normal root volume primary storage. Chosen host can also
access
> both
> >>> > > > storage pools.
> >>> > > > - New Xen Server plug-in for modifying VHD parent id.
> >>> > > >
> >>> > > > Is there some place for me to submit my design and code.
Can I
> >>> write a
> >>> > > new
> >>> > > > proposal in CS wiki ?
> >>> > > >
> >>> > > > [1]:
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> http://support.citrix.com/proddocs/topic/xendesktop-rho/cds-choose-scheme-type-rho.html
> >>> > > >
> >>> > > >
> >>> > > > On Mon, Jun 2, 2014 at 9:04 PM, Mike Tutkowski <
> >>> > > > mike.tutkowski@solidfire.com
> >>> > > > > wrote:
> >>> > > >
> >>> > > > > It is an interesting idea. If the constraints you face
at your
> >>> > company
> >>> > > > can
> >>> > > > > be corrected somewhat by implementing this, then you
should go
> for
> >>> > it.
> >>> > > > >
> >>> > > > > It sounds like writes will be placed on the slower storage
> pool.
> >>> This
> >>> > > > means
> >>> > > > > as you update OS components, those updates will be placed
on
> the
> >>> > slower
> >>> > > > > storage pool. As such, your performance is likely to
somewhat
> >>> > decrease
> >>> > > > over
> >>> > > > > time (as more and more writes end up on the slower storage
> pool).
> >>> > > > >
> >>> > > > > That may be OK for your use case(s), though.
> >>> > > > >
> >>> > > > > You'll have to update the storage-pool orchestration
logic to
> take
> >>> > this
> >>> > > > new
> >>> > > > > scheme into account.
> >>> > > > >
> >>> > > > > Also, we'll have to figure out how this ties into storage
> tagging
> >>> (if
> >>> > > at
> >>> > > > > all).
> >>> > > > >
> >>> > > > > I'd be happy to review your design and code.
> >>> > > > >
> >>> > > > >
> >>> > > > > On Mon, Jun 2, 2014 at 1:54 AM, Hieu LE <hieulq19@gmail.com>
> >>> wrote:
> >>> > > > >
> >>> > > > > > Thanks Mike and Punith for quick reply.
> >>> > > > > >
> >>> > > > > > Both solutions you gave here are absolutely correct.
But as I
> >>> > > mentioned
> >>> > > > > in
> >>> > > > > > the first email, I want another better solution
for current
> >>> > > > > infrastructure
> >>> > > > > > at my company.
> >>> > > > > >
> >>> > > > > > Creating a high IOPS primary storage using storage
tags is
> good
> >>> but
> >>> > > it
> >>> > > > > will
> >>> > > > > > be very waste of disk capacity. For example, if
I only have
> 1TB
> >>> SSD
> >>> > > and
> >>> > > > > > deploy 100 VM from a 100GB template.
> >>> > > > > >
> >>> > > > > > So I think about a solution where a high IOPS primary
storage
> >>> can
> >>> > > only
> >>> > > > > > store golden image (master image), and a child
image of this
> VM
> >>> > will
> >>> > > be
> >>> > > > > > stored in another normal (NFS, ISCSI...) storage.
In this
> case,
> >>> > with
> >>> > > > 1TB
> >>> > > > > > SSD Primary Storage I can store as much golden
image as I
> need.
> >>> > > > > >
> >>> > > > > > I have also tested it with 256 GB SSD mounted on
Xen Server
> >>> 6.2.0
> >>> > > with
> >>> > > > > 2TB
> >>> > > > > > local storage 10000RPM, 6TB NFS share storage with
1GB
> network.
> >>> The
> >>> > > > IOPS
> >>> > > > > of
> >>> > > > > > VMs which have golden image (master image) in SSD
and child
> >>> image
> >>> > in
> >>> > > > NFS
> >>> > > > > > increate more than 30-40% compare with VMs which
have both
> >>> golden
> >>> > > image
> >>> > > > > and
> >>> > > > > > child image in NFS. The boot time of each VM is
also
> decrease.
> >>> > > ('cause
> >>> > > > > > golden image in SSD only reduced READ IOPS).
> >>> > > > > >
> >>> > > > > > Do you think this approach OK ?
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > On Mon, Jun 2, 2014 at 12:50 PM, Mike Tutkowski
<
> >>> > > > > > mike.tutkowski@solidfire.com> wrote:
> >>> > > > > >
> >>> > > > > > > Thanks, Punith - this is similar to what I
was going to
> say.
> >>> > > > > > >
> >>> > > > > > > Any time a set of CloudStack volumes share
IOPS from a
> common
> >>> > pool,
> >>> > > > you
> >>> > > > > > > cannot guarantee IOPS to a given CloudStack
volume at a
> given
> >>> > time.
> >>> > > > > > >
> >>> > > > > > > Your choices at present are:
> >>> > > > > > >
> >>> > > > > > > 1) Use managed storage (where you can create
a 1:1 mapping
> >>> > between
> >>> > > a
> >>> > > > > > > CloudStack volume and a volume on a storage
system that has
> >>> QoS).
> >>> > > As
> >>> > > > > > Punith
> >>> > > > > > > mentioned, this requires that you purchase
storage from a
> >>> vendor
> >>> > > who
> >>> > > > > > > provides guaranteed QoS on a volume-by-volume
bases AND has
> >>> this
> >>> > > > > > integrated
> >>> > > > > > > into CloudStack.
> >>> > > > > > >
> >>> > > > > > > 2) Create primary storage in CloudStack that
is not
> managed,
> >>> but
> >>> > > has
> >>> > > > a
> >>> > > > > > high
> >>> > > > > > > number of IOPS (ex. using SSDs). You can then
storage tag
> this
> >>> > > > primary
> >>> > > > > > > storage and create Compute and Disk Offerings
that use this
> >>> > storage
> >>> > > > tag
> >>> > > > > > to
> >>> > > > > > > make sure their volumes end up on this storage
pool
> (primary
> >>> > > > storage).
> >>> > > > > > This
> >>> > > > > > > will still not guarantee IOPS on a CloudStack
> volume-by-volume
> >>> > > basis,
> >>> > > > > but
> >>> > > > > > > it will at least place the CloudStack volumes
that need a
> >>> better
> >>> > > > chance
> >>> > > > > > of
> >>> > > > > > > getting higher IOPS on a storage pool that
could provide
> the
> >>> > > > necessary
> >>> > > > > > > IOPS. A big downside here is that you want
to watch how
> many
> >>> > > > CloudStack
> >>> > > > > > > volumes get deployed on this primary storage
because you'll
> >>> need
> >>> > to
> >>> > > > > > > essentially over-provision IOPS in this primary
storage to
> >>> > increase
> >>> > > > the
> >>> > > > > > > probability that each and every CloudStack
volume that uses
> >>> this
> >>> > > > > primary
> >>> > > > > > > storage gets the necessary IOPS (and isn't
as likely to
> suffer
> >>> > from
> >>> > > > the
> >>> > > > > > > Noisy Neighbor Effect). You should be able
to tell
> CloudStack
> >>> to
> >>> > > only
> >>> > > > > > use,
> >>> > > > > > > say, 80% (or whatever) of the storage you're
providing to
> it
> >>> (so
> >>> > as
> >>> > > > to
> >>> > > > > > > increase your effective IOPS per GB ratio).
This
> >>> > over-provisioning
> >>> > > of
> >>> > > > > > IOPS
> >>> > > > > > > to control Noisy Neighbors is avoided in option
1. In that
> >>> > > situation,
> >>> > > > > you
> >>> > > > > > > only provision the IOPS and capacity you actually
need. It
> is
> >>> a
> >>> > > much
> >>> > > > > more
> >>> > > > > > > sophisticated approach.
> >>> > > > > > >
> >>> > > > > > > Thanks,
> >>> > > > > > > Mike
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > > On Sun, Jun 1, 2014 at 11:36 PM, Punith S
<
> >>> > punith.s@cloudbyte.com>
> >>> > > > > > wrote:
> >>> > > > > > >
> >>> > > > > > > > hi hieu,
> >>> > > > > > > >
> >>> > > > > > > > your problem is the bottle neck we see
as a storage
> vendors
> >>> in
> >>> > > the
> >>> > > > > > cloud,
> >>> > > > > > > > meaning all the vms in the cloud have
not been guaranteed
> >>> iops
> >>> > > from
> >>> > > > > the
> >>> > > > > > > > primary storage, because in your case
i'm assuming you
> are
> >>> > > running
> >>> > > > > > > 1000vms
> >>> > > > > > > > on a xen cluster whose all vm's disks
are lying on a same
> >>> > primary
> >>> > > > nfs
> >>> > > > > > > > storage mounted to the cluster,
> >>> > > > > > > > hence you won't get the dedicated iops
for each vm since
> >>> every
> >>> > vm
> >>> > > > is
> >>> > > > > > > > sharing the same storage. to solve this
issue in
> cloudstack
> >>> we
> >>> > > the
> >>> > > > > > third
> >>> > > > > > > > party vendors have implemented the plugin(namely
> cloudbyte ,
> >>> > > > > solidfire
> >>> > > > > > > etc)
> >>> > > > > > > > to support managed storage(dedicated
volumes with
> guaranteed
> >>> > qos
> >>> > > > for
> >>> > > > > > each
> >>> > > > > > > > vms) , where we are mapping each root
disk(vdi) or data
> disk
> >>> > of a
> >>> > > > vm
> >>> > > > > > with
> >>> > > > > > > > one nfs or iscsi share coming out of
a pool, also we are
> >>> > > proposing
> >>> > > > > the
> >>> > > > > > > new
> >>> > > > > > > > feature to change volume iops on fly
in 4.5, where you
> can
> >>> > > increase
> >>> > > > > or
> >>> > > > > > > > decrease your root disk iops while booting
or at peak
> times.
> >>> > but
> >>> > > to
> >>> > > > > use
> >>> > > > > > > > this plugin you have to buy our storage
solution.
> >>> > > > > > > >
> >>> > > > > > > > if not , you can try creating a nfs share
out of ssd pool
> >>> > storage
> >>> > > > and
> >>> > > > > > > > create a primary storage in cloudstack
out of it named as
> >>> > golden
> >>> > > > > > primary
> >>> > > > > > > > storage with specific tag like gold,
and create a compute
> >>> > > offering
> >>> > > > > for
> >>> > > > > > > your
> >>> > > > > > > > template with the storage tag as gold,
hence all the vm's
> >>> you
> >>> > > > create
> >>> > > > > > will
> >>> > > > > > > > sit on this gold primary storage with
high iops. and
> other
> >>> data
> >>> > > > disks
> >>> > > > > > on
> >>> > > > > > > > other primary storage but still here
you cannot guarantee
> >>> the
> >>> > qos
> >>> > > > at
> >>> > > > > vm
> >>> > > > > > > > level.
> >>> > > > > > > >
> >>> > > > > > > > thanks
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > > On Mon, Jun 2, 2014 at 10:12 AM, Hieu
LE <
> >>> hieulq19@gmail.com>
> >>> > > > wrote:
> >>> > > > > > > >
> >>> > > > > > > >> Hi all,
> >>> > > > > > > >>
> >>> > > > > > > >> There are some problems while deploying
a large amount
> of
> >>> VMs
> >>> > in
> >>> > > > my
> >>> > > > > > > >> company
> >>> > > > > > > >> with CloudStack. All VMs are deployed
from same template
> >>> (e.g:
> >>> > > > > Windows
> >>> > > > > > > 7)
> >>> > > > > > > >> and the quantity is approximately
~1000VMs. The problems
> >>> here
> >>> > is
> >>> > > > low
> >>> > > > > > > IOPS,
> >>> > > > > > > >> low performance of VM (about ~10-11
IOPS, boot time is
> very
> >>> > > high).
> >>> > > > > The
> >>> > > > > > > >> storage of my company is SAN/NAS
with NFS and Xen Server
> >>> > 6.2.0.
> >>> > > > All
> >>> > > > > > Xen
> >>> > > > > > > >> Server nodes have standard server
HDD disk raid.
> >>> > > > > > > >>
> >>> > > > > > > >> I have found some solutions for this
such as:
> >>> > > > > > > >>
> >>> > > > > > > >>    - Enable Xen Server Intellicache
and some tweaks in
> >>> > > CloudStack
> >>> > > > > > codes
> >>> > > > > > > to
> >>> > > > > > > >>    deploy and start VM in Intellicache
mode. But this
> >>> solution
> >>> > > > will
> >>> > > > > > > >> transfer
> >>> > > > > > > >>    all IOPS from shared storage to
all local storage,
> hence
> >>> > > affect
> >>> > > > > and
> >>> > > > > > > >> limit
> >>> > > > > > > >>    some CloudStack features.
> >>> > > > > > > >>    - Buying some expensive storage
solutions and
> network to
> >>> > > > increase
> >>> > > > > > > IOPS.
> >>> > > > > > > >>    Nah..
> >>> > > > > > > >>
> >>> > > > > > > >> So, I am thinking about a new feature
that (may be)
> >>> increasing
> >>> > > > IOPS
> >>> > > > > > and
> >>> > > > > > > >> performance of VMs:
> >>> > > > > > > >>
> >>> > > > > > > >>    1. Separate golden image in high
IOPS partition:
> buying
> >>> new
> >>> > > > SSD,
> >>> > > > > > plug
> >>> > > > > > > >> in
> >>> > > > > > > >>    Xen Server and deployed a new
VM in NFS storage WITH
> >>> golden
> >>> > > > image
> >>> > > > > > in
> >>> > > > > > > >> this
> >>> > > > > > > >>    new SSD partition. This can reduce
READ IOPS in
> shared
> >>> > > storage
> >>> > > > > and
> >>> > > > > > > >> decrease
> >>> > > > > > > >>    boot time of VM. (Currenty, VM
deployed in Xen Server
> >>> > always
> >>> > > > > have a
> >>> > > > > > > >> master
> >>> > > > > > > >>    image (golden image - in VMWare)
always in the same
> >>> storage
> >>> > > > > > > repository
> >>> > > > > > > >> with
> >>> > > > > > > >>    different image (child image)).
We can do this trick
> by
> >>> > > > tweaking
> >>> > > > > in
> >>> > > > > > > VHD
> >>> > > > > > > >>    header file with new Xen Server
plug-in.
> >>> > > > > > > >>    2. Create golden primary storage
and VM template that
> >>> > enable
> >>> > > > this
> >>> > > > > > > >>    feature.
> >>> > > > > > > >>    3. So, all VMs deployed from template
that had
> enabled
> >>> this
> >>> > > > > feature
> >>> > > > > > > >> will
> >>> > > > > > > >>    have a golden image stored in
golden primary storage
> >>> (SSD
> >>> > or
> >>> > > > some
> >>> > > > > > > high
> >>> > > > > > > >> IOPS
> >>> > > > > > > >>    partition), and different image
(child image) stored
> in
> >>> > other
> >>> > > > > > normal
> >>> > > > > > > >>    primary storage.
> >>> > > > > > > >>
> >>> > > > > > > >> This new feature will not transfer
all IOPS from shared
> >>> > storage
> >>> > > to
> >>> > > > > > local
> >>> > > > > > > >> storage (because high IOPS partition
can be another high
> >>> IOPS
> >>> > > > shared
> >>> > > > > > > >> storage) and require less money than
buying new storage
> >>> > > solution.
> >>> > > > > > > >>
> >>> > > > > > > >> What do you think ? If possible,
may I write a proposal
> in
> >>> > > > > CloudStack
> >>> > > > > > > >> wiki ?
> >>> > > > > > > >>
> >>> > > > > > > >> BRs.
> >>> > > > > > > >>
> >>> > > > > > > >> Hieu Lee
> >>> > > > > > > >>
> >>> > > > > > > >> --
> >>> > > > > > > >> -----BEGIN GEEK CODE BLOCK-----
> >>> > > > > > > >> Version: 3.1
> >>> > > > > > > >> GCS/CM/IT/M/MU d-@? s+(++):+(++)
!a C++++(++++)$
> >>> > ULC++++(++)$ P
> >>> > > > > > > >> L++(+++)$ E
> >>> > > > > > > >> !W N* o+ K w O- M V- PS+ PE++ Y+
PGP+ t 5 X R tv+
> >>> b+(++)>+++
> >>> > DI-
> >>> > > > D+
> >>> > > > > G
> >>> > > > > > > >> e++(+++) h-- r(++)>+++ y-
> >>> > > > > > > >> ------END GEEK CODE BLOCK------
> >>> > > > > > > >>
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > > --
> >>> > > > > > > > regards,
> >>> > > > > > > >
> >>> > > > > > > > punith s
> >>> > > > > > > > cloudbyte.com
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > > --
> >>> > > > > > > *Mike Tutkowski*
> >>> > > > > > > *Senior CloudStack Developer, SolidFire Inc.*
> >>> > > > > > > e: mike.tutkowski@solidfire.com
> >>> > > > > > > o: 303.746.7302
> >>> > > > > > > Advancing the way the world uses the cloud
> >>> > > > > > > <http://solidfire.com/solution/overview/?video=play>*™*
> >>> > > > > > >
> >>> > > > > >
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > --
> >>> > > > > > -----BEGIN GEEK CODE BLOCK-----
> >>> > > > > > Version: 3.1
> >>> > > > > > GCS/CM/IT/M/MU d-@? s+(++):+(++) !a C++++(++++)$
> ULC++++(++)$ P
> >>> > > > > L++(+++)$
> >>> > > > > > E
> >>> > > > > > !W N* o+ K w O- M V- PS+ PE++ Y+ PGP+ t 5 X R tv+
b+(++)>+++
> DI-
> >>> > D+ G
> >>> > > > > > e++(+++) h-- r(++)>+++ y-
> >>> > > > > > ------END GEEK CODE BLOCK------
> >>> > > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > > --
> >>> > > > > *Mike Tutkowski*
> >>> > > > > *Senior CloudStack Developer, SolidFire Inc.*
> >>> > > > > e: mike.tutkowski@solidfire.com
> >>> > > > > o: 303.746.7302
> >>> > > > > Advancing the way the world uses the cloud
> >>> > > > > <http://solidfire.com/solution/overview/?video=play>*™*
> >>> > > > >
> >>> > > >
> >>> > > >
> >>> > > >
> >>> > > > --
> >>> > > > -----BEGIN GEEK CODE BLOCK-----
> >>> > > > Version: 3.1
> >>> > > > GCS/CM/IT/M/MU d-@? s+(++):+(++) !a C++++(++++)$ ULC++++(++)$
P
> >>> > > L++(+++)$
> >>> > > > E
> >>> > > > !W N* o+ K w O- M V- PS+ PE++ Y+ PGP+ t 5 X R tv+ b+(++)>+++
DI-
> D+
> >>> G
> >>> > > > e++(+++) h-- r(++)>+++ y-
> >>> > > > ------END GEEK CODE BLOCK------
> >>> > > >
> >>> > >
> >>> > >
> >>> > >
> >>> > > --
> >>> > > *Mike Tutkowski*
> >>> > > *Senior CloudStack Developer, SolidFire Inc.*
> >>> > > e: mike.tutkowski@solidfire.com
> >>> > > o: 303.746.7302
> >>> > > Advancing the way the world uses the cloud
> >>> > > <http://solidfire.com/solution/overview/?video=play>*™*
> >>> > >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > -----BEGIN GEEK CODE BLOCK-----
> >>> > Version: 3.1
> >>> > GCS/CM/IT/M/MU d-@? s+(++):+(++) !a C++++(++++)$ ULC++++(++)$ P
> >>> L++(+++)$
> >>> > E
> >>> > !W N* o+ K w O- M V- PS+ PE++ Y+ PGP+ t 5 X R tv+ b+(++)>+++ DI-
D+ G
> >>> > e++(+++) h-- r(++)>+++ y-
> >>> > ------END GEEK CODE BLOCK------
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the cloud
> >>> <http://solidfire.com/solution/overview/?video=play>*™*
> >>>
> >>
> >>
> >>
> >> --
> >> -----BEGIN GEEK CODE BLOCK-----
> >> Version: 3.1
> >> GCS/CM/IT/M/MU d-@? s+(++):+(++) !a C++++(++++)$ ULC++++(++)$ P
> L++(+++)$
> >> E !W N* o+ K w O- M V- PS+ PE++ Y+ PGP+ t 5 X R tv+ b+(++)>+++ DI- D+ G
> >> e++(+++) h-- r(++)>+++ y-
> >> ------END GEEK CODE BLOCK------
> >>
> >
> >
> >
> > --
> > regards,
> >
> > punith s
> > cloudbyte.com
>



-- 
-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GCS/CM/IT/M/MU d-@? s+(++):+(++) !a C++++(++++)$ ULC++++(++)$ P L++(+++)$ E
!W N* o+ K w O- M V- PS+ PE++ Y+ PGP+ t 5 X R tv+ b+(++)>+++ DI- D+ G
e++(+++) h-- r(++)>+++ y-
------END GEEK CODE BLOCK------

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message