Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Date: Fri, 14 Jun 2013 14:20:11 -0400 (EDT)
From: Simon Weller <sweller@ena.com>
To: dev@cloudstack.apache.org
Message-ID: <317159240.32874772.1371234011924.JavaMail.root@ena.com>
In-Reply-To: 
 <CACOnxCjrXgekJnGsey+UYK=sbQDT=Uk_9rhfih4QaN3V4crs8w@mail.gmail.com>
Subject: Re: [MERGE] disk_io_throttling to MASTER
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_32874771_1788076594.1371234011922"

------=_Part_32874771_1788076594.1371234011922
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

I'd like to comment on this briefly.=20


I think an assumption is being made that the SAN is being dedicated to a CS=
 instance.=20

My person opinion that this whole IOPS calculation is getting rather compli=
cated, and could probably be much simpler than this. Over subscription is a=
 fact of life on virtually all storage, and is really no different in conce=
pt than multiple virt instances on a single piece of hardware. All decent S=
ANs offer many management options for the storage engineers to keep track o=
f IOPS utilization, and plan for spindle augmentation as required.=20
Is it really the job of CS to become yet another management layer on top of=
 this?=20

----- Original Message -----

From: "Mike Tutkowski" <mike.tutkowski@solidfire.com>=20
To: dev@cloudstack.apache.org=20
Cc: "John Burwell" <jburwell@basho.com>, "Wei Zhou" <ustcweizhou@gmail.com>=
=20
Sent: Friday, June 14, 2013 1:00:26 PM=20
Subject: Re: [MERGE] disk_io_throttling to MASTER=20

1) We want number of IOPS currently supported by the SAN.=20

2) We want the number of IOPS that are committed (sum of min IOPS for each=
=20
volume).=20

We could do the following to keep track of IOPS:=20

The plug-in could have a timer thread that goes off every, say, 1 minute.=
=20

It could query the SAN for the number of nodes that make up the SAN and=20
multiple this by 50,000. This is essentially the number of supported IOPS=
=20
of the SAN.=20

The next API call could be to get all of the volumes on the SAN. Iterate=20
through them all and add up their min IOPS values. This is the number of=20
IOPS the SAN is committed to.=20

These two numbers can then be updated in the storage_pool table (a column=
=20
for each value).=20

The allocators can get these values as needed (and they would be as=20
accurate as the last time the thread asked the SAN for this info).=20

These two fields, the min IOPS of the volume to create, and the overcommit=
=20
ratio of the plug-in would tell the allocator if it can select the given=20
storage pool.=20

What do you think?=20


On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <=20
mike.tutkowski@solidfire.com> wrote:=20

> "As I mentioned previously, I am very reluctant for any feature to come=
=20
> into master that can exhaust resources."=20
>=20
> Just wanted to mention that, worst case, the SAN would fail creation of=
=20
> the volume before allowing a new volume to break the system.=20
>=20
>=20
> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <=20
> mike.tutkowski@solidfire.com> wrote:=20
>=20
>> Hi John,=20
>>=20
>> Are you thinking we add a column on to the storage pool table,=20
>> IOPS_Count, where we add and subtract committed IOPS?=20
>>=20
>> That is easy enough.=20
>>=20
>> How do you want to determine what the SAN is capable of supporting IOPS=
=20
>> wise? Remember we're dealing with a dynamic SAN here...as you add storag=
e=20
>> nodes to the cluster, the number of IOPS increases. Do we have a thread =
we=20
>> can use to query external devices like this SAN to update the supported=
=20
>> number of IOPS?=20
>>=20
>> Also, how do you want to enforce the IOPS limit? Do we pass in an=20
>> overcommit ration to the plug-in when it's created? We would need to sto=
re=20
>> this in the storage_pool table, as well, I believe.=20
>>=20
>> We should also get Wei involved in this as his feature will need similar=
=20
>> functionality.=20
>>=20
>> Also, we should do this FAST as we have only two weeks left and many of=
=20
>> us will be out for several days for the CS Collab Conference.=20
>>=20
>> Thanks=20
>>=20
>>=20
>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jburwell@basho.com>wrote=
:=20
>>=20
>>> Mike,=20
>>>=20
>>> Querying the SAN only indicates the number of IOPS currently in use.=20
>>> The allocator needs to check the number of IOPS committed which is trac=
ked=20
>>> by CloudStack. For 4.2, we should be able to query the number of IOPS=
=20
>>> committed to a DataStore, and determine whether or not the number reque=
sted=20
>>> can be fulfilled by that device. It seems to be that a=20
>>> DataStore#getCommittedIOPS() : Long method would be sufficient.=20
>>> DataStore's that don't support provisioned IOPS would return null.=20
>>>=20
>>> As I mentioned previously, I am very reluctant for any feature to come=
=20
>>> into master that can exhaust resources.=20
>>>=20
>>> Thanks,=20
>>> -John=20
>>>=20
>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <=20
>>> mike.tutkowski@solidfire.com> wrote:=20
>>>=20
>>> > Yeah, I'm not sure I could come up with anything near an accurate=20
>>> > assessment of how many IOPS are currently available on the SAN (or=20
>>> even a=20
>>> > total number that are available for volumes). Not sure if there's yet=
=20
>>> an=20
>>> > API call for that.=20
>>> >=20
>>> > If I did know this number (total number of IOPS supported by the SAN)=
,=20
>>> we'd=20
>>> > still have to keep track of the total number of volumes we've created=
=20
>>> from=20
>>> > CS on the SAN in terms of their IOPS. Also, if an admin issues an API=
=20
>>> call=20
>>> > directly to the SAN to tweak the number of IOPS on a given volume or=
=20
>>> set of=20
>>> > volumes (not supported from CS, but supported via the SolidFire API),=
=20
>>> our=20
>>> > numbers in CS would be off.=20
>>> >=20
>>> > I'm thinking verifying sufficient number of IOPS is a really good ide=
a=20
>>> for=20
>>> > a future release.=20
>>> >=20
>>> > For 4.2 I think we should stick to having the allocator detect if=20
>>> storage=20
>>> > QoS is desired and if the storage pool in question supports that=20
>>> feature.=20
>>> >=20
>>> > If you really are over provisioned on your SAN in terms of IOPS or=20
>>> > capacity, the SAN can let the admin know in several different ways=20
>>> (e-mail,=20
>>> > SNMP, GUI).=20
>>> >=20
>>> >=20
>>> > On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jburwell@basho.com>=20
>>> wrote:=20
>>> >=20
>>> >> Mike,=20
>>> >>=20
>>> >> Please see my comments in-line below.=20
>>> >>=20
>>> >> Thanks,=20
>>> >> -John=20
>>> >>=20
>>> >> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <=20
>>> mike.tutkowski@solidfire.com>=20
>>> >> wrote:=20
>>> >>=20
>>> >>> Comments below in red.=20
>>> >>>=20
>>> >>> Thanks=20
>>> >>>=20
>>> >>>=20
>>> >>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jburwell@basho.com>=
=20
>>> >> wrote:=20
>>> >>>=20
>>> >>>> Mike,=20
>>> >>>>=20
>>> >>>> Overall, I agree with the steps to below for 4.2. However, we may=
=20
>>> want=20
>>> >> to=20
>>> >>>> throw an exception if we can not fulfill a requested QoS. If the=
=20
>>> user=20
>>> >> is=20
>>> >>>> expecting that the hypervisor will provide a particular QoS, and=
=20
>>> that is=20
>>> >>>> not possible, it seems like we should inform them rather than=20
>>> silently=20
>>> >>>> ignoring their request.=20
>>> >>>>=20
>>> >>>=20
>>> >>> Sure, that sounds reasonable.=20
>>> >>>=20
>>> >>> We'd have to come up with some way for the allocators to know about=
=20
>>> the=20
>>> >>> requested storage QoS and then query the candidate drivers.=20
>>> >>>=20
>>> >>> Any thoughts on how we might do that?=20
>>> >>>=20
>>> >>>=20
>>> >>>>=20
>>> >>>> To collect my thoughts from previous parts of the thread, I am=20
>>> >>>> uncomfortable with the idea that the management server can=20
>>> overcommit a=20
>>> >>>> resource. You had mentioned querying the device for available IOPS=
.=20
>>> >> While=20
>>> >>>> that would be nice, it seems like we could fall back to a max IOPS=
=20
>>> and=20
>>> >>>> overcommit factor manually calculated and entered by the=20
>>> >>>> administrator/operator. I think such threshold and allocation rail=
s=20
>>> >> should=20
>>> >>>> be added for both provisioned IOPS and throttled I/O -- it is a=20
>>> basic=20
>>> >>>> feature of any cloud orchestration platform.=20
>>> >>>>=20
>>> >>>=20
>>> >>> Are you thinking this ability would make it into 4.2? Just curious=
=20
>>> what=20
>>> >>> release we're talking about here. For the SolidFire SAN, you might=
=20
>>> have,=20
>>> >>> say, four separate storage nodes to start (200,000 IOPS) and then=
=20
>>> later=20
>>> >> add=20
>>> >>> a new node (now you're at 250,000 IOPS). CS would have to have a wa=
y=20
>>> to=20
>>> >>> know that the number of supported IOPS has increased.=20
>>> >>=20
>>> >> Yes, I think we need some *basic*/conservative rails in 4.2. For=20
>>> example,=20
>>> >> we may only support expanding capacity in 4.2, and not handle any=20
>>> >> re-balance scenarios -- node failure, addition, etc. Extrapolating=
=20
>>> a=20
>>> >> bit, the throttled I/O enhancement seems like it needs a similar set=
=20
>>> of=20
>>> >> rails defined per host.=20
>>> >>=20
>>> >>>=20
>>> >>>=20
>>> >>>>=20
>>> >>>> For 4.3, I don't like the idea that a QoS would be expressed in a=
=20
>>> >>>> implementation specific manner. I think we need to arrive at a=20
>>> general=20
>>> >>>> model that can be exploited by the allocators and planners. We=20
>>> should=20
>>> >>>> restrict implementation specific key-value pairs (call them detail=
s,=20
>>> >>>> extended data, whatever) to information that is unique to the=20
>>> driver and=20
>>> >>>> would provide no useful information to the management server's=20
>>> >>>> orchestration functions. A resource QoS does not seem to fit those=
=20
>>> >>>> criteria.=20
>>> >>>>=20
>>> >>>=20
>>> >>> I wonder if this would be a good discussion topic for Sunday's CS=
=20
>>> Collab=20
>>> >>> Conf hack day that Joe just sent out an e-mail about?=20
>>> >>=20
>>> >> Yes, it would -- I will put something in the wiki topic. It will=20
>>> also be=20
>>> >> part of my talk on Monday -- How to Run from Zombie which include=20
>>> some of=20
>>> >> my opinions on the topic.=20
>>> >>=20
>>> >>>=20
>>> >>>=20
>>> >>>>=20
>>> >>>> Thanks,=20
>>> >>>> -John=20
>>> >>>>=20
>>> >>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <=20
>>> >> mike.tutkowski@solidfire.com>=20
>>> >>>> wrote:=20
>>> >>>>=20
>>> >>>>> So, here's my suggestion for 4.2:=20
>>> >>>>>=20
>>> >>>>> Accept the values as they are currently required (four new fields=
=20
>>> for=20
>>> >>>> Wei's=20
>>> >>>>> feature or two new fields for mine).=20
>>> >>>>>=20
>>> >>>>> The Add Disk Offering dialog needs three new radio buttons:=20
>>> >>>>>=20
>>> >>>>> 1) No QoS=20
>>> >>>>>=20
>>> >>>>> 2) Hypervisor QoS=20
>>> >>>>>=20
>>> >>>>> 3) Storage Qos=20
>>> >>>>>=20
>>> >>>>> The admin needs to specify storage tags that only map to storage=
=20
>>> that=20
>>> >>>>> supports Storage QoS.=20
>>> >>>>>=20
>>> >>>>> The admin needs to be aware for Hypervisor QoS that unless all=20
>>> >>>> hypervisors=20
>>> >>>>> in use support the new fields, they may not be enforced.=20
>>> >>>>>=20
>>> >>>>> Post 4.3:=20
>>> >>>>>=20
>>> >>>>> Come up with a way to more generally enter these parameters=20
>>> (probably=20
>>> >>>> just=20
>>> >>>>> key/value pairs sent to the drivers).=20
>>> >>>>>=20
>>> >>>>> Have the drivers expose their feature set so the allocators can=
=20
>>> >> consider=20
>>> >>>>> them more fully and throw an exception if there is not a sufficie=
nt=20
>>> >>>> match.=20
>>> >>>>>=20
>>> >>>>>=20
>>> >>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <=20
>>> >>>>> mike.tutkowski@solidfire.com> wrote:=20
>>> >>>>>=20
>>> >>>>>> My thinking is, for 4.2, while not ideal, we will need to put so=
me=20
>>> >>>> burden=20
>>> >>>>>> on the admin to configure a Disk Offering in a way that makes=20
>>> sense.=20
>>> >> For=20
>>> >>>>>> example, if he wants to use storage QoS with supported Min and M=
ax=20
>>> >>>> values,=20
>>> >>>>>> he'll have to put in a storage tag that references the SolidFire=
=20
>>> >> primary=20
>>> >>>>>> storage (plug-in). If he puts in a storage tag that doesn't, the=
n=20
>>> he's=20
>>> >>>> not=20
>>> >>>>>> going to get the Min and Max feature. We could add help text to=
=20
>>> the=20
>>> >>>> pop-up=20
>>> >>>>>> dialog that's displayed when you click in the Min and Max text=
=20
>>> fields.=20
>>> >>>>>>=20
>>> >>>>>> Same idea for Wei's feature.=20
>>> >>>>>>=20
>>> >>>>>> Not idea, true...perhaps we can brainstorm on a more comprehensi=
ve=20
>>> >>>>>> approach for post 4.2.=20
>>> >>>>>>=20
>>> >>>>>> Maybe in the future we could have the drivers advertise their=20
>>> >>>> capabilities=20
>>> >>>>>> and if the allocator feels a request is not being satisfied (say=
=20
>>> Min=20
>>> >> was=20
>>> >>>>>> entered, but it not's supported by any storage plug-in) it can=
=20
>>> throw=20
>>> >> an=20
>>> >>>>>> exception.=20
>>> >>>>>>=20
>>> >>>>>>=20
>>> >>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <=20
>>> >>>>>> mike.tutkowski@solidfire.com> wrote:=20
>>> >>>>>>=20
>>> >>>>>>> Comments below in red.=20
>>> >>>>>>>=20
>>> >>>>>>> Thanks=20
>>> >>>>>>>=20
>>> >>>>>>>=20
>>> >>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <=20
>>> jburwell@basho.com>=20
>>> >>>> wrote:=20
>>> >>>>>>>=20
>>> >>>>>>>> Mike,=20
>>> >>>>>>>>=20
>>> >>>>>>>> Please see my comment in-line below.=20
>>> >>>>>>>>=20
>>> >>>>>>>> Thanks,=20
>>> >>>>>>>> -John=20
>>> >>>>>>>>=20
>>> >>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <=20
>>> >>>>>>>> mike.tutkowski@solidfire.com> wrote:=20
>>> >>>>>>>>=20
>>> >>>>>>>>> Hi John,=20
>>> >>>>>>>>>=20
>>> >>>>>>>>> I've put comments below in red.=20
>>> >>>>>>>>>=20
>>> >>>>>>>>> Thanks!=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <=20
>>> jburwell@basho.com=20
>>> >>>=20
>>> >>>>>>>> wrote:=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>> Mike,=20
>>> >>>>>>>>>>=20
>>> >>>>>>>>>> First and foremost, we must ensure that these two features a=
re=20
>>> >>>>>>>> mutually=20
>>> >>>>>>>>>> exclusive in 4.2. We don't want to find a configuration that=
=20
>>> >>>>>>>> contains both=20
>>> >>>>>>>>>> hypervisor and storage IOPS guarantees that leads to=20
>>> >>>> non-deterministic=20
>>> >>>>>>>>>> operations.=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>> Agreed=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>> Restricting QoS expression to be either hypervisor or storag=
e=20
>>> >>>> oriented=20
>>> >>>>>>>>>> solves the problem in short term. As I understand storage=20
>>> tags,=20
>>> >> we=20
>>> >>>>>>>> have no=20
>>> >>>>>>>>>> means of expressing this type of mutual exclusion. I wasn't=
=20
>>> >>>>>>>> necessarily=20
>>> >>>>>>>>>> intending that we implement this allocation model in 4.3, bu=
t=20
>>> >>>> instead,=20
>>> >>>>>>>>>> determine if this type model would be one we would want to=
=20
>>> support=20
>>> >>>> in=20
>>> >>>>>>>> the=20
>>> >>>>>>>>>> future. If so, I would encourage us to ensure that the data=
=20
>>> model=20
>>> >>>> and=20
>>> >>>>>>>>>> current implementation would not preclude evolution in that=
=20
>>> >>>>>>>> direction. My=20
>>> >>>>>>>>>> view is that this type of allocation model is what user's=20
>>> expect=20
>>> >> of=20
>>> >>>>>>>> "cloud"=20
>>> >>>>>>>>>> systems -- selecting the best available resource set to=20
>>> fulfill a=20
>>> >>>>>>>> set of=20
>>> >>>>>>>>>> system requirements.=20
>>> >>>>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>> I believe we have meet your requirement here in that what we'=
ve=20
>>> >>>>>>>> implemented=20
>>> >>>>>>>>> should not make refinement difficult in the future. If we don=
't=20
>>> >>>> modify=20
>>> >>>>>>>>> allocators for 4.2, but we do for 4.3, we've made relatively=
=20
>>> simple=20
>>> >>>>>>>> changes=20
>>> >>>>>>>>> to enhance the current functioning of the system.=20
>>> >>>>>>>>>=20
>>> >>>>>>>>=20
>>> >>>>>>>> Looking through both patches, I have to say that the aggregate=
d=20
>>> >> result=20
>>> >>>>>>>> seems a bit confusing. There are six new attributes for=20
>>> throttled=20
>>> >>>> I/O and=20
>>> >>>>>>>> two for provisioned IOPS with no obvious grouping. My concern=
=20
>>> is=20
>>> >> not=20
>>> >>>>>>>> technical, but rather, about maintainability. At minimum,=20
>>> Javadoc=20
>>> >>>> should=20
>>> >>>>>>>> be added explaining the two sets of attributes and their mutua=
l=20
>>> >>>> exclusion.=20
>>> >>>>>>>>=20
>>> >>>>>>>=20
>>> >>>>>>> I agree: We need JavaDoc to explain them and their mutual=20
>>> exclusion.=20
>>> >>>>>>>=20
>>> >>>>>>>=20
>>> >>>>>>>>=20
>>> >>>>>>>> The other part that is interesting is that throttled I/O=20
>>> provides=20
>>> >> both=20
>>> >>>>>>>> an IOPS and byte measured QoS as a rate where provisioned IOPS=
=20
>>> uses=20
>>> >> a=20
>>> >>>>>>>> range. In order to select the best available resource to=20
>>> fulfill a=20
>>> >>>> QoS, we=20
>>> >>>>>>>> would need to have the QoS expression normalized in terms of=
=20
>>> units=20
>>> >>>> (IOPS or=20
>>> >>>>>>>> bytes) and their expression (rate vs. range). If we want to=20
>>> >> achieve a=20
>>> >>>>>>>> model like I described, I think we would need to resolve this=
=20
>>> issue=20
>>> >>>> in 4.2=20
>>> >>>>>>>> to ensure a viable migration path.=20
>>> >>>>>>>=20
>>> >>>>>>>=20
>>> >>>>>>> I think we're not likely to be able to normalize the input for=
=20
>>> 4.2.=20
>>> >>>> Plus=20
>>> >>>>>>> people probably want to input the data in terms they're familia=
r=20
>>> with=20
>>> >>>> for=20
>>> >>>>>>> the products in question.=20
>>> >>>>>>>=20
>>> >>>>>>> Ideally we would fix the way we do storage tagging and let the=
=20
>>> user=20
>>> >>>> send=20
>>> >>>>>>> key/value pairs to each vendor that could be selected due to a=
=20
>>> given=20
>>> >>>>>>> storage tag. I'm still not sure that would solve it because wha=
t=20
>>> >>>> happens if=20
>>> >>>>>>> you change the storage tag of a given Primary Storage after=20
>>> having=20
>>> >>>> created=20
>>> >>>>>>> a Disk Offering?=20
>>> >>>>>>>=20
>>> >>>>>>> Basically storage tagging is kind of a mess and we should=20
>>> re-think=20
>>> >> it.=20
>>> >>>>>>>=20
>>> >>>>>>> Also, we need to have a way for the drivers to expose their=20
>>> supported=20
>>> >>>>>>> feature sets so the allocators can make good choices.=20
>>> >>>>>>>=20
>>> >>>>>>>=20
>>> >>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>>=20
>>> >>>>>>>>>> As I think through the implications of these requirements an=
d=20
>>> >>>> reflect=20
>>> >>>>>>>> on=20
>>> >>>>>>>>>> the reviews, I don't understand why they haven't already=20
>>> impacted=20
>>> >>>> the=20
>>> >>>>>>>>>> allocators and planners. As it stands, the current=20
>>> provisioned=20
>>> >> IOPS=20
>>> >>>>>>>> has no=20
>>> >>>>>>>>>> checks to ensure that the volumes are allocated to devices=
=20
>>> that=20
>>> >> have=20
>>> >>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I=20
>>> understand=20
>>> >>>> the=20
>>> >>>>>>>>>> current patch, we can overcommit storage resources --=20
>>> potentially=20
>>> >>>>>>>> causing=20
>>> >>>>>>>>>> none of the QoS obligations from being fulfilled. It seems=
=20
>>> to me=20
>>> >>>>>>>> that a=20
>>> >>>>>>>>>> DataStore supporting provisioned IOPS should express the=20
>>> maximum=20
>>> >>>> IOPS=20
>>> >>>>>>>> which=20
>>> >>>>>>>>>> it can support and some type of overcommitment factor. This=
=20
>>> >>>>>>>> information=20
>>> >>>>>>>>>> should be used by the storage allocators to determine the=20
>>> device=20
>>> >>>> best=20
>>> >>>>>>>> able=20
>>> >>>>>>>>>> to support the resources needs of a volume. It seems that a=
=20
>>> >> similar=20
>>> >>>>>>>> set of=20
>>> >>>>>>>>>> considerations would need to be added to the Hypervisor laye=
r=20
>>> >> which=20
>>> >>>>>>>>>> allocating a VM to a host to prevent oversubscription.=20
>>> >>>>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>> Yeah, for this first release, we just followed the path that=
=20
>>> was=20
>>> >>>>>>>> previously=20
>>> >>>>>>>>> established for other properties you see on dialogs in CS: Ju=
st=20
>>> >>>> because=20
>>> >>>>>>>>> they're there doesn't mean the vendor your VM is deployed to=
=20
>>> >> supports=20
>>> >>>>>>>> them.=20
>>> >>>>>>>>> It is then up to the admin to make sure he inputs, say, a=20
>>> storage=20
>>> >> tag=20
>>> >>>>>>>> that=20
>>> >>>>>>>>> confines the deployment only to storage that supports the=20
>>> selected=20
>>> >>>>>>>>> features. This is not ideal, but it's kind of the way=20
>>> CloudStack=20
>>> >>>> works=20
>>> >>>>>>>>> today.=20
>>> >>>>>>>>=20
>>> >>>>>>>> I understand the tag functionality, and the need for the=20
>>> >> administrator=20
>>> >>>>>>>> to very carefully construct offerings. My concern is that we=
=20
>>> over=20
>>> >>>>>>>> guarantee a resource's available IOPS. For the purposes of=20
>>> >>>> illustration,=20
>>> >>>>>>>> let's say we have a SolidFire, and the max IOPS for that devic=
e=20
>>> is=20
>>> >>>> 100000.=20
>>> >>>>>>>> We also know that we can safely oversubscribe by 50%.=20
>>> Therefore,=20
>>> >> we=20
>>> >>>>>>>> need to ensure that we don't allocate more than 150,000=20
>>> guaranteed=20
>>> >>>> IOPS=20
>>> >>>>>>>> from that device. Intuitively, it seems like the DataStore=20
>>> >>>> configuration=20
>>> >>>>>>>> should have a max assignable IOPS and overcommitment factor.=
=20
>>> As we=20
>>> >>>>>>>> allocate volumes and attach VMs, we need to ensure that=20
>>> guarantee=20
>>> >>>> more IOPS=20
>>> >>>>>>>> exceed the configured maximum for a DataStore. Does that make=
=20
>>> >> sense?=20
>>> >>>>>>>>=20
>>> >>>>>>>=20
>>> >>>>>>> I think that's a good idea for a future enhancement. I'm not ev=
en=20
>>> >> sure=20
>>> >>>> I=20
>>> >>>>>>> can query the SAN to find out how many IOPS safely remain. I'd=
=20
>>> have=20
>>> >> to=20
>>> >>>> get=20
>>> >>>>>>> all of the min values for all of the volumes on the SAN and tot=
al=20
>>> >> them=20
>>> >>>> up,=20
>>> >>>>>>> I suppose, and subtract it from the total (user facing) support=
ed=20
>>> >> IOPS=20
>>> >>>> of=20
>>> >>>>>>> the system.=20
>>> >>>>>>>=20
>>> >>>>>>>=20
>>> >>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>>=20
>>> >>>>>>>>>> Another question occurs to me -- should we allow non-QoS=20
>>> resources=20
>>> >>>> to=20
>>> >>>>>>>> be=20
>>> >>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For=20
>>> >> provisioned=20
>>> >>>>>>>> IOPS, I=20
>>> >>>>>>>>>> think a side effect of the current implementation is SolidFi=
re=20
>>> >>>> volumes=20
>>> >>>>>>>>>> always have a QoS. However, for hypervisor throttled I/O, it=
=20
>>> >> seems=20
>>> >>>>>>>>>> entirely possible for non-QoS VMs to allocated side-by-side=
=20
>>> with=20
>>> >> QoS=20
>>> >>>>>>>> VMs.=20
>>> >>>>>>>>>> In this scenario, a greedy, unbounded VM could potentially=
=20
>>> starve=20
>>> >>>> out=20
>>> >>>>>>>> the=20
>>> >>>>>>>>>> other VMs on the host -- preventing the QoSes defined the=20
>>> >> collocated=20
>>> >>>>>>>> VMs=20
>>> >>>>>>>>>> from being fulfilled.=20
>>> >>>>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>> You can make SolidFire volumes (inside and outside of CS) and=
=20
>>> not=20
>>> >>>>>>>> specify=20
>>> >>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the=
=20
>>> >>>> defaults=20
>>> >>>>>>>> we=20
>>> >>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN, yo=
u=20
>>> will=20
>>> >>>>>>>> have=20
>>> >>>>>>>>> your Mins met.=20
>>> >>>>>>>>>=20
>>> >>>>>>>>> It sounds like you're perhaps looking for a storage tags=20
>>> exclusions=20
>>> >>>>>>>> list,=20
>>> >>>>>>>>> which might be nice to have at some point (i.e. don't deploy =
my=20
>>> >>>> volume=20
>>> >>>>>>>> to=20
>>> >>>>>>>>> storage with these following tags).=20
>>> >>>>>>>>=20
>>> >>>>>>>> I don't like the idea of a storage tags exclusion list as it=
=20
>>> would=20
>>> >>>>>>>> complicate component assembly. It would require a storage=20
>>> plugin to=20
>>> >>>>>>>> anticipate all of the possible component assemblies and=20
>>> determine=20
>>> >> the=20
>>> >>>>>>>> invalid relationships. I prefer that drivers express their=20
>>> >>>> capabilities=20
>>> >>>>>>>> which can be matched to a set of requested requirements.=20
>>> >>>>>>>>=20
>>> >>>>>>>=20
>>> >>>>>>> I'm not sure why a storage plug-in would care about inclusion o=
r=20
>>> >>>>>>> exclusion lists. It just needs to advertise its functionality i=
n=20
>>> a=20
>>> >> way=20
>>> >>>> the=20
>>> >>>>>>> allocator understands.=20
>>> >>>>>>>=20
>>> >>>>>>>=20
>>> >>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it only=
=20
>>> >> limits=20
>>> >>>>>>>> IOPS,=20
>>> >>>>>>>>> it does not solve the Noisy Neighbor problem. Only a system=
=20
>>> with=20
>>> >>>>>>>> guaranteed=20
>>> >>>>>>>>> minimum IOPS does this.=20
>>> >>>>>>>>=20
>>> >>>>>>>> As I said, for SolidFire, it sounds like this problem does not=
=20
>>> >> exist.=20
>>> >>>>>>>> However, I am concerned with the more general case as we=20
>>> supported=20
>>> >>>> more=20
>>> >>>>>>>> devices with provisioned IOPS.=20
>>> >>>>>>>>=20
>>> >>>>>>>=20
>>> >>>>>>> Post 4.2 we need to investigate a way to pass vendor-specific=
=20
>>> values=20
>>> >> to=20
>>> >>>>>>> drivers. Min and Max and pretty industry standard for provision=
ed=20
>>> >>>> IOPS, but=20
>>> >>>>>>> what if you break them out by read and write or do something=20
>>> else? We=20
>>> >>>> need=20
>>> >>>>>>> a more general mechanism.=20
>>> >>>>>>>=20
>>> >>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>>=20
>>> >>>>>>>>>> In my opinion, we need to ensure that hypervisor throttled=
=20
>>> I/O=20
>>> >> and=20
>>> >>>>>>>>>> storage provisioned IOPS are mutually exclusive per volume.=
=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>> Agreed=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>> We also need to understand the implications of these QoS=20
>>> >> guarantees=20
>>> >>>> on=20
>>> >>>>>>>>>> operation of the system to ensure that the underlying hardwa=
re=20
>>> >>>>>>>> resources=20
>>> >>>>>>>>>> can fulfill them. Given the time frame, we will likely be=20
>>> forced=20
>>> >> to=20
>>> >>>>>>>> make=20
>>> >>>>>>>>>> compromises to achieve these goals, and refine the=20
>>> implementation=20
>>> >> in=20
>>> >>>>>>>> 4.3.=20
>>> >>>>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>> I agree, John. I also think you've come up with some great=20
>>> ideas=20
>>> >> for=20
>>> >>>>>>>> 4.3. :)=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>>=20
>>> >>>>>>>>>> Thanks,=20
>>> >>>>>>>>>> -John=20
>>> >>>>>>>>>>=20
>>> >>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <=20
>>> >>>>>>>> mike.tutkowski@solidfire.com>=20
>>> >>>>>>>>>> wrote:=20
>>> >>>>>>>>>>=20
>>> >>>>>>>>>>> Hi,=20
>>> >>>>>>>>>>>=20
>>> >>>>>>>>>>> Yeah, Alex, I think that's the way we were planning (with=
=20
>>> storage=20
>>> >>>>>>>> tags).=20
>>> >>>>>>>>>> I=20
>>> >>>>>>>>>>> believe John was just throwing out an idea that - in=20
>>> addition to=20
>>> >>>>>>>> storage=20
>>> >>>>>>>>>>> tags - we could look into these allocators (storage QoS bei=
ng=20
>>> >>>>>>>> preferred,=20
>>> >>>>>>>>>>> then hypervisor QoS if storage QoS is not available, but=20
>>> >> hypervisor=20
>>> >>>>>>>> QoS=20
>>> >>>>>>>>>> is).=20
>>> >>>>>>>>>>>=20
>>> >>>>>>>>>>> I think John's concern is that you can enter in values for=
=20
>>> Wei's=20
>>> >>>> and=20
>>> >>>>>>>> my=20
>>> >>>>>>>>>>> feature that are not honored by other vendors (at least=20
>>> yet), so=20
>>> >> he=20
>>> >>>>>>>> was=20
>>> >>>>>>>>>>> hoping - in addition to storage tags - for the allocators t=
o=20
>>> >> prefer=20
>>> >>>>>>>> these=20
>>> >>>>>>>>>>> vendors when these fields are filled in. As it stands today=
=20
>>> in=20
>>> >>>>>>>>>> CloudStack,=20
>>> >>>>>>>>>>> we already have this kind of an issue with other features=
=20
>>> (fields=20
>>> >>>> in=20
>>> >>>>>>>>>>> dialogs for features that not all vendors support). Perhaps=
=20
>>> post=20
>>> >>>> 4.2=20
>>> >>>>>>>> we=20
>>> >>>>>>>>>>> could look into generic name/value pairs (this is how=20
>>> OpenStack=20
>>> >>>>>>>> addresses=20
>>> >>>>>>>>>>> the issue).=20
>>> >>>>>>>>>>>=20
>>> >>>>>>>>>>> Honestly, I think we're too late in the game (two weeks unt=
il=20
>>> >> code=20
>>> >>>>>>>>>> freeze)=20
>>> >>>>>>>>>>> to go too deeply down that path in 4.2.=20
>>> >>>>>>>>>>>=20
>>> >>>>>>>>>>> It's probably better if we - at least for 4.2 - keep Wei's=
=20
>>> fields=20
>>> >>>>>>>> and my=20
>>> >>>>>>>>>>> fields as is, make sure only one or the other feature has=
=20
>>> data=20
>>> >>>>>>>> entered=20
>>> >>>>>>>>>> for=20
>>> >>>>>>>>>>> it (or neither), and call it good for 4.2.=20
>>> >>>>>>>>>>>=20
>>> >>>>>>>>>>> Then let's step back and look into a more general-purpose=
=20
>>> design=20
>>> >>>>>>>> that can=20
>>> >>>>>>>>>>> be applied throughout CloudStack where we have these=20
>>> situations.=20
>>> >>>>>>>>>>>=20
>>> >>>>>>>>>>> What do you think?=20
>>> >>>>>>>>>>>=20
>>> >>>>>>>>>>>=20
>>> >>>>>>>>>>>=20
>>> >>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <=20
>>> >> jburwell@basho.com>=20
>>> >>>>>>>>>> wrote:=20
>>> >>>>>>>>>>>=20
>>> >>>>>>>>>>>> Mike,=20
>>> >>>>>>>>>>>>=20
>>> >>>>>>>>>>>> I just published my review @=20
>>> >> https://reviews.apache.org/r/11479/.=20
>>> >>>>>>>>>>>>=20
>>> >>>>>>>>>>>> I apologize for the delay,=20
>>> >>>>>>>>>>>> -John=20
>>> >>>>>>>>>>>>=20
>>> >>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <=20
>>> >>>>>>>>>> mike.tutkowski@solidfire.com>=20
>>> >>>>>>>>>>>> wrote:=20
>>> >>>>>>>>>>>>=20
>>> >>>>>>>>>>>>> No problem, John.=20
>>> >>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>> I still want to re-review it by myself before coming up=
=20
>>> with a=20
>>> >>>> new=20
>>> >>>>>>>>>> patch=20
>>> >>>>>>>>>>>>> file.=20
>>> >>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be=
=20
>>> checked=20
>>> >>>> in=20
>>> >>>>>>>> and=20
>>> >>>>>>>>>>>>> merge those into mine before generating a new patch file?=
=20
>>> >>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <=20
>>> >>>> jburwell@basho.com=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>>>> wrote:=20
>>> >>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>> Mike,=20
>>> >>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>> I just realized that I forgot to publish my review. I am=
=20
>>> >>>> offline=20
>>> >>>>>>>> ATM,=20
>>> >>>>>>>>>>>>>> but I will publish it in the next couple of hours.=20
>>> >>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>> Do you plan to update your the patch in Review Board?=20
>>> >>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>> Sorry for the oversight,=20
>>> >>>>>>>>>>>>>> -John=20
>>> >>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski=20
>>> >>>>>>>>>>>>>> <mike.tutkowski@solidfire.com> wrote:=20
>>> >>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading=
=20
>>> this :)=20
>>> >>>> ),=20
>>> >>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the=
=20
>>> areas=20
>>> >> we=20
>>> >>>>>>>> wanted=20
>>> >>>>>>>>>>>>>>> addressed.=20
>>> >>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or=20
>>> >> afternoon,=20
>>> >>>>>>>> then=20
>>> >>>>>>>>>>>> send=20
>>> >>>>>>>>>>>>>> in=20
>>> >>>>>>>>>>>>>>> another patch.=20
>>> >>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>> Thanks for all the work on this everyone!=20
>>> >>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <=20
>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:=20
>>> >>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>> Sure, that sounds good.=20
>>> >>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <=20
>>> >>>>>>>> ustcweizhou@gmail.com>=20
>>> >>>>>>>>>>>>>> wrote:=20
>>> >>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>> Hi Mike,=20
>>> >>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts i=
n=20
>>> Java=20
>>> >>>>>>>> code,=20
>>> >>>>>>>>>>>>>> except=20
>>> >>>>>>>>>>>>>>>>> the cloudstack UI.=20
>>> >>>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling=
=20
>>> branch=20
>>> >>>> into=20
>>> >>>>>>>>>>>> master=20
>>> >>>>>>>>>>>>>>>>> this=20
>>> >>>>>>>>>>>>>>>>> week, so that you can develop based on it.=20
>>> >>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>> -Wei=20
>>> >>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mike.tutkowski@solidfire.co=
m=20
>>> >=20
>>> >>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>> Hey John,=20
>>> >>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the=20
>>> object_store=20
>>> >>>>>>>> branch,=20
>>> >>>>>>>>>> but=20
>>> >>>>>>>>>>>> -=20
>>> >>>>>>>>>>>>>> as=20
>>> >>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge th=
e=20
>>> >>>>>>>> SolidFire=20
>>> >>>>>>>>>>>> branch=20
>>> >>>>>>>>>>>>>>>>> into=20
>>> >>>>>>>>>>>>>>>>>> the object_store branch before object_store goes int=
o=20
>>> >>>> master.=20
>>> >>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into th=
is=20
>>> >> merge=20
>>> >>>>>>>>>>>> strategy.=20
>>> >>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.=20
>>> >>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <=20
>>> >>>>>>>>>> jburwell@basho.com>=20
>>> >>>>>>>>>>>>>>>>> wrote:=20
>>> >>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>> Mike,=20
>>> >>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The=20
>>> >>>>>>>>>> disk_io_throttling,=20
>>> >>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number=
=20
>>> of=20
>>> >>>>>>>>>> overlapping=20
>>> >>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between the=
=20
>>> >>>> patches=20
>>> >>>>>>>> to=20
>>> >>>>>>>>>> be=20
>>> >>>>>>>>>>>> as=20
>>> >>>>>>>>>>>>>>>>>>> follows:=20
>>> >>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling=20
>>> >>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects of=
=20
>>> >>>> SolidFire=20
>>> >>>>>>>> are=20
>>> >>>>>>>>>>>>>>>>> additive=20
>>> >>>>>>>>>>>>>>>>>>> to the object_store branch or there are circular=20
>>> >> dependency=20
>>> >>>>>>>>>> between=20
>>> >>>>>>>>>>>>>>>>> the=20
>>> >>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph,=
=20
>>> we=20
>>> >> can=20
>>> >>>>>>>>>>>> determine=20
>>> >>>>>>>>>>>>>>>>> the=20
>>> >>>>>>>>>>>>>>>>>>> best approach to land the changes in master.=20
>>> >>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>> Thanks,=20
>>> >>>>>>>>>>>>>>>>>>> -John=20
>>> >>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <=20
>>> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>=20
>>> >>>>>>>>>>>>>>>>>>> wrote:=20
>>> >>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code=
=20
>>> into=20
>>> >> his=20
>>> >>>>>>>> branch=20
>>> >>>>>>>>>>>>>>>>> before=20
>>> >>>>>>>>>>>>>>>>>>>> going into master, I am good with that.=20
>>> >>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code=20
>>> after his=20
>>> >>>>>>>> merge=20
>>> >>>>>>>>>> and=20
>>> >>>>>>>>>>>>>>>>> we=20
>>> >>>>>>>>>>>>>>>>>> can=20
>>> >>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.=20
>>> >>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <=
=20
>>> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:=20
>>> >>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:=
=20
>>> >>>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor=20
>>> code in=20
>>> >>>> the=20
>>> >>>>>>>>>>>> storage=20
>>> >>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage=
=20
>>> >> plug-in,=20
>>> >>>>>>>> which=20
>>> >>>>>>>>>>>>>>>>>>> currently=20
>>> >>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use=
=20
>>> >> (although=20
>>> >>>>>>>> it=20
>>> >>>>>>>>>> does=20
>>> >>>>>>>>>>>>>>>>> not=20
>>> >>>>>>>>>>>>>>>>>>> know=20
>>> >>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is=20
>>> actually=20
>>> >> in=20
>>> >>>>>>>> use))=20
>>> >>>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>> 2) managed=3Dtrue or managed=3Dfalse can be place=
d in=20
>>> the=20
>>> >> url=20
>>> >>>>>>>> field=20
>>> >>>>>>>>>>>> (if=20
>>> >>>>>>>>>>>>>>>>>> not=20
>>> >>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is store=
d=20
>>> in=20
>>> >> the=20
>>> >>>>>>>>>>>>>>>>>>>>> storage_pool_details table.=20
>>> >>>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the=20
>>> hypervisor=20
>>> >> in=20
>>> >>>>>>>>>>>>>>>>> question, we=20
>>> >>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the=
=20
>>> place=20
>>> >> of=20
>>> >>>>>>>> the=20
>>> >>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).=20
>>> >>>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor=
=20
>>> >> checks=20
>>> >>>>>>>> for=20
>>> >>>>>>>>>> the=20
>>> >>>>>>>>>>>>>>>>>> managed=20
>>> >>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary=20
>>> >> hypervisor=20
>>> >>>>>>>> data=20
>>> >>>>>>>>>>>>>>>>>>> structure is=20
>>> >>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command=20
>>> executes to=20
>>> >>>>>>>> attach=20
>>> >>>>>>>>>> the=20
>>> >>>>>>>>>>>>>>>>>>> volume.=20
>>> >>>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked t=
o=20
>>> >>>> detach a=20
>>> >>>>>>>>>>>> volume,=20
>>> >>>>>>>>>>>>>>>>>> the=20
>>> >>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor da=
ta=20
>>> >>>>>>>> structure=20
>>> >>>>>>>>>> is=20
>>> >>>>>>>>>>>>>>>>>>> removed.=20
>>> >>>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOP=
S=20
>>> in=20
>>> >> 4.2=20
>>> >>>>>>>>>> unless=20
>>> >>>>>>>>>>>>>>>>> it is=20
>>> >>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. I=
f=20
>>> we=20
>>> >>>> have=20
>>> >>>>>>>> some=20
>>> >>>>>>>>>>>>>>>>> idea,=20
>>> >>>>>>>>>>>>>>>>>>>>> that'd be cool.=20
>>> >>>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>> Thanks!=20
>>> >>>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <=
=20
>>> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:=20
>>> >>>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while=20
>>> avoiding=20
>>> >>>>>>>>>>>> implementation=20
>>> >>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details=
=20
>>> >> field.=20
>>> >>>> I=20
>>> >>>>>>>>>> think=20
>>> >>>>>>>>>>>>>>>>> we=20
>>> >>>>>>>>>>>>>>>>>>> should=20
>>> >>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation=20
>>> regarding=20
>>> >>>> the=20
>>> >>>>>>>>>>>>>>>>> expected=20
>>> >>>>>>>>>>>>>>>>>>> format=20
>>> >>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details=
=20
>>> are=20
>>> >>>> not=20
>>> >>>>>>>>>>>>>>>>> returned on=20
>>> >>>>>>>>>>>>>>>>>>> the=20
>>> >>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or=20
>>> >> listStoragePool=20
>>> >>>>>>>>>>>> response.=20
>>> >>>>>>>>>>>>>>>>>> Why=20
>>> >>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be=20
>>> useful=20
>>> >>>> for=20
>>> >>>>>>>>>>>> clients=20
>>> >>>>>>>>>>>>>>>>> to=20
>>> >>>>>>>>>>>>>>>>>> be=20
>>> >>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details=20
>>> field."=20
>>> >>>>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS=
=20
>>> here.=20
>>> >>>>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk=20
>>> >>>> Offering-by-Disk=20
>>> >>>>>>>>>>>> Offering=20
>>> >>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have=
=20
>>> to be=20
>>> >>>>>>>> able to=20
>>> >>>>>>>>>>>>>>>>>> associate=20
>>> >>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a=20
>>> disk_offering_details=20
>>> >>>> table.=20
>>> >>>>>>>>>> Maybe=20
>>> >>>>>>>>>>>>>>>>> it=20
>>> >>>>>>>>>>>>>>>>>>> could=20
>>> >>>>>>>>>>>>>>>>>>>>>> go there?=20
>>> >>>>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst=
=20
>>> IOPS=20
>>> >> in=20
>>> >>>>>>>> the=20
>>> >>>>>>>>>> GUI=20
>>> >>>>>>>>>>>>>>>>> if=20
>>> >>>>>>>>>>>>>>>>>>> it's=20
>>> >>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in th=
e=20
>>> DB.=20
>>> >>>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>> --=20
>>> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*=20
>>> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*=20
>>> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com=20
>>> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302=20
>>> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<=20
>>> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=3Dpla=
y>=20
>>> >>>>>>>>>>>>>>>>>>>>> *=E2=84=A2*=20
>>> >>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>>> --=20
>>> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*=20
>>> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*=20
>>> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com=20
>>> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302=20
>>> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the=20
>>> >>>>>>>>>>>>>>>>>>>> cloud<=20
>>> >> http://solidfire.com/solution/overview/?video=3Dplay>=20
>>> >>>>>>>>>>>>>>>>>>>> *=E2=84=A2*=20
>>> >>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>>> --=20
>>> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*=20
>>> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*=20
>>> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com=20
>>> >>>>>>>>>>>>>>>>>> o: 303.746.7302=20
>>> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the=20
>>> >>>>>>>>>>>>>>>>>> cloud<=20
>>> http://solidfire.com/solution/overview/?video=3Dplay>=20
>>> >>>>>>>>>>>>>>>>>> *=E2=84=A2*=20
>>> >>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>> --=20
>>> >>>>>>>>>>>>>>>> *Mike Tutkowski*=20
>>> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*=20
>>> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com=20
>>> >>>>>>>>>>>>>>>> o: 303.746.7302=20
>>> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<=20
>>> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=3Dplay>=20
>>> >>>>>>>>>>>>>>>> *=E2=84=A2*=20
>>> >>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>>> --=20
>>> >>>>>>>>>>>>>>> *Mike Tutkowski*=20
>>> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*=20
>>> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com=20
>>> >>>>>>>>>>>>>>> o: 303.746.7302=20
>>> >>>>>>>>>>>>>>> Advancing the way the world uses the=20
>>> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=3Dp=
lay=20
>>> >=20
>>> >>>>>>>>>>>>>>> *=E2=84=A2*=20
>>> >>>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>>=20
>>> >>>>>>>>>>>>> --=20
>>> >>>>>>>>>>>>> *Mike Tutkowski*=20
>>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*=20
>>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com=20
>>> >>>>>>>>>>>>> o: 303.746.7302=20
>>> >>>>>>>>>>>>> Advancing the way the world uses the=20
>>> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=3Dpla=
y>=20
>>> >>>>>>>>>>>>> *=E2=84=A2*=20
>>> >>>>>>>>>>>>=20
>>> >>>>>>>>>>>>=20
>>> >>>>>>>>>>>=20
>>> >>>>>>>>>>>=20
>>> >>>>>>>>>>> --=20
>>> >>>>>>>>>>> *Mike Tutkowski*=20
>>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*=20
>>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com=20
>>> >>>>>>>>>>> o: 303.746.7302=20
>>> >>>>>>>>>>> Advancing the way the world uses the=20
>>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=3Dplay>=
=20
>>> >>>>>>>>>>> *=E2=84=A2*=20
>>> >>>>>>>>>>=20
>>> >>>>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>>=20
>>> >>>>>>>>> --=20
>>> >>>>>>>>> *Mike Tutkowski*=20
>>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*=20
>>> >>>>>>>>> e: mike.tutkowski@solidfire.com=20
>>> >>>>>>>>> o: 303.746.7302=20
>>> >>>>>>>>> Advancing the way the world uses the=20
>>> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=3Dplay>=
=20
>>> >>>>>>>>> *=E2=84=A2*=20
>>> >>>>>>>>=20
>>> >>>>>>>>=20
>>> >>>>>>>=20
>>> >>>>>>>=20
>>> >>>>>>> --=20
>>> >>>>>>> *Mike Tutkowski*=20
>>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*=20
>>> >>>>>>> e: mike.tutkowski@solidfire.com=20
>>> >>>>>>> o: 303.746.7302=20
>>> >>>>>>> Advancing the way the world uses the cloud<=20
>>> >>>> http://solidfire.com/solution/overview/?video=3Dplay>=20
>>> >>>>>>> *=E2=84=A2*=20
>>> >>>>>>>=20
>>> >>>>>>=20
>>> >>>>>>=20
>>> >>>>>>=20
>>> >>>>>> --=20
>>> >>>>>> *Mike Tutkowski*=20
>>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*=20
>>> >>>>>> e: mike.tutkowski@solidfire.com=20
>>> >>>>>> o: 303.746.7302=20
>>> >>>>>> Advancing the way the world uses the cloud<=20
>>> >>>> http://solidfire.com/solution/overview/?video=3Dplay>=20
>>> >>>>>> *=E2=84=A2*=20
>>> >>>>>>=20
>>> >>>>>=20
>>> >>>>>=20
>>> >>>>>=20
>>> >>>>> --=20
>>> >>>>> *Mike Tutkowski*=20
>>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*=20
>>> >>>>> e: mike.tutkowski@solidfire.com=20
>>> >>>>> o: 303.746.7302=20
>>> >>>>> Advancing the way the world uses the=20
>>> >>>>> cloud<http://solidfire.com/solution/overview/?video=3Dplay>=20
>>> >>>>> *=E2=84=A2*=20
>>> >>>>=20
>>> >>>>=20
>>> >>>=20
>>> >>>=20
>>> >>> --=20
>>> >>> *Mike Tutkowski*=20
>>> >>> *Senior CloudStack Developer, SolidFire Inc.*=20
>>> >>> e: mike.tutkowski@solidfire.com=20
>>> >>> o: 303.746.7302=20
>>> >>> Advancing the way the world uses the=20
>>> >>> cloud<http://solidfire.com/solution/overview/?video=3Dplay>=20
>>> >>> *=E2=84=A2*=20
>>> >>=20
>>> >>=20
>>> >=20
>>> >=20
>>> > --=20
>>> > *Mike Tutkowski*=20
>>> > *Senior CloudStack Developer, SolidFire Inc.*=20
>>> > e: mike.tutkowski@solidfire.com=20
>>> > o: 303.746.7302=20
>>> > Advancing the way the world uses the=20
>>> > cloud<http://solidfire.com/solution/overview/?video=3Dplay>=20
>>> > *=E2=84=A2*=20
>>>=20
>>>=20
>>=20
>>=20
>> --=20
>> *Mike Tutkowski*=20
>> *Senior CloudStack Developer, SolidFire Inc.*=20
>> e: mike.tutkowski@solidfire.com=20
>> o: 303.746.7302=20
>> Advancing the way the world uses the cloud<http://solidfire.com/solution=
/overview/?video=3Dplay>=20
>> *=E2=84=A2*=20
>>=20
>=20
>=20
>=20
> --=20
> *Mike Tutkowski*=20
> *Senior CloudStack Developer, SolidFire Inc.*=20
> e: mike.tutkowski@solidfire.com=20
> o: 303.746.7302=20
> Advancing the way the world uses the cloud<http://solidfire.com/solution/=
overview/?video=3Dplay>=20
> *=E2=84=A2*=20
>=20


--=20
*Mike Tutkowski*=20
*Senior CloudStack Developer, SolidFire Inc.*=20
e: mike.tutkowski@solidfire.com=20
o: 303.746.7302=20
Advancing the way the world uses the=20
cloud<http://solidfire.com/solution/overview/?video=3Dplay>=20
*=E2=84=A2*=20


------=_Part_32874771_1788076594.1371234011922--