Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Received-SPF: pass (athena.apache.org: domain of Nitin.Mehta@citrix.com
 designates 203.166.19.134 as permitted sender)
From: Nitin Mehta <Nitin.Mehta@citrix.com>
To: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>
Subject: Re: Object_Store storage refactor Meeting Notes
Thread-Topic: Object_Store storage refactor Meeting Notes
Thread-Index: AQHOaFR/UJEoEEsJgk+4LBBWE9ShSpk0tGcA
Date: Fri, 14 Jun 2013 08:16:37 +0000
Message-ID: <CDE0C0CC.1DDB2%nitin.mehta@citrix.com>
In-Reply-To: <56AAC636-B931-44D7-B21B-866AA2029825@basho.com>
Accept-Language: en-US
Content-Language: en-US
user-agent: Microsoft-MacOutlook/14.2.0.120402
Content-Type: text/plain; charset="windows-1254"
Content-ID: <4DCB2637BE62DB4B9C37767A71195BB9@citrix.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0


On 13/06/13 10:08 PM, "John Burwell" <jburwell@basho.com> wrote:

>All,
>
>Edison Su, Min Chen, Animesh Chaturvedi, and myself met via
>teleconference on 11 June 2013 @ 1:30 PM EDT.  The goal of the meeting
>was determine the path forward for merging the object_store branch by the
>4.2 freeze date, 30 June 2013.  The conversation focused on the following
>topics:
>
>	* Staging area mechanism
>	* Removing dependencies from the Storage to the Hypervisor layer
>	* Dependencies of other patches on object_store
>	* QA's desire to start testing the patch ASAP
>
>Min, Edison, and I agreed that the staging mechanism must age out files
>and use a reference count to ensure that file in-use are not prematurely
>purged.  While we agree that some form of reservation system is required,
>Edison is concerned that it will be too conservative and create
>bottlenecks. =20

Can you please elaborate on the fact that it is too conservative - we just
can't purge the files when they are still in use correct ? We can use a
combination of LRU + reference count, trying to purge the LRU files if
their reference count <=3D 0 as a start ?

>
>As we delved deeper into the subject of the storage to hypervisor
>dependencies and the reservation mechanism, we determined that NFS
>storage would still need to be the size of the secondary storage data
>set.  Since the hypervisor layer has not been completely fitted to the
>new storage layer, NFS would be still required for a number of
>operations.  Based on this realization, we decided to de-scope the
>staging mechanism, and leave the 4.2 object store functionality the same
>as 4.1.  Therefore, NFS will remain the secondary storage of record, and
>object storage will serve as backup/multi-zone sync.

I am not sure how we can comment its going to be the same as 4.1 - is it
from the end user perspective ? The internal semantics and their flow have
changed. This needs to be elaborated and properly documented. Also I am
not sure if the feature is supported on the upgrade path or is it ? Need
more documentation here.


>In 4.3, we will fit the hypervisor layer for the new storage layer which
>will allow object stores to server as secondary storage.  This work will
>include removing the storage to hypervisor dependencies.  For 4.2, Edison
>and Min have implemented the critical foundation necessary to establish
>our next generation storage layer.  There simply was not enough time in
>this development cycle to make these changes and fit the hypervisor layer.
>
>Due to the size of the patch, Animesh voiced QA's concerned regarding
>test scope and impact.  As such, we want to get the merge completed as
>soon as possible to allow testing to begin.  We discussed breaking up the
>patch, but we could not devise a reasonable set of chunks there were both
>isolated and significantly testable.  Therefore, the patch can only be
>delivered in its current state.  We also walked through potential
>dependencies between the storage framework changes and the solidfire
>branch.  It was determined that these two merges could occur
>independently.
>
>Finally, Animesh is going to setup a meeting at Citrix's Santa Clara
>office on 26 June 2013 (the day after Collab ends) to discuss the path
>forward for 4.3 and work through a high-level design/approach to fitting
>the hypervisor layer to exploit the new storage capabilities.  Details
>will be published to the dev mailing list.
>
>Thanks,
>-John
>
>On Jun 11, 2013, at 2:08 AM, Min Chen <min.chen@citrix.com> wrote:
>
>> It is 11th June. John is not free between 9:15am to 10am, that is why we
>> schedule it at 10:30am.
>>=20
>> Thanks
>> -min
>>=20
>> On 6/10/13 10:52 PM, "Nitin Mehta" <Nitin.Mehta@citrix.com> wrote:
>>=20
>>> Hi Min,
>>> When you say tomorrow, what date is it 11th June or 12th ? Can the
>>>time be
>>> preponed by an hour please - its too late here ?
>>>=20
>>> Thanks,
>>> -Nitin
>>>=20
>>> On 11/06/13 11:06 AM, "Min Chen" <min.chen@citrix.com> wrote:
>>>=20
>>>> Hi there,
>>>>=20
>>>> To reach consensus on some remaining NFS cache issues on object_store
>>>> storage refactor work in a more effective manner, John, Edison and I
>>>>have
>>>> scheduled a GoToMeeting tomorrow to discuss them over the phone, any
>>>> interested parties are welcome to join and brainstorm. Here are
>>>>detailed
>>>> GTM information:
>>>>=20
>>>> Meeting Time: 10:30 AM =AD 12:30 PM PST
>>>>=20
>>>> Meeting Details:
>>>>=20
>>>> 1.  Please join my meeting.
>>>> https://www1.gotomeeting.com/join/188620897
>>>>=20
>>>> 2.  Use your microphone and speakers (VoIP) - a headset is
>>>>recommended.
>>>> Or, call in using your telephone.
>>>>=20
>>>> United States: +1 (626) 521-0017
>>>> United States (toll-free): 1 877 309 2070
>>>>=20
>>>> Access Code: 188-620-897
>>>> Audio PIN: Shown after joining the meeting
>>>>=20
>>>> Meeting ID: 188-620-897
>>>>=20
>>>> GoToMeeting=AE
>>>> Online Meetings Made Easy=AE
>>>>=20
>>>> Not at your computer? Click the link to join this meeting from your
>>>> iPhone=AE, iPad=AE or Android=AE device via the GoToMeeting app.
>>>>=20
>>>> Thanks
>>>> -min
>>>=20
>>=20
>