incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Likitha Shetty <likitha.she...@citrix.com>
Subject RE: [ASFCS41] AWS-Style Regions
Date Thu, 10 Jan 2013 13:04:07 GMT
Please find my answers to the AWSAPI related query inline.

Thank you,
Likitha
>-----Original Message-----
>From: Kishan Kavala [mailto:Kishan.Kavala@citrix.com]
>Sent: Thursday, January 10, 2013 5:03 PM
>To: Sangeetha Hariharan; cloudstack-dev@incubator.apache.org
>Subject: RE: [ASFCS41] AWS-Style Regions
>
>Sangeetha,
>  Please find my response inline.
>
>-----Original Message-----
>From: Sangeetha Hariharan
>Sent: Thursday, 10 January 2013 7:08 AM
>To: cloudstack-dev@incubator.apache.org; Kishan Kavala
>Subject: RE: [ASFCS41] AWS-Style Regions
>
>Kishan,
>
>Can you Explain the actual Flow that is involved in the following use case in FS:
>
>We have a set up with 1 management server serving 1 region with 1 zone.
>There are few domains, accounts and projects that have been already been
>created.
>Create few resources as these accounts.
>
>Now I want to install another management server and make it part of the same
>cloud as the above management server  to service another region.
>
>How will I make this management server join the existing cloud ? What is the
>exact API call that will be made ? This should result in all the existing domains,
>accounts and projects to be duplicated in this management servers DB. How will
>this get done?
>[KK] addRegion API should be used to add a new region to the existing Region.
>This will not immediately copy all the existing domains and accounts to the new
>region. They will be copied on-demand whenever these resources are referred
>in the new Region. Resources added after region addition, will be propagated to
>all Regions immediately
>
>Also how different will the above sequence of events be when a new
>management server is connecting for the first time to a cloud that has multiple
>management severs each servicing 1 region?
>[KK] addRegion API call should be made to each of the existing regions. Rest of
>the process will remain same.
>
>Could you please point us to the latest build from the feature branch , where we
>can test parts of the feature that is available so far to get a better
>understanding?
>[KK] Feature is being developed in regions branch. I don't think there was any
>build created so far from regions branch.
>
>I have the following questions after reviewing the FS ( I have included the
>questions that was sent out earlier as well) :
>
>
>1. PRD states - "Each Region should have access to an object store. The object
>store should be common to all Zones within the Region. I.e., any object stored
>from a Zone should be visible across all the other zones within the region"
>
>Does this mean that we will have 1 Secondary Storage for the entire region and
>this would mean only 1 SSVM/region.
>
>[KK] This will be part of S3 based secondary storage feature CLOUDSTACK-714.
>Nitin should be able to answer this.
>
>In "Assumptions" section - following is state:
>
>Object Store: This feature assumes that S3 like object store is available. Either
>implemented in CloudStack or through integrations
>
>Wanted to confirm that we should be able to use NFS for Secondary Storage in
>such cases where we will share the same Secondary storage for multiple zones
>within the same region.
>
>In case of Upgrade scenario , where we have multiple zones with multiple
>Secondary storages ( 1 per zone) to begin with and we upgrade to a set up
>where all the zones become part of the same Region , what is the plan for
>handling the secondary storage merge ? Do we ask customers to do a manual
>merge of all the templates and snapshots from all the Secondary storages to one
>shared secondary storage before proceeding with the upgrade?
>
>[KK] This will be addressed through CLOUDSTACK-714
>
>2. Will  the following template API calls continue to refer to zoneId -
>createTemplate(), registerTemplate(),extractTemplate() ?
>
>In case of multiple zones with in the same region with only 1 secondary storage ,
>it seems confusing to allow users to create templates for only 1 zone ?
>
>Also will the semantics of the existing copyTemplate() be changed to supporting
>copying of templates between regions ? Currently this API lets us copy templates
>between zones which will not be needed anymore. Or are we introducing a new
>API for supporting copying of templates between regions.
>
>[KK] This again is part of CLOUDSTACK-714. Copying templates between regions
>is currently out of scope for this release.
>
>3. AWS impact - With the notion of "Regions" being introduced and having
>accounts/domains being replicated  across multiple management server clusters
>, there will be a need to do the same kind of replication of "usercredentials"
>table in cloud_bridge as part of AWS user registeration.
>
[Likitha] Based on the discussions we had in the mailing list [1], I have submitted a patch
to remove user registration for EC2 Query API calls. If this is accepted and applied, then
for EC2 Query API's we will longer be using the cloud_bridge db. And as far as registration
for EC2 SOAP API's is concerned, looks like it is being deprecated by AWS which was mentioned
in the list here [2]. So, for now I don't think we need to worry about the replication of
the "usercredentials" table in the cloud_bridge db.

[1] http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-dev/201212.mbox/%3C64FB1554ABC9B44FAA773FBD6CB889C20109380FEBD2@BANPMAILBOX01.citrite.net%3E
[2] http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-dev/201208.mbox/%3CCA+96GG4Zia=NnTJYM18wXEv=j1=xW_i_SJ4HmYhKQYB6-kB27A@mail.gmail.com%3E

>Will there be a need to expose the region parameter for any of the supported
>AWS api calls ?
>[KK] I need to investigate more on this. I'll get back with answers.
>
[Likitha] From what I understand, while making the EC2 connection to CS AWSAPI, users will
now need to use the region parameter to set the endpoint and this will ensure the calls are
made to the right region. Does that make sense?

>4. For the following authentication related features mentioned in the FS:
>
>"Authentication Service
>An authentication service will available for components like object-store to
>authenticate and get account information using getUser API
>
>Custom Authentication Adapter
>Existing UserAuthenticator will be enhanced to support provisioning from
>directory services like LDAP and Active Directory.
>
>Account can also be auto-provisioned using the directory service."
>
>
>Are these authentication adapters specific to "Regions". Some of these already
>exist in the product without "Regions" concept. Wanted to know why FS mentions
>these authenticators specifically ? Could you please provide more insight into
>how these authenticators are affected because of the notion of "Regions" being
>introduced ?
>[KK] Existing adapters only authenticate an user. They don't provision an account
>currently. These adapters are not specific to Regions.
>
>5. With Accounts being shared across multiple "Regions", How are we planning
>to manage the various resource limits that are enforced at the account
>level/Domain level/Project level ? Should we expect the Resource counts to be
>honored across multiple regions? For example , if I have an account level
>resource limit for Vms to be set at 10 , then this account should be allowed to
>deploy only a maximum of 10 Vms across all the regions ?
>[KK] Limits will be per Region. Counts won't be summed across Regions.
>
>6. PRD states - "Global Configurations: All of the administrative hierarchy
>(Domains, Accounts, Users, Projects) related Global Configurations should have
>consistent values across all regions and has to be propagated when a user
>modifies a configuration on one of the region."
>
>Can we include all the global parameters that need to be synced across multiple
>regions in FS ? Do we expect this sync to happen when these global parameters
>actually get changed ? Global parameter changes require restart of
>management servers to take effect ? In case of global parameters that need to
>be in sync across multiple regions ,do we alert the users that all management
>servers need to be restarted?
>[KK] I'll update the FS with the list of global configs to be synced across regions.
>Yes, we need to alert the admin to restart all management servers.
>
>7. Could you list all the DB changes introduced  - 1) All new tables and 2)
>Changes done to existing tables [KK] Sure. I'll add DB changes to FS
>
>
>8. In "Changes to existing APIs" section:
>
>1. CreateAccount API - the new parameters listed are all optional ? Are these
>new parameters used only when 1 managemement server propagates  Account
>creation to other Regions?
>[KK] Yes, new params are used only when MS propagates details to other
>Regions
>
>2. Only account related API calls - createAccount, deleteAccount,
>updateAccount, disableAccount, enableAccount are listed.
>
>  I assume the domain related APIs also have similar changes ?
>[KK] True. Domain related APIs also will have similar changes
>
>
>9. Projects is being treated as a special kind of account as well in DB ? Would
>this mean that we will have to replicate data relating to projects as well across
>regions ? Would this also mean that there will be API changes to project related
>APIs ?
>[KK] Project data is not replicated.
>
>10. Will createZone() and listZone() API responses include regionId they belong
>to ?
>[KK] No. Any API call is sent to a particular Region (its endpoint), and it'll list or
>create zone only in that Region. Zones in Region A will not be visible to Region B.
>
>"Data synchronization"
>
>1. Why should there be a notion of an account/Domain being owned by a
>"Region"?  Does this region have any more
>
>information about this account/Domain it owns than other regions ?
>[KK] It doesn't have any more information. But this region will be responsible to
>propagate changes to rest of the Regions.
>
>2. Is there a reason why Edit and Deletes of account and domains cannot be
>handled only by the owner Region.
>[KK] This is to avoid parallel update operations on the same resource from
>different Regions.
>
>
>-Thanks
>Sangeetha
>
>-----Original Message-----
>From: Sangeetha Hariharan
>Sent: Monday, January 07, 2013 2:14 PM
>To: cloudstack-dev@incubator.apache.org
>Subject: RE: [ASFCS41] AWS-Style Regions
>
>Kishan,
>
>I am planning to test this feature.
>
>I have the following queries after going over the FS and PRD:
>
>1. PRD states - "Each Region should have access to an object store. The object
>store should be common to all Zones within the Region. I.e., any object stored
>from a Zone should be visible across all the other zones within the region"
>
>Does this mean that we will have 1 Secondary Storage for the entire region and
>this would mean only 1 SSVM/region.
>
>In "Assumptions" section - following is state:
>
>Object Store: This feature assumes that S3 like object store is available. Either
>implemented in CloudStack or through integrations
>
>Wanted to confirm that we should be able to use NFS for Secondary Storage in
>such cases where we will share the same Secondary storage for multiple zones
>within the same region.
>
>In case of Upgrade scenario , where we have multiple zones with multiple
>Secondary storages ( 1 per zone) to begin with and we upgrade to a set up
>where all the zones become part of the same Region , what is the plan for
>handling the secondary storage merge ? Do we ask customers to do a manual
>merge of all the templates and snapshots from all the Secondary storages to one
>shared secondary storage before proceeding with the upgrade?
>
>
>2. Will  the following template API calls continue to refer to zoneId -
>createTemplate(), registerTemplate(),extractTemplate() ?
>
>In case of multiple zones with in the same region with only 1 secondary storage ,
>it seems confusing to allow users to create templates for only 1 zone ?
>
>Also will the semantics of the existing copyTemplate() be changed to supporting
>copying of templates between regions ? Currently this API lets us copy templates
>between zones which will not be needed anymore. Or are we introducing a new
>API for supporting copying of templates between regions.
>
>
>3. AWS impact - With the notion of "Regions" being introduced and having
>accounts/domains being replicated  across multiple management server clusters
>, there will be a need to do the same kind of replication of "usercredentials"
>table in cloud_bridge as part of AWS user registeration.
>
>Will there be a need to expose the region parameter for any of the supported
>AWS api calls ?
>
>
>4. For the following authentication related features mentioned in the FS:
>
>"Authentication Service
>An authentication service will available for components like object-store to
>authenticate and get account information using getUser API
>
>Custom Authentication Adapter
>Existing UserAuthenticator will be enhanced to support provisioning from
>directory services like LDAP and Active Directory.
>
>Account can also be auto-provisioned using the directory service."
>
>
>Are these authentication adapters specific to "Regions". Some of these already
>exist in the product without "Regions" concept. Wanted to know why FS mentions
>these authenticators specifically ? Could you please provide more insight into
>how these authenticators are affected because of the notion of "Regions" being
>introduced ?
>
>5. With Accounts being shared across multiple "Regions", How are we planning
>to manage the various resource limits that are enforced at the account
>level/Domain level/Project level ? Should we expect the Resource counts to be
>honored across multiple regions? For example , if I have an account level
>resource limit for Vms to be set at 10 , then this account should be allowed to
>deploy only a maximum of 10 Vms across all the regions ?
>
>6. PRD states - "Global Configurations: All of the administrative hierarchy
>(Domains, Accounts, Users, Projects) related Global Configurations should have
>consistent values across all regions and has to be propagated when a user
>modifies a configuration on one of the region."
>
>Can we include all the global parameters that need to be synced across multiple
>regions in FS ? Do we expect this sync to happen when these global parameters
>actually get changed ? Global parameter changes require restart of
>management servers to take effect ? In case of global parameters that need to
>be in sync across multiple regions ,do we alert the users that all management
>servers need to be restarted?
>
>7. Could you list all the DB changes introduced  - 1) All new tables and 2)
>Changes done to existing tables
>
>
>-Thanks
>Sangeetha
>
>-----Original Message-----
>From: Kishan Kavala [mailto:Kishan.Kavala@citrix.com]
>Sent: Monday, January 07, 2013 3:50 AM
>To: cloudstack-dev@incubator.apache.org
>Subject: RE: [ASFCS41] AWS-Style Regions
>
>I've updated the spec and moved it to ASF wiki.
>https://cwiki.apache.org/confluence/display/CLOUDSTACK/AWS-
>Style+Regions+Functional+Spec
>
>Most of the implementation was already done on the Regions feature branch.
>Data synchronization across regions is done via regular API calls with admin
>keys.
>
>All update/delete operations are sent to the source Region (i.e. the region where
>the resource was initially created).  If the source region is down, update/delete
>operations will fail.
>Is this an acceptable limitation? Should we consider other options instead of
>using API calls for data synchronization?
>Basically the problem we trying to solve is to keep data in certain tables
>(account, users, domains) consistent across multiple distributed databases (one
>database in each region).
>
>-----Original Message-----
>From: Chip Childers [mailto:chip.childers@sungard.com]
>Sent: Thursday, 13 December 2012 11:14 PM
>To: cloudstack-dev@incubator.apache.org
>Subject: Re: [ASFCS41] AWS-Style Regions
>
>On Thu, Dec 13, 2012 at 12:02 PM, Manan Shah <manan.shah@citrix.com>
>wrote:
>> Yes, Chip, this is the same feature. I didn't see the requirements and
>> so added them and linked the JIRA tickets and requirements together.
>>
>> I am not a developer of this feature and so I added the requirements
>> in the "Not committed to a release" page but I am proposing that this
>> feature be completed in time for the end of Jan feature freeze time.
>
>Thanks for clarifying.
>
>Kishan appears to have the Jira record assigned to him.  Kishan - is this
>something you are hoping to make it into 4.1?
>
>Thanks!
>
>> Regards,
>> Manan Shah
>>
>>
>>
>>
>> On 12/12/12 9:19 PM, "Chip Childers" <chip.childers@sungard.com> wrote:
>>
>>>On Wed, Dec 12, 2012 at 7:25 PM, Manan Shah <manan.shah@citrix.com>
>wrote:
>>>> Hi,
>>>>
>>>> I would like to propose a new feature for AWS-Style Regions. I have
>>>>created a JIRA ticket and provided the requirements at the following
>>>>location.  Please provide feedback on the requirements.
>>>>
>>>> JIRA Ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-241
>>>> Requirements:
>>>>https://cwiki.apache.org/confluence/display/CLOUDSTACK/AWS-Style+Regi
>>>>ons
>>>>
>>>> Regards,
>>>> Manan Shah
>>>
>>>Thanks for sharing Manan!
>>>
>>>One question though: Is this the same, or different, from what was
>>>discussed in [1]?  I'm asking, because I'm trying to track down all
>>>features that are being proposed to ensure that we don't have any lose
>>>ends.
>>>
>>>Also, are you proposing that this be completed in time for the feature
>>>freeze at the end of January?
>>>
>>>-chip
>>>
>>>[1] http://markmail.org/thread/iellnivoxq3kgz5w
>>
>>

Mime
View raw message