cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Ough <alex.o...@sungard.com>
Subject Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
Date Thu, 06 Feb 2014 14:29:28 GMT
All,

I just sent a review request, so please take a look at it and let me know
if you have any comments/suggests.

https://reviews.apache.org/r/17790/

Thanks
Alex Ough


On Mon, Jan 13, 2014 at 11:17 AM, Alex Ough <alex.ough@sungard.com> wrote:

> All,
>
> I'd like to have some suggestion about 2 things related with this.
>
> 1. The 'Full Scan' management
> Now, I set it running every time a user logs in to the UI, but I think it
> will be necessary to make it run with some interval also.
> But I'm not familiar with the config file, so can anyone give some
> directions how to manage the time interval in the config file and the best
> way to run it with the time interval?
>
> 2. Repository of regions with their login information.
> To send/receive requests to/from other regions using API interfaces, we
> need the region information including login info of each region.
> I was planning to use a table as a repository, but I think it is better to
> store it in the config file to make the access a little lighter.
> Any recommendation on this?
>
> Your reply with directions & comments will be very appreciated.
> Thanks
> Alex Ough
>
>
> On Wed, Jan 8, 2014 at 2:17 PM, Alex Ough <alex.ough@sungard.com> wrote:
>
>> All,
>>
>> A little bit of updates after a long vacation,
>> I'm currently creating automated test scripts that randomly
>> create/delete/update domain/account/user objects in random regions to
>> trigger the sync-up and full scans regularly.
>> Once they are completed, I'll post it in the github also and submit the
>> review requests for this implementation.
>>
>> Let me know if you have any comments.
>> Thanks
>> Alex Ough
>>
>>
>> On Wed, Dec 18, 2013 at 3:39 PM, Alex Ough <alex.ough@sungard.com> wrote:
>>
>>> All,
>>>
>>> I updated the wiki after some logic changes, so please review them,
>>> especially "Full Scan", which is newly introduced.
>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions
>>>
>>> And I implemented this functionality in Java and you can get the pull
>>> request of it here.
>>> (This does not include the 'full scan' yet and I'm currently working
>>> on this to finalize.)
>>> https://github.com/alexoughsg/Albatross/pull/1
>>>
>>> Especially, I really want to have your review on the "Full Scan" logic
>>> to confirm if it does not miss any cases.
>>> Thanks for your interest and your feedback will be very helpful.
>>> Alex Ough
>>>
>>> On Tue, Nov 12, 2013 at 6:00 PM, Alex Ough <alex.ough@sungard.com>
>>> wrote:
>>> > Good point, Chiradeep,
>>> >
>>> > I'm not sure if you reviewed my design doc in the wiki, but my design
>>> is to
>>> > just skip any actions for target resources that already took place by
>>> any
>>> > means.
>>> > But the issue is when conflict actions in the same resources (like
>>> create &
>>> > delete the same users) are enqueued in reversed orders, which is
>>> hopefully
>>> > rare.
>>> >
>>> > And to support consistency in the AP system, I'd like to provide a
>>> full sync
>>> > up, which will sync up all data in all region servers by selecting a
>>> region
>>> > as a master and force its data to other regions.
>>> >
>>> > Let me know what you think.
>>> > Thanks
>>> > Alex Ough
>>> >
>>> >
>>> > On Tue, Nov 12, 2013 at 1:22 PM, Chiradeep Vittal
>>> > <Chiradeep.Vittal@citrix.com> wrote:
>>> >>
>>> >> Missed this one. In a single region, the CloudStack DB is the master
>>> for
>>> >> most operations. If the infra is not in the state the DB says it
>>> should
>>> >> be, generally the approach is to whack it and make it conform. For
>>> some
>>> >> exceptions (live migration/related use cases are exceptions) the DB
>>> is the
>>> >> slave -- the point is that the inconsistency that inevitably arise in
>>> an
>>> >> AP system need a conflict resolution system. In a single region, the
>>> >> default is to assume that the MySQL DB is correct and handle other
>>> cases
>>> >> carefully.
>>> >>
>>> >> In a multi-region case, there is no master: disable an account in one
>>> >> region, and it may not propagate to the other regions for many
>>> hours/days.
>>> >> You could add a user in one region and then add the same user in a
>>> >> different region and conflict before the sync happens.
>>> >>
>>> >> This is of course not a problem unique to CloudStack -- people pay
>>> mucho
>>> >> dinero for Global AD/LDAP sync. I don't think this is a problem for
>>> >> CloudStack core to solve: I support the event-based mechanism for
>>> those
>>> >> who want this facility.
>>> >>
>>> >> Distributed systems are hard and research continues to try and make
>>> >> building these systems easier, but there are very few solutions for
>>> global
>>> >> state synchronization (Google Spanner comes to mind)
>>> >>
>>> >> --
>>> >> Chiradeep
>>> >>
>>> >>
>>> >> On 11/8/13 4:53 PM, "Chip Childers" <chip.childers@gmail.com>
wrote:
>>> >>
>>> >> >We are already (generally) AP for most infra changes really. I'd
use
>>> that
>>> >> >model. Eventual consistency is better in this scenario.
>>> >> >
>>> >> >> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal
>>> >> >><Chiradeep.Vittal@citrix.com> wrote:
>>> >> >>
>>> >> >> I'd also like to highlight that it isn't a trivial problem.
>>> >> >> Let's say there's 3 regions: this means there are 3 copies
of the
>>> user
>>> >> >> database that are geographically separated by network links
that
>>> fail
>>> >> >> quite often (orders of magnitude more than intra-DC networks).
>>> >> >>
>>> >> >> Here we run into the consequences of the CAP theorem [1].
>>> >> >> We can either have a CP or AP system: either approach makes
some
>>> >> >>tradeoffs:
>>> >> >> 1. If we run a AP system, then the challenge is to resolve
>>> conflicting
>>> >> >> updates
>>> >> >> 2. If we run a CP system, then the challenge is to detect
>>> partitions
>>> >> >> reliably and disallow updates during partitions.
>>> >> >>
>>> >> >> [1] http://en.wikipedia.org/wiki/CAP_theorem
>>> >> >>
>>> >> >>> On 11/7/13 11:58 AM, "Chip Childers" <chipchilders@apache.org>
>>> wrote:
>>> >> >>>
>>> >> >>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal
>>> >> >>> <Chiradeep.Vittal@citrix.com> wrote:
>>> >> >>>> It may be an admin burden, but it has to be optional.
There are
>>> other
>>> >> >>>> ways
>>> >> >>>> to achieve global sync (e.g., LDAP/AD/Oauth).
>>> >> >>>> A lot of service providers who run cloudstack have
their own user
>>> >> >>>> database
>>> >> >>>> / portal. In their implementations the CloudStack database
is
>>> not the
>>> >> >>>> master source of user records, but a slave.
>>> >> >>>
>>> >> >>> +1 to it being optional.
>>> >> >>
>>> >>
>>> >>
>>> >
>>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message