cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chiradeep Vittal <Chiradeep.Vit...@citrix.com>
Subject Re: [DISCUSS] Remove async APIs from storage framework
Date Thu, 05 Sep 2013 19:57:17 GMT
+1 for removing complexity especially if the sync pattern is being used on
top of the async pattern.
I see this behavior in the AgentManager.send as well -- even thought the
AM 
is capable of async, practically nobody uses it as such.

But I guess the question will arise : what if I do want more than 10^n
long-running storage jobs
(cos my cloud is as successful as AWS :))

On 9/4/13 5:03 PM, "Darren Shepherd" <darren.s.shepherd@gmail.com> wrote:

>I've been reading over the storage code and have come to the conclusion
>that the async aspects of the storage framework should be removed.
>
>Whenever one introduces an async pattern you have to give a lot of
>consideration to its use, benefits, and impact.  Within the context of
>ACS and given the current state of its code, I do not think it will be
>possible to realize any benefits of the current callback approach.
>Since nothing else in ACS uses callbacks, all of the async methods are
>essentially wrapped in synchronous calls.  So nothing as it stands is
>actually async.
>
>Besides the current implementation, you need to conciser how you would
>expect an implementation of the storage framework to use the callback.
>The problem with callbacks is that they assume some in memory state.
>This means if the process/server crashes that state is lost.  Many will
>say just serialize the callback to the DB, but that is very impractical.
>
>Since ACS doesn't actually stand in the data path, an async pattern
>won't really even allow it to have better performance.  ACS is just
>waiting for some storage operation to happen.  ACS can easily spawn 1000
>threads and have them all wait.  If you were to get to this point, you'd
>find that downstream you'll most likely have issues as you have 1000
>create template operations so its killing your filer.  So you will
>throttle storage operations to a level that won't kill your
>infrastructure and that level is no where near the scalability limits of
>threads.
>
>The callbacks pattern really complicates the code and I see no real
>benefit.  Instead of spending a lot of effort trying to make all of ACS
>async to make it beneficial, I'd say that effort should be spent on
>making ACS idempotent and crash-only.  The point being, there's more
>beneficial things we can do with our time.
>
>Given that only solidfire implements the new framework (and ACS legacy
>too), I would assume its a simple things for Edison to just go and
>quickly change it non-async.
>
>Darren


Mime
View raw message