Mailing-List: contact libcloud-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: libcloud@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of tomaz@cloudkick.com
 designates 209.85.214.47 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <3693C551-7009-4248-AA94-A76FCE3E3655@apache.org>
References: <AANLkTimB1+zO3HsQfWKT58289cWoBEe1xRk=4ba74nna@mail.gmail.com>
	<AANLkTikr2gB-kpJrVpKjGfXKUguSU1ekKGZC_JR2UmyO@mail.gmail.com>
	<AANLkTintc8VdONn1Q59BDOpq5oeED5kzd_4KfJS+qrrW@mail.gmail.com>
	<3693C551-7009-4248-AA94-A76FCE3E3655@apache.org>
Date: Mon, 7 Feb 2011 19:03:27 +0100
Message-ID: <AANLkTik=q2e8Qgy7NOmjHqGRnXCjA3-vAypjUpR50=fD@mail.gmail.com>
From: Tomaz Muraus <tomaz@cloudkick.com>
To: libcloud@incubator.apache.org
Content-Type: multipart/alternative; boundary=000325556c8a79ca58049bb50cf8
Subject: Re: [libcloud] libcloud roadmap and plans for the future

--000325556c8a79ca58049bb50cf8
Content-Type: text/plain; charset=ISO-8859-1

On Mon, Feb 7, 2011 at 4:04 PM, Jerry Chen <jerry@apache.org> wrote:

>
> On Feb 7, 2011, at 8:48 AM, Tomaz Muraus wrote:
>
> > On Mon, Feb 7, 2011 at 4:57 AM, Jed Smith <jed@jedsmith.org> wrote:
> I agree. Speak up, people!
>
> BTW, I started up a wish list on the wiki a while back:
> http://wiki.apache.org/incubator/LibcloudWishlist
>
> I'm not convinced it has the best voting mechanism (it acts like a IRC
> topic vote!) but at least we can get an idea of the weighting.
>

Cool, I have updated the wiki with other ideas.

On Mon, Feb 7, 2011 at 4:24 PM, Jed Smith <jed@jedsmith.org> wrote:

> On Mon, Feb 7, 2011 at 10:04 AM, Jerry Chen <jerry@apache.org> wrote:
> >
> > As long as the abstraction doesn't create weird race conditions, nor
> forces us to modify each driver for async stuff to work, I would be okay
> with this capability. I just can't wrap my head around how one would test
> for correctness and stability.
>
> I believe it would. You now need to track what requests you have out
> and call the appropriate callback, a very difficult (and tedious
> task). A supposedly stateless library just inherited a /lot/ of state.
>

In Twisted and other event-driven system everything runs inside a single
thread (reactor) and as long you are only using asynchronous API, you don't
need to worry about race conditions.

In any case, adapting the library for async stuff would definitely require
some refactoring, but I could argue that this might not be such a bad idea.

Anecdote: I solved this problem for the Linode iPhone app. If you're
> not familiar with UIKit/Foundation development for iOS, it's all
> asynchronous. I spent very many hours debugging the code that did the
> linkage between APIRequest instances and the callback event on them.
> It got so difficult to do properly with Apple's APIs that I eventually
> stumbled upon a hopelessly inelegant solution: every request the
> iPhone app makes is a batch, with a test.echo action containing an ID
> which is a key into the pending-requests dictionary. Seriously.
>
> That's what asynchronous ends up becoming in a request-receive
> environment. It's a rabbit hole of inelegant solutions and a lot of
> tedious state tracking in the library.


Yes, I agree, debugging in a lot of callback oriented frameworks can be a
huge pain. One of the good examples of this is NodeJS - with a lot of nested
callbacks and no long stack traces finding a root cause can be pretty
painful (Twisted partially solves this with defer.SetDebugging(True)).

So, yes, in general debugging and writing asynchronous code can be painful,
but Twited has enough of syntactic sugar which makes it a lot less painful
(deferreds, inlineCallbacks, ...).


> --
> Jed Smith
> jed@jedsmith.org
>

--000325556c8a79ca58049bb50cf8--