incubator-libcloud mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Nottingham <>
Subject Re: [libcloud] libcloud roadmap and plans for the future
Date Sun, 06 Feb 2011 21:25:14 GMT

On 07/02/2011, at 6:11 AM, Tomaz Muraus wrote:

> Yes, Twisted is a big beast, but in my opinion it is still a lot better to
> use Twisted than something like asyncore.
> If we would use something like asyncore this means that we need to implement
> our own http client and a lot of other things which already exist in Twisted
> and have been battle tested.

True, but it's not that hard. I did an async HTTP client in a few hundred lines of Python,
and it has been battle-tested.

> On the other hand, we are not building a non-blocking Python framework so
> using something like epoll, kqueue, etc. directly and implementing our own
> abstractions is out of the question imo.
> There are also a lot of other non-blocking frameworks available for Python,
> but Twisted is pretty much the de-facto standard.

Instead of buying into Python's moral equivalent of J2EE, how about changing languages? You
can get very far, very quickly with node.js, for example, and it's async from the ground up.

> On Sun, Feb 6, 2011 at 12:17 AM, Grig Gheorghiu <>wrote:
>> I agree regarding Twisted. It's a road that can lead to perdition ;-)
>> On Sat, Feb 5, 2011 at 3:06 PM, Mark Nottingham <> wrote:
>>> FWIW, my experience is that creating event-driven applications is easier
>> if you just use asyncore and/or epoll directly. Twisted is too complex /
>> over-engineered / under-documented for me (and many others).
>>> On 06/02/2011, at 7:31 AM, Tomaz Muraus wrote:
>>>> I know that the development of the storage API has started just
>> recently,
>>>> but I would still love talk about the libcloud roadmap and plans for the
>>>> future.
>>>> I would eventually like to see the following features:
>>>> - non-blocking / asynchronous version of libcloud - I know that this is
>> a
>>>> major feature and requires a lot of work, but it would be very cool if,
>> in
>>>> the future you could do something like this:
>>>> from libcloud.compute.drivers.sync import EC2Driver - imports blocking
>>>> version of the driver
>>>> from libcloud.compute.drivers.async import EC2Driver - imports
>> not-blocking
>>>> version of the driver
>>>> Since Twisted is pretty much a de-facto standard for creating
>> non-blocking
>>>> Python applications, I think we should also use it for a non-blocking
>>>> version of out library.
>>>> - "Resources" concept - this feature is also be a pretty major one and
>>>> requires a lot of thought.
>>>> A "Resource" would be a generic concept which would represent a some
>> kind of
>>>> "Cloud resource" - a resource could be an IP address, load balancer,
>> etc.
>>>> This would basically allow us to implement a lot of (currently) provider
>>>> specific feature, but in a generic way so it would later be easy to
>> adapt it
>>>> to work with a different provider.
>>>> - Python 3.x support - Jed has already suggested this on IRC. I actually
>>>> haven't tested libcloud with Python 3.x yet, so I don't know how many
>> things
>>>> needs to be changed to make it work (without using 2to3 or a similar
>> tool).
>>>> Thoughts, opinions?
>>>> Thanks,
>>>> Tomaz
>>> --
>>> Mark Nottingham

Mark Nottingham

View raw message