incubator-libcloud mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Davis <...@dislocatedday.com>
Subject Re: [libcloud] libcloud roadmap and plans for the future
Date Mon, 07 Feb 2011 02:39:00 GMT
On Feb 6, 2011, at 4:25 PM, Mark Nottingham <mnot@mnot.net> wrote:

> 
> On 07/02/2011, at 6:11 AM, Tomaz Muraus wrote:
> 
>> Yes, Twisted is a big beast, but in my opinion it is still a lot better to
>> use Twisted than something like asyncore.
>> 
>> If we would use something like asyncore this means that we need to implement
>> our own http client and a lot of other things which already exist in Twisted
>> and have been battle tested.
> 
> True, but it's not that hard. I did an async HTTP client in a few hundred lines of Python,
and it has been battle-tested.

Regardless of anything else, I agree with Tomaz that asyncore is an unnecessarily low level
of abstraction for libcloud's needs. Unless your http client has all the necessary features
and a robust test suite, it seems like another existing solution would be preferable.

> 
> 
>> On the other hand, we are not building a non-blocking Python framework so
>> using something like epoll, kqueue, etc. directly and implementing our own
>> abstractions is out of the question imo.
>> 
>> There are also a lot of other non-blocking frameworks available for Python,
>> but Twisted is pretty much the de-facto standard.
> 
> Instead of buying into Python's moral equivalent of J2EE, how about changing languages?
You can get very far, very quickly with node.js, for example, and it's async from the ground
up.

My vote is for Twisted, but I'm pretty biased towards it. As for changing languages, I think
that should be a last resort considering you lose the copious amount of libcloud that's been
implemented with no specific requirement for synchronous code (use inlineCallbacks in Twisted
and you could probably just make a new Request class, stick "yield" in front of most calls
to request(), and call it a day). There are other Python solutions that would allow code reuse,
unless I'm missing something about node's ability to run Python code. Reinventing the wheel
is hardly justified by a dislike of one choice, in my opinion.

> 
> 
>> On Sun, Feb 6, 2011 at 12:17 AM, Grig Gheorghiu <grig.gheorghiu@gmail.com>wrote:
>> 
>>> I agree regarding Twisted. It's a road that can lead to perdition ;-)
>>> 
>>> On Sat, Feb 5, 2011 at 3:06 PM, Mark Nottingham <mnot@mnot.net> wrote:
>>>> FWIW, my experience is that creating event-driven applications is easier
>>> if you just use asyncore and/or epoll directly. Twisted is too complex /
>>> over-engineered / under-documented for me (and many others).
>>>> 
>>>> 
>>>> On 06/02/2011, at 7:31 AM, Tomaz Muraus wrote:
>>>> 
>>>>> I know that the development of the storage API has started just
>>> recently,
>>>>> but I would still love talk about the libcloud roadmap and plans for
the
>>>>> future.
>>>>> 
>>>>> I would eventually like to see the following features:
>>>>> 
>>>>> - non-blocking / asynchronous version of libcloud - I know that this
is
>>> a
>>>>> major feature and requires a lot of work, but it would be very cool if,
>>> in
>>>>> the future you could do something like this:
>>>>> 
>>>>> from libcloud.compute.drivers.sync import EC2Driver - imports blocking
>>>>> version of the driver
>>>>> from libcloud.compute.drivers.async import EC2Driver - imports
>>> not-blocking
>>>>> version of the driver
>>>>> 
>>>>> Since Twisted is pretty much a de-facto standard for creating
>>> non-blocking
>>>>> Python applications, I think we should also use it for a non-blocking
>>>>> version of out library.
>>>>> 
>>>>> - "Resources" concept - this feature is also be a pretty major one and
>>>>> requires a lot of thought.
>>>>> 
>>>>> A "Resource" would be a generic concept which would represent a some
>>> kind of
>>>>> "Cloud resource" - a resource could be an IP address, load balancer,
>>> etc.
>>>>> 
>>>>> This would basically allow us to implement a lot of (currently) provider
>>>>> specific feature, but in a generic way so it would later be easy to
>>> adapt it
>>>>> to work with a different provider.
>>>>> 
>>>>> - Python 3.x support - Jed has already suggested this on IRC. I actually
>>>>> haven't tested libcloud with Python 3.x yet, so I don't know how many
>>> things
>>>>> needs to be changed to make it work (without using 2to3 or a similar
>>> tool).
>>>>> 
>>>>> Thoughts, opinions?
>>>>> 
>>>>> Thanks,
>>>>> Tomaz
>>>> 
>>>> --
>>>> Mark Nottingham   http://www.mnot.net/
>>>> 
>>>> 
>>>> 
>>>> 
>>> 
> 
> --
> Mark Nottingham   http://www.mnot.net/
> 
> 
> 

Mime
View raw message