esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ethan Jewett <>
Subject Re: FYI - Twitter Streaming API
Date Tue, 24 Nov 2009 19:50:57 GMT
On Tue, Nov 24, 2009 at 10:09 AM, David Pollak
<> wrote:
> On Tue, Nov 24, 2009 at 7:40 AM, Richard Hirsch <>wrote:
>> What I think is also  interesting is how Twitter documents this new
>> API. maybe we can use this as an example.
>> What are the advantages of our method vs. their method?
> The advantage of their mechanism is that it's a smoother experience.  What
> we've done with chunking/long polling is to simulate a stream of data on top
> of a non-streaming protocol.  What Twitter has done is to say "this is a
> one-way conversation, we've got an open TCP/IP connection, so let's use it."
> Implementing what they have would require going below the current set of
> abstractions that Lift provides above the Servlets.
> At a practical level, the difference is at one layer... the one dealing with
> the HTTP requests.  At the layers above, events flow either way.

A significant disadvantage of Twitter's design is the requirement of
only one streaming connection per account. As much as I dislike the
approach of using session cookies to uniquely identify API message
queues, it is a heck of a lot better than what is going to happen when
Twitter clients start to implement this API, which will be:

1. I log in with Seesmic Web (which has implemented the Twitter streaming API)
2. I receive messages 1, 2, and 3.
3. I log in on a different computer with Twhirl (which has also
implemented the Twitter streaming API)
(3.1 Twitter disconnects the Seesmic connection invisibly from the user)
4. I receive message 4 in Twhirl
(4.1 Seesmic tries to reconnect, which results in Twhirl being disconnected)
5. I receive message 5 in Seesmic
6. And so on....

End result: 1 really confused user trying to connect from two banned
IP addresses.

I think this is a good illustration of why we need some
client-specific identifier for a streaming/delta-queue API. It doesn't
need to be a session, but that's working pretty nicely for now.

I would prefer to stick with what Lift provides for the moment. I need
to do the conceptual exercise, but on first glance I don't think
Twitter's approach results in much of a gain over our approach. Fewer
connection attempts, which will help a lot at Twitter-scale, but which
I'm not sure makes a big difference at Enterprise-scale.

Another drawback (and I'm really not sure on this one) is that I don't
think a lot of HTTP client libraries give easy access to a request
that is still open. The way that David and Vassil designed the queue
API is extremely simple from a client programming perspective. I think
that's a big upside.

>>  Or are there
>> basic philosophical differences?
> At the basic philosophical level, Twitter's implementation is purer.  It
> treats a stream of information as a stream of information.

I agree. I like it a lot on this level.

> I like it, but I'm not sure what the benefits would be vs. the development
> costs of implementing such a mechanism (unless there's an en mass migration
> of microblogging clients to such a mechanism).

Agreed also.

View raw message