esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Hirsch <>
Subject Re: Streaming design for the api2 endpoint - request for comment
Date Tue, 15 Dec 2009 04:39:08 GMT
Yes it sounds reasonable. I don't think it makes much sense to have
separate streams for each tag, etc...

I agree option 2 is the best choice.


On Mon, Dec 14, 2009 at 12:29 AM, Ethan Jewett <> wrote:
> All,
> In the cwiki, we've documented 5 parts of the API we would like to
> stream. They are briefly: user timeline, tags, tracks, conversations,
> and pools (and possibly the public timeline)
> Of these, one has been implemented: user timeline
> Today I've been able to take some time to start digging into what
> needs to be done to implement the rest of the streaming interfaces.
> The way the user timeline streaming interface is implemented in the
> old and new APIs is the same (because I just copied and slightly
> modified the code). The basic idea is that when a session is created,
> the streaming API starts "listening" for new messages. When the user
> makes a request to the streaming interface for new messages, all the
> messages that have built up are delivered.
> This approach poses some significant problems for other types of
> streams. For example, if we were going to stream tags in this manner,
> we would end up creating a listener for every single active tag in the
> system at the time the user initiates a session. We would also have
> the dilemma of creating listeners for new tags as the tags are created
> in the middle of a session.
> As such, I'm thinking of implementing the other streaming interfaces
> differently. Instead of creating listeners when the session is
> initiated, I'll create them when the first streaming request for a
> tag, pool, track, or conversation comes in. These listeners would then
> live on for the rest of the session. This is, I think the best of
> several options.
> To summarize the options available:
> 1. Create listeners for everything at the beginning of the session -
> not efficient, suffers from difficulties with new tags, pools, etc.
> created during the session
> 2. Create listeners for streams as the user requests them and have
> these listeners live on for the rest of the session
> 3. Create disposable listeners for each streaming/long-polling request
> that are destroyed once the request is answered - this is problematic
> because messages that occur between requests will be missed
> 4. Variation of option 2 and 3: Create listeners for streams as the
> user requests them and have these listeners life on for the rest of
> the session or a specific period of time, whichever comes first (so
> the user would have to make occasional requests to ensure the
> continuity of the message stream) - I think this is over-complicated
> and potentially confusing to developers, but could be a good option if
> we run into performance problems with option 2
> What we'll be left with is that the user timeline will use option 1
> and the other streams will use option 2. The user timeline might
> switch to option 2 at some point in the future.
> And that was all a very long way of saying, does that sound reasonable
> to everyone?
> Ethan

View raw message