esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ethan Jewett <esjew...@gmail.com>
Subject Re: Streaming design for the api2 endpoint - request for comment
Date Tue, 15 Dec 2009 13:49:33 GMT
Hi Daniel,

Yes, that's pretty much what the current /api2/user/messages endpoint
provides. It just gives you all the messages in the user's timeline.
(Though it doesn't currently provide all the necessary information
with each message, which I need to fix.)

Ethan

On Tue, Dec 15, 2009 at 8:38 AM, Daniel Koller <dakoller@googlemail.com> wrote:
> just a comment on the side:...is it an option to provide a powerful and
> scalable  - but generic getMessages()-function with a bunch of parameters
> (tags/text/author...date...) and to let the collection e.g. by tags be gone
> by the requestor?
>
> On Tue, Dec 15, 2009 at 2:34 PM, Ethan Jewett <esjewett@gmail.com> wrote:
>
>> Actually, having separate streams for each tag is what I'm suggesting,
>> I'm just trying to determine when best to create them. If a client
>> requests all the streams, they will all be created. Should we talk
>> about not streaming individual tags at all? Maybe put a limit on the
>> number of streams a client can have open? I'm not sure what the
>> performance impact will look like.
>>
>> Also, I need to amend the original email. I think that because we are
>> using Lift Sessions, we will be killing off the session and the
>> streams attached to it after a period of time. So I think option 2 and
>> option 4 are the same or very similar.
>>
>> Ethan
>>
>> On Mon, Dec 14, 2009 at 11:39 PM, Richard Hirsch <hirsch.dick@gmail.com>
>> wrote:
>> > Yes it sounds reasonable. I don't think it makes much sense to have
>> > separate streams for each tag, etc...
>> >
>> > I agree option 2 is the best choice.
>> >
>> > D.
>> >
>> > On Mon, Dec 14, 2009 at 12:29 AM, Ethan Jewett <esjewett@gmail.com>
>> wrote:
>> >> All,
>> >>
>> >> In the cwiki, we've documented 5 parts of the API we would like to
>> >> stream. They are briefly: user timeline, tags, tracks, conversations,
>> >> and pools (and possibly the public timeline)
>> >>
>> >> Of these, one has been implemented: user timeline
>> >>
>> >> Today I've been able to take some time to start digging into what
>> >> needs to be done to implement the rest of the streaming interfaces.
>> >> The way the user timeline streaming interface is implemented in the
>> >> old and new APIs is the same (because I just copied and slightly
>> >> modified the code). The basic idea is that when a session is created,
>> >> the streaming API starts "listening" for new messages. When the user
>> >> makes a request to the streaming interface for new messages, all the
>> >> messages that have built up are delivered.
>> >>
>> >> This approach poses some significant problems for other types of
>> >> streams. For example, if we were going to stream tags in this manner,
>> >> we would end up creating a listener for every single active tag in the
>> >> system at the time the user initiates a session. We would also have
>> >> the dilemma of creating listeners for new tags as the tags are created
>> >> in the middle of a session.
>> >>
>> >> As such, I'm thinking of implementing the other streaming interfaces
>> >> differently. Instead of creating listeners when the session is
>> >> initiated, I'll create them when the first streaming request for a
>> >> tag, pool, track, or conversation comes in. These listeners would then
>> >> live on for the rest of the session. This is, I think the best of
>> >> several options.
>> >>
>> >> To summarize the options available:
>> >>
>> >> 1. Create listeners for everything at the beginning of the session -
>> >> not efficient, suffers from difficulties with new tags, pools, etc.
>> >> created during the session
>> >>
>> >> 2. Create listeners for streams as the user requests them and have
>> >> these listeners live on for the rest of the session
>> >>
>> >> 3. Create disposable listeners for each streaming/long-polling request
>> >> that are destroyed once the request is answered - this is problematic
>> >> because messages that occur between requests will be missed
>> >>
>> >> 4. Variation of option 2 and 3: Create listeners for streams as the
>> >> user requests them and have these listeners life on for the rest of
>> >> the session or a specific period of time, whichever comes first (so
>> >> the user would have to make occasional requests to ensure the
>> >> continuity of the message stream) - I think this is over-complicated
>> >> and potentially confusing to developers, but could be a good option if
>> >> we run into performance problems with option 2
>> >>
>> >>
>> >> What we'll be left with is that the user timeline will use option 1
>> >> and the other streams will use option 2. The user timeline might
>> >> switch to option 2 at some point in the future.
>> >>
>> >> And that was all a very long way of saying, does that sound reasonable
>> >> to everyone?
>> >>
>> >> Ethan
>> >>
>> >
>>
>
>
>
> --
> ---
> Daniel Koller
> Jahnstrasse 20
> 80469 München * dakoller@googlemail.com
>

Mime
View raw message