esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Koller <>
Subject Re: [UI] data model and functionality
Date Fri, 20 Feb 2009 13:27:25 GMT
Hi Bill,

thx for compiling such an overview: my comments follow inline:

On Thu, Feb 19, 2009 at 11:25 PM, Bill Fernandez <>wrote:

> ============================================================
> by Bill Fernandez, 19feb09
> ============================================================
> ____________________________________________________________
> o Let's assume that we're limiting our discussion to a single server, and
> that we'll deal with federations of servers in a future discussion.
> o I'm assuming that for each ESME server there will be one and only one
> user directory server (such as an LDAP server).

In my opinion we should allow more sources as a user database: jdbc, ldap,
active directory also in combination.

> o I'm assuming that an ESME server will only recognize users listed in the
> directory server.
> o I'm assuming that the directory server is managed by organization-level
> administrators, and that only these administrators can add, edit and delete
> users.
> o WHO can use a given ESME server is thus out of control of the ESME
> server, and in control of the user directory administrator(s).
> o Since the ESME server will have to keep track of a number of
> ESME-specific data for each user (e.g. "who do I follow?"), the ESME server
> will have to maintain a database of "preferences" that are keyed to each
> user in the directory server.

The ESME server should use an user identifier (from a directory), but should
keep the preferences in an own data store independent from the directoy.
(--> I am thinking a environments, where ESME can read/use the directory,
but not modify it)

> o Within the ESME end-user UI, this results in there being some info that
> comes from the directory server and thus that the end user CAN'T change
> (e.g. name, work phone, department, mailstop, etc.) and info that only
> applies within the ESME context and that the end user CAN change (e.g.
> nickname, maybe his/her picture or icon, list of people he/she follows,
> etc.).
> o Are the above assumptions and implications correct?
> o Do we want to allow the administrator of an ESME server to be able to
> create and maintain it's own private database of authorized users in
> addition to those provided by the organization's directory server?
>      PROs: Makes it easy to set up a test/evaluation ESME server. Makes it
> possible for small organizations (those without corporate LDAP, etc.
> directories) to use ESME.
>      CONs: Introduces the possibility of duplicate user entries.  If used
> extensively makes synchronization with the organizational directory
> impossible.  Provides an end run around policies that the organization is
> counting on the organizational directory to enforce.
> o What ESME-specific, end-user-defined, user-profile data do we want to
> support?
>      ESME Nickname? -- But what if organizational policy dictates the use
> of only a person's "official" user name?

We should support an ESME specific nicknames, because username policy might
also mean that I am "Z000EC1G", which is ok for windows domains, but not for
a microblogging tool.

>      Picture/Icon? -- But what if the organizational directory server
> supplies one?  Do we use the "official" image by default, and let any
> user-defined image override it?

If you let the image_url property be either editable or not, this topic
should be solved.

> o Would our strategy be that the ESME administrator could enable/disable
> any of these user-profile-customization fields as required by organizational
> policy, that we use the organizational-server-supplied data by default, but
> override it with the user-supplied values (if any)?

Good point.

> ____________________________________________________________
> David says he hates the idea of groups (though he likes "pools") and of
> sending things to groups (though he likes "posting to pools"). Nevertheless
> I'm going to use the term "groups" because it helps me describe certain
> functionality, and I hope we can decide on terminology after the
> functionality's agreed to.
> With that, I'd like to suggest some concepts and see how the team feels
> about them.
> o I'd like to propose that each server can support an abitrary number of
> groups.
> o A group is a logical construct consisting of a set of data:
>    -- A set of users.
>    -- A pool of messages.
>    -- A set of metadata.
>    -- Perhaps a set of business rules.
> o All users are members of a group called something like "Public", which by
> definition contains the set of all users of the ESME server.
> o A user may be a member of as many additional groups as desired.
> o Any user can create a group (or would we like the ESME administrator to
> be able to grant a "create group" privilege to only certain users?).
> o The creator of the group is it's first (and initially only)
> administrator.
> o A group's administrator(s) can add and remove users at will, and sets the
> privileges for each user.
> o A group's administrator(s) can edit the group's metadata (e.g. it's
> name), and edit any business rules we support for groups.

I would love a feature to populate groups automatically based on business
rules such as "create a group with all people with jobtitle=controller in
department="XYZ" (based on regular expressions this could be quite powerful.

> o Within a group, each user has one of the following privilege levels:
>      -- Read.  (Let's you read messages in the group's pool)
>      -- Read and Write. (adds the ability to post messages)
>      -- Read, Write and Administer (adds all admin privileges)
> o The above model establishes three roles:  those of viewer, participant
> and administrator.
> o Any user having at least READ privileges against a group has PERMISSION
> to see ALL messages in the group's pool.
> o Anyone with admin privileges is an administrator.  Thus a group can have
> as many admins as desired.
> o There's no concept of "ownership", only of "control".  All administrators
> have equal control, and the creator of the group can drop off the set of
> administrators to pass control on to others.
> o Over time, organizations will inevitably want finer-grained admin
> privileges.  Do we want to try to build them in from the beginning? For
> example:
>      -- Moderate group (lets user delete messages?)
>      -- Delete group (lets user destroy the group)
>      -- Add/remove read and/or write users
>      -- Add/remove, promote/demote, admin users
> o Before posting a message you must choose a group.  This defines the set
> of users who will have permission to read the message.
> o By default, all users are members of the Public group, the Public group
> is the only group that is required for an ESME server, and thus in the
> simplest case all messages are posted to the public pool and accessible to
> everyone.
> o Every message posted to a group goes into that group's message pool;
> which is the set of all messages posted to/within that group.
> o Which messages a given user sees depends upon what the user has chosen to
> see (who he/she follows, what tags he/she tracks, etc.), and/or what he/she
> searches for.
> o So each message exists in one and only one pool, and only users of that
> pool's group have permission to access that message.
> o Therefore by choosing a group before you post a message you are choosing
> the scope of privacy for that message; because (a) ALL members of that group
> will have a way (by searching, tracking, following, etc.) to SEE that
> message, (b) NO users outside that group will ever be able to see that
> message.
> o Note that since a given message can only exist in one pool over its
> lifetime, if you want multiple groups to recieve the same message you must
> post it to each of those groups separately.  From an intuitive, end-user
> standpoint this appears to be a case of posting the same message to multiple
> groups, but to the system each posting is a separate logical entity (each
> has it's own unique ID, even if the text of each is the same).
> o So if you see a message in one pool that you want users of another pool
> to see, you have to "copy" it from the one and "paste" it into the other.
> o This gives us a simple, safe privacy model, yet makes it easy and
> convenient for users to circumvent the rules when necessary.  It takes the
> burden off the system to manage access control of cross-pool messaging, and
> puts the burden of responibility on the user.
> o Thus the fundamental access control model is:  (a) create groups of users
> that need to communicate with each other, (b) post messages to the
> appropriate group, (c) the system will keep your messages private to the
> group.
> o I think we also need to make it possible to have private conversations
> between any two users without having to formally set up a group.  It should
> be easy to make it so that you choose a user in the UI as the "group", post
> a message, and see all the messages in this ad-hoc, private pool.
> o But then, of course, the two users are going to want to bring others in
> on some conversations, so maybe it would be better to make private, ad-hoc
> group messaging a property of conversations....  I don't yet have a
> suggestion for how to handle this anticipated end-user desire.
> ____________________________________________________________
> o OK, so I've suggested that: messages be clustered into pools, each pool
> has a wall of privacy around it that ony allows users in it's group to
> "fish" for messages from that pool, and that a given user may be a member of
> multiple groups and thus have access to multiple pools of messages.
> o So how to we convert the user's permissions against this multi-pooled
> dataspace into useful and usable views?
> o I'm assuming that messages are basically records in a relational
> database, and therefore that creating end-user views is (mechanically) a
> matter of running a query and formatting the output.
> o Most of the time the view a user wants is an ordered list of messages
> meeting certain criteria.  I call these lists "streams".
> o In various contexts I have heard the use of terms like "timeline",
> "mailbox", "tracking", "conversations", etc.  In all cases the common
> element is that they all consist of a query against the set of messages, a
> results set that is limted by a user's access controls, a sort order
> (usually reverse-chronological order), and the results displayed as a list
> of messages.  I would call all of these streams.
> o Here are some common streams:
>      -- The Public timeline.
>      -- A conversation.
>      -- A track (a timeline defined by your tracking criteria).
>      -- The messages directed (via @yourname) to you.
>      -- Replies to your messages.
>      -- Each group would have its own timeline.
>      -- The set of messages sent by users you follow.
> o The system should create certain predefined streams for you on demand.
>  You should simply be able to ask to view, and get a stream for:
>      o The messages in a selected group's pool.
>      o The messages sent by a selected user.
>      o The messages with a seleted tag.
>      o The messages posted by users you follow.
>      o All messages in the Public pool.
>      o The messages you've sent.
>      o etc.
All messages containing a defined expression?

> o In all cases, each user only gets to see those messages that their access
> privileges allow them to see.
> o For example, if you follow someone who posts to groups of which you are
> not a member, you will never under any circumstances be able to see THOSE
> messages.  On the other hand, any messages posted to the Public group (of
> which you ARE a member), and posted to groups of which you are a member;
> you'll be able to see in the streams you request.
> o So what happens if a poster directs a message (via @username) to a given
> user, but does so when posting to a group of which the user is not a member.
>  In that case, by the rule of group privacy, the targeted user would never
> see the message.

The sneder should get a feedback on that.

> o We should make it possible (and easy) for users to filter a stream.
> o For example, let's say that a given server has only one group defined so
> far, the Public group, making all messages in the server available (in
> theory) to all users.
> o By default, on viewing the public stream, we would display only the most
> recent (say) 20 messages, and let the user page back in history to see
> previous messages.
> o We should also let the user apply ad-hoc filtering criteria to the stream
> without having to formally do a search.  So the user could ask to see only
> messages with a certain tag, and/or by a certain user, and the stream should
> re-display with most recent (say) 20 messages from the now-filtered list.
>  Underr the covers I expect this to be implemented as an ad-hoc query,
> taking some search criteria from the basic stream definition and adding some
> terms from the user-selected criteria.
> o Tracking seems to me to be nothing more than stored queries that produce
> streams.  So for example when you say "show me my track of the ESME tag",
> the server runs a query against all the groups to which you have access for
> messages tagged with ESME, sorts them in reverse chronological order, and
> shows you the first (say) 20 results.
> o So far I've described timelines and tracking as ways to generate streams
> on-demand by running stored queries.  The only real difference is that at
> the UI level we are calling them different things for the user's benefit.
> o I think we should also make it possible for users to explicitly "do a
> search".
> o users should be able to create a query by specifying:
>        o which group pool(s) to search.
>        o what tags to look for.
>        o what date range to search.
>        o whether files, URLs, etc. are attached.
>        o what user sent them.
>        o etc.
> And to be able to show the results in various ways:
>        o list of messages.
>        o list of files or URLs sent as attachments.
>        o a graph of message volume over time.
>        o a chart showing the most/least active users.
>        o a cloud of tags, or users, or keywords, etc.
> o I think of a cloud as another kind of view onto the results of a query;
> although a cloud is special in that for performance reasons you need to
> pre-compute much of the results set (e.g. by indexing.).
> o So, for example, the cloud of tags used by JohnDoe would be the
> precompiled list of tags, filtered by their association with JohnDow.
> o Since cloud data has to be pre-compiled, we have to define what kinds of
> clouds we're interested in.  We currently have tag clouds, and that seems
> like it should stay.  We currently have word-frequency clouds, and I'm not
> sure we should keep this.  If we want to have user-participation clouds,
> etc. we'll need to say so up front.

Additionally we should provide the content of clouds also via an api. (this
decouples ESME from means to present the cloud content.

> o System administrators will need to see charts of different kinds to
> monitor system loading, performance, error rates, etc.
> o Maybe there are charts that end-users would like to see too.  For
> example, would group administrators want to see charts of activity levels
> over time?
> ____________________________________________________________
> o My idea for conversations is that if you reply to a message you now have
> a two-message conversation: which illustrates the rule that a conversation
> is a set of messages linked in a tree-shaped structure, linked from child to
> parent by virtue of the child being a reply to the parent.
> o Within any conversation tree there will thus be parents, children and
> siblings, and users might want to see any/all of those in "viewing" a
> conversation.
> o I could see one view of a conversation that is simple a
> forward-chronological listing (a stream) of the messages in the
> conversation.
> o But I could also image users wanting to navigate the tree to follow the
> threads of discourse; which means a different UI than just a list of
> messages.  Perhaps, then, we'd have to open conversations into their own
> windows to view them...
> ____________________________________________________________
> o So what is a message (and consequently, what can you do with it)?
> o To a first approximation, a message is a short text string, posted by a
> single user, to a designated message pool.
> o But there is also a lot of metadata that's automatically associated with
> the message:
>      -- Sender
>      -- Posting timepoint
>      -- Group (or pool)
>      -- is this a reply?
>      -- if so, then to what message?
>      -- etc.
>  o There is also metadata that the user can add at his/her discretion:
>       -- tags
>       -- directed recipients (via @username)
>       -- expiration timepoint?
>  o I suggest that we also allow users to attach "payloads" to messages.
>  o One form of payload is a URL.  A URL is a long text string, that if
> contained in the message body would make the message too long.  So we should
> make it possible to select a run of text in the message to use as the "name"
> of the URL, then to define the text of the URL elsewhere.  There could be
> two UIs for this:  For the novice, let the user type the URL in a separate
> text box.  For the expert, let the user use a special syntax to type the URL
> inline with the message body.
> o Another type of payload would me an email address...
> o Another type of payload could be a file...
> o In all cases we might require that each payload have a run of text in the
> message body to act as the name, anchor and access point for the payload.
>  payload names might be styled to look like HTML links; and clicking on them
> might invoke the payload (opening a link in a new window or tab, opening an
> email address in the user's mail client, downloading a file to the user's
> computer, etc.).
> o It should thus be possible for a message to have an arbitrary number of
> payloads.
> o If we require that every payload have some "name" in the body of the
> message, then it should be easy to copy and paste these names (and thus the
> payloads) between messages.  This would be useful, for example, when you
> want to share a file or a link from a message you received with another
> user; or to include it in a new message you are sending out.
> o My first idea about payloads that are files is that the ESME server
> should act like a file store and forward server.  As long as there is any
> message in any pool that references a given file we should store it, and
> allow it to be retrieved whenever a user asks for it.

Good point: if we see this extension, we should provide an interface for
virus scanners.
Another one: if there is a library to shorten urls by default, we could
incorporate it here.

> o Privacy and security is, to a first approximation, enforced by the group
> pools:  if a file is attached to a message, it can only be accessed by users
> in that pool.  If a user chooses to copy the name of (and hence a link to) a
> file and to paste it into a message posted to another pool, then it becomes
> available to that pool too; at the responsibility of the user.
> o One might ask how to remove a file from circulation once it's been
> attached to an intial message and uploaded to the ESME server.  Once might
> ask if the file has an owner, who can get a list of his/her files and force
> them to be deleted.  Reasonable questions.  But I'm wondering if it makes
> sense to treat files the same way we do URLs. Each URL points to a resource
> (typically a file of SOME sort) on the internet, and once published there's
> no way to retract the distribution of a URL string.  So maybe we could treat
> files the same way....

How would it be if we supply a personlized list of documents posted and an
option for the user (and the admin) to explicitly delete a certain file.

> o Or we might say that when the original message either expires (because
> the sender put an expiration timepoint on it) or is purged from the system
> (because there's a server policy in place that purges all messages older
> than a certain age), that only then does the file get deleted from the file
> store.

We should provide a means to archive conversations from ESME.

> o It's easy for a payload reference (in essence, perhaps just a URL) to be
> embedded in the message string itself if the message string is HTML.  If so,
> then we embed URLs as part of anchor tags and the web browser automagically
> displays just the name of the anchor while carrying with it the full URL
> string.
> o On the other hand, what happens when we forward a message to a
> non-HTML-based system; such as cellphone text messaging (SMS)?  In these
> cases we'd have to strip out the payload, and perhaps just insert a payload
> stub -- a special symbol that means "there used to be a payload attached to
> this word", etc.
> ____________________________________________________________
> o I'd also like us to nail down the at least the basic set of special
> characters and commands that can be embedded into messages.
> o We use @username to make sure a user sees a message (subject to access
> rules and depending on what timeline he/she's viewing).
> o We also use #tagname to mark embedded tags.
> o It also occurs to me that we could use $servicename to direct a message
> to a service (a machine that will respond to our message)..

Depends on whether we want to handle bots different from users, otherwise we
could call bots also via @

> .
> o Twitter has a number of letter commands, such as "d" at the beginning of
> a message to make it private. I remember seeing a long list of special
> commands recognized by Twitter, but they seemed: (a) fragile, and (b) hard
> to extend the command set.
> o I wonder about a command language with a syntax like "p: @fred @joe
> here's my message" to cause a private message to be sent to only fred and
> joe?  Does anyone have a handle on the set of commands we envision for our
> command-line language?
> Kind regards,


Daniel Koller
Jahnstrasse 20
80469 M√ľnchen *

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message