esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Pollak <>
Subject Turtles all the way down (or how I learned to love math in computing)
Date Tue, 04 Aug 2009 23:04:04 GMT

First, I'm going to give a code example:

tcw: (client) re: (what) [date] [timespan] => {type: 'tcw' client: client
topic: duration length: timespan when: date or update_time}

call log += {type: 'tcw'}

What does this mean?

The transformation is pattern matching an incoming ESME update against
anything that starts with the string "tcw:" followed by the name of a client
followed by the string "re:" followed by any text followed by an optional
date and timespan will get transformed to a JSON object and included as part
of the update.

The second line appends the update to the "call log" table.

These two lines allow you to create an ESME-based call log application.

Second, "oh wow, relational databases are turtles all the way down."

I've only taken 3 CS courses in my life... and nearly got thrown out of or
gotten flunked out of all of them (no one likes a smart ass).  When I was in
law school, I took a database grad-level course.  I thought it'd be an easy
grade given that I'd written a moderately successful commercial flat file
database.  It was a mind-blower that relational databases were (1) based on
real math (set theory) and (2) that the database schema was defined by data
in a set of tables.

Over the years, I've been constantly reminded of the simple purity of
relational databases.  They are meta-defined.  They are logs and snapshots.
They are immensely fast and powerful embodiments of math... not the
messy/sloppy stuff of imperative programs.

Stick with me... there's a little more free association here... but there is
a point that I hope will be wrapped up in a nice bow.

Third, Twitter, VisiCalc, and DBase 2 (which is still better than a bilge
pump <>) were all, IMHO,
groundbreaking applications.  Each one of them was dead simple to use and at
their core embrace composition.  Many small (and even medium sized)
businesses are powered by Excel and FileMaker and/or Access (the linear
successors).  All of the applications that have grown up around the 140
character messages demonstrate the power of a line of text.  VisiCalc
(improved by 1-2-3, Excel, Mesa <> and others
and brilliantly re-conceived by Improv) demonstrated the compositional power
of programming by relating data cells to one another.  "Normal people" can
use FileMaker/Access to write a whole lot of powerful applications.  We can
throw in some other marginal (but cool) apps including
These apps are high on "making sense of unstructured data" scale... which I
believe is critical to usability.

Fourth, ESME's "actions" are a huge differentiator.  They embody a lot of
what external apps do with Twitter and put the same functionality (it's
pretty hard coded) into ESME directly.  Vassil's excellent (and dangerous)
suggestion of allowing Scala scripts to be actions triggered by matching
updates was a great idea.  Further, Vassil (okay, this is all Vassil's
fault) had the great idea of putting updates relating to user
change/add/login/etc. into the user's timeline.  All this stuff drives at
(1) allowing users a ton of flexibility in behavior makes ESME different
already and (2) everything should be an event.

But... what if we took the theoretical underpinnings of "everything is an
event" and merged it with pattern matching event "annotation" so that we can
process events as they are sent (and as they arrive in our mailbox) and then
modify tabular data structures based on matching the events and the event
contents.  Formulas can be triggered based on the new calculations and
trigger the creation of new updates and/or perform external actions.  But
everything, I mean everything including system start-up, user creation, etc.
is all event based.  That means we've got a Twitter/VisiCalc/DBase thingy
that might be powerful and flexible because it allows business people to
manage complex data stream in simple, composable (and debuggable) ways.

An event-driven ESME with appropriately easy/powerful pattern matching and
data structure definition/mutation will, I believe, bring a lot of the power
of VisiCalc/1-2-3/Excel and FileMaker/Access to the networked world.  People
can write (or share) applications for time billing, project management, to
do lists, calendaring, etc.  But the front end is not part of the app... a
web page can post to a REST interface that will result in an event being
generated (think about the power of the forms-based front end to Google
spreadsheets).  The back end is not part of the app... events are just made
available to other services, UIs, etc.

So, there's non-trivial work in terms of defining grammars for end users and
building a system that is a meta self-hosted system (because if everything
is an event, then users must be data structures stored just as any other
user-defined data structure... turtles all the way down.)  On the other
hand, once defined, it's a system that invites broader participation is
"coding" because it's stuff that anyone who writes Excel formulas can
contribute to.

I think that on the 1 year anniversary (today is the 1 year anniversary give
or take a week of my participation on ESME and the registration of,
we should explore what we could make of ESME.

Personally, I have not found a business model around putting social
messaging behind the firewall.  However, I see a very nice business in an
ESME hub (that can bridge between federated behind the firewall ESME
instances) that also provides "applications" built on top of the platform.
This provides a nice incentive for me to spend a lot of time making ESME G2
into something viable.  Further, we've seen a lot of problems baked into the
"hurry up and get it out" current version of ESME including a very brittle
User class, UI that's part of the core Scala code, etc.  Also, ESME has been
very XML oriented... I've been leaning a lot more towards JSON as data

So... am I smoking crack or does the concept of G2 sound reasonable?  If it
does sound reasonable and I could get something running by mid September,
what do you folks think of putting ESME resources into G2?  If some of you
are on the fence, what questions do you have to push you to one side of the
fence or the other?



PS -- I've bcc'ed a couple of folks that I'm trying to lure into the ESME G2
PPS -- I told Dick I'd run this email by him before sending it to the
list... I didn't do that and I hope it does not catch him too unaware...
especially coming back from vacation and all.

Lift, the simply functional web framework
Beginning Scala
Follow me:
Git some:

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message