Mailing-List: contact esme-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: esme-dev@incubator.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Date: Wed, 5 Aug 2009 13:30:56 +0200 (CEST)
From: "dhague@fortybeans.com" <dhague@fortybeans.com>
To: esme-dev@incubator.apache.org
Message-ID: 
 <1489565421.75630.1249471856457.JavaMail.open-xchange@oxltgw15.schlund.de>
In-Reply-To: <cdbebedf0908041604w4d4196f8jf24f57ef41c43c04@mail.gmail.com>
References: <cdbebedf0908041604w4d4196f8jf24f57ef41c43c04@mail.gmail.com>
Subject: Re: Turtles all the way down (or how I learned to love math in
 computing)
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_75629_797136406.1249471856396"

------=_Part_75629_797136406.1249471856396
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Yes, yes and yes. Maybe. ;-)=20


I very much like the idea of having an elegant event-handling framework at =
the
heart of ESME. However, it is also critically important that what we delive=
r to
users does not require them to understand this.


By analogy: Access is based on SQL and the whole set-theory thing. Most Acc=
ess
developers neither know nor care about this.


In our case, we still need to deliver a *usable* micromessaging system - bu=
t if
that system has a really powerful core that is accessible to power develope=
rs,
then we have a huge opportunity.


Remember: the business case is not "there is a really powerful engine here"=
, but
is more "you can use this to do X and Y more reliably, and less expensively=
,
than any other alternative you may be considering".


So with those caveats, I am very much in favour of the new approach. With
suitable layering of code, API users and UI developers need not notice any
significant differences.


Cheers,
Darren

On 05 August 2009 at 01:04 David Pollak <feeder.of.the.bears@gmail.com> wro=
te:

> Folks,
>=20
> First, I'm going to give a code example:
>=20
> *transform:*
> tcw: (client) re: (what) [date] [timespan] =3D> {type: 'tcw' client: clie=
nt
> topic: duration length: timespan when: date or update_time}
>=20
> *update:*
> call log +=3D {type: 'tcw'}
>=20
> What does this mean?
>=20
> The transformation is pattern matching an incoming ESME update against
> anything that starts with the string "tcw:" followed by the name of a cli=
ent
> followed by the string "re:" followed by any text followed by an optional
> date and timespan will get transformed to a JSON object and included as p=
art
> of the update.
>=20
> The second line appends the update to the "call log" table.
>=20
> These two lines allow you to create an ESME-based call log application.
>=20
> Second, "oh wow, relational databases are turtles all the way down."
>=20
> I've only taken 3 CS courses in my life... and nearly got thrown out of o=
r
> gotten flunked out of all of them (no one likes a smart ass).=C2=A0 When =
I was in
> law school, I took a database grad-level course.=C2=A0 I thought it'd be =
an easy
> grade given that I'd written a moderately successful commercial flat file
> database.=C2=A0 It was a mind-blower that relational databases were (1) b=
ased on
> real math (set theory) and (2) that the database schema was defined by da=
ta
> in a set of tables.
>=20
> Over the years, I've been constantly reminded of the simple purity of
> relational databases.=C2=A0 They are meta-defined.=C2=A0 They are logs an=
d snapshots.
> They are immensely fast and powerful embodiments of math... not the
> messy/sloppy stuff of imperative programs.
>=20
> Stick with me... there's a little more free association here... but there=
 is
> a point that I hope will be wrapped up in a nice bow.
>=20
> Third, Twitter, VisiCalc, and DBase 2 (which is still better than a bilge
> pump <http://en.wikipedia.org/wiki/Ashton-Tate#Trivia>) were all, IMHO,
> groundbreaking applications.=C2=A0 Each one of them was dead simple to us=
e and at
> their core embrace composition.=C2=A0 Many small (and even medium sized)
> businesses are powered by Excel and FileMaker and/or Access (the linear
> successors).=C2=A0 All of the applications that have grown up around the =
140
> character messages demonstrate the power of a line of text.=C2=A0 VisiCal=
c
> (improved by 1-2-3, Excel, Mesa <http://www.plsys.co.uk/mesa.htm> and oth=
ers
> and brilliantly re-conceived by Improv) demonstrated the compositional po=
wer
> of programming by relating data cells to one another.=C2=A0 "Normal peopl=
e" can
> use FileMaker/Access to write a whole lot of powerful applications.=C2=A0=
 We can
> throw in some other marginal (but cool) apps including
> SBook<http://simson.net/ref/sbook5/>.
> These apps are high on "making sense of unstructured data" scale... which=
 I
> believe is critical to usability.
>=20
> Fourth, ESME's "actions" are a huge differentiator.=C2=A0 They embody a l=
ot of
> what external apps do with Twitter and put the same functionality (it's
> pretty hard coded) into ESME directly.=C2=A0 Vassil's excellent (and dang=
erous)
> suggestion of allowing Scala scripts to be actions triggered by matching
> updates was a great idea.=C2=A0 Further, Vassil (okay, this is all Vassil=
's
> fault) had the great idea of putting updates relating to user
> change/add/login/etc. into the user's timeline.=C2=A0 All this stuff driv=
es at
> (1) allowing users a ton of flexibility in behavior makes ESME different
> already and (2) everything should be an event.
>=20
> But... what if we took the theoretical underpinnings of "everything is an
> event" and merged it with pattern matching event "annotation" so that we =
can
> process events as they are sent (and as they arrive in our mailbox) and t=
hen
> modify tabular data structures based on matching the events and the event
> contents.=C2=A0 Formulas can be triggered based on the new calculations a=
nd
> trigger the creation of new updates and/or perform external actions.=C2=
=A0 But
> everything, I mean everything including system start-up, user creation, e=
tc.
> is all event based.=C2=A0 That means we've got a Twitter/VisiCalc/DBase t=
hingy
> that might be powerful and flexible because it allows business people to
> manage complex data stream in simple, composable (and debuggable) ways.
>=20
> An event-driven ESME with appropriately easy/powerful pattern matching an=
d
> data structure definition/mutation will, I believe, bring a lot of the po=
wer
> of VisiCalc/1-2-3/Excel and FileMaker/Access to the networked world.=C2=
=A0 People
> can write (or share) applications for time billing, project management, t=
o
> do lists, calendaring, etc.=C2=A0 But the front end is not part of the ap=
p... a
> web page can post to a REST interface that will result in an event being
> generated (think about the power of the forms-based front end to Google
> spreadsheets).=C2=A0 The back end is not part of the app... events are ju=
st made
> available to other services, UIs, etc.
>=20
> So, there's non-trivial work in terms of defining grammars for end users =
and
> building a system that is a meta self-hosted system (because if everythin=
g
> is an event, then users must be data structures stored just as any other
> user-defined data structure... turtles all the way down.)=C2=A0 On the ot=
her
> hand, once defined, it's a system that invites broader participation is
> "coding" because it's stuff that anyone who writes Excel formulas can
> contribute to.
>=20
> I think that on the 1 year anniversary (today is the 1 year anniversary g=
ive
> or take a week of my participation on ESME and the registration of esme.u=
s),
> we should explore what we could make of ESME.
>=20
> Personally, I have not found a business model around putting social
> messaging behind the firewall.=C2=A0 However, I see a very nice business =
in an
> ESME hub (that can bridge between federated behind the firewall ESME
> instances) that also provides "applications" built on top of the platform=
.
> This provides a nice incentive for me to spend a lot of time making ESME =
G2
> into something viable.=C2=A0 Further, we've seen a lot of problems baked =
into the
> "hurry up and get it out" current version of ESME including a very brittl=
e
> User class, UI that's part of the core Scala code, etc.=C2=A0 Also, ESME =
has been
> very XML oriented... I've been leaning a lot more towards JSON as data
> format.
>=20
> So... am I smoking crack or does the concept of G2 sound reasonable?=C2=
=A0 If it
> does sound reasonable and I could get something running by mid September,
> what do you folks think of putting ESME resources into G2?=C2=A0 If some =
of you
> are on the fence, what questions do you have to push you to one side of t=
he
> fence or the other?
>=20
> Thanks,
>=20
> David
>=20
> PS -- I've bcc'ed a couple of folks that I'm trying to lure into the ESME=
 G2
> project
> PPS -- I told Dick I'd run this email by him before sending it to the
> list... I didn't do that and I hope it does not catch him too unaware...
> especially coming back from vacation and all.
>=20
>=20
>=20
>=20
> --=20
> Lift, the simply functional web framework http://liftweb.net
> Beginning Scala http://www.apress.com/book/view/1430219890
> Follow me: http://twitter.com/dpp
> Git some: http://github.com/dpp
------=_Part_75629_797136406.1249471856396--