incubator-wave-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Hansen <a2br...@gmail.com>
Subject Re: Help me test algorithm performance
Date Fri, 09 Aug 2013 18:15:43 GMT
I'd like to help; this looks like a place where I can pitch in.  Please
include me?


On Thu, Aug 8, 2013 at 2:22 PM, Terry Hardin <thardin137@gmail.com> wrote:

> I would like to participate.
>
>
> On Thu, Aug 8, 2013 at 3:02 PM, Zachary “Gamer_Z.” Yaro <zmyaro@gmail.com
> >wrote:
>
> > I will definitely help out if I can.
> >
> >
> > —Zachary “Gamer_Z.” Yaro
> >
> >
> > On 8 August 2013 16:29, Yuri Z <vega113@gmail.com> wrote:
> >
> > > I think you can create an event on Google+.
> > >
> > >
> > > On Thu, Aug 8, 2013 at 11:15 PM, Joseph Gentle <josephg@gmail.com>
> > wrote:
> > >
> > > > tldr; I need some volunteers to collaboratively edit a document
> > > > together, so we can systematically evaluate algorithmic performance.
> > > >
> > > >
> > > > So recently Michael linked me to a paper[1] which evaluates a bunch
> of
> > > > different concurrency algorithms on speed & memory usage. They got
a
> > > > bunch of students to collaboratively edit two documents and used the
> > > > operations generated in their benchmarks.
> > > >
> > > > The paper has some glaring omissions[2], and the data they gathered
> > > > isn't publicly available. Of course, I also want to test Torben's
> > > > algorithm to see how well it performs with realistic usage.
> > > >
> > > > So I'd like to reproduce their experiment. To do this I need a few
> > > > volunteers to collaboratively edit some documents. We should
> construct
> > > > realistic editing scenarios. The paper did two things:
> > > > - Transcribe an episode of big bang theory
> > > > - Write a report
> > > > I'm open to suggestions on what we should do - we could also try
> > > > collaborative creative writing, writing notes on a youtube video, or
> > > > something. It doesn't really matter so long as the activity is
> > > > focused, realistic (no keyboard mashing) and involves collaboration.
> > > > (Sequential editing scenarios aren't interesting)
> > > >
> > > > To do this, I'll set up a special instance of ShareJS with ~1s of
> > > > artificially induced latency and extra logging for the experiment. I
> > > > want to run this experiment either late next week or on the weekend.
> > > >
> > > > The more experimental runs the better - although I suspect most of
> > > > what we learn will be from the first couple logs.
> > > >
> > > > I will publish the raw data from the logs and send out a followup
> > > > email. The experiment will be anonymous, but don't say anything you
> > > > wouldn't want publicly known.
> > > >
> > > > How does that sound? Who's willing to help out?
> > > >
> > > > -J
> > > >
> > > >
> > > >
> > > > [1]
> > > >
> > >
> >
> http://hal.archives-ouvertes.fr/docs/00/62/95/03/PDF/doce63-ahmednacer.pdf
> > > > [2] Criticisms:
> > > > - Operations only insert or remove a single character, which means
> > > > that a copy+paste that one of the users did resulted in 5000
> > > > operations, each of which needed to be transformed individually.
> > > > - Their text editor didn't batch changes - which is really stupid and
> > > > unrealistic.
> > > > - The students were all working locally (on a LAN), so there would
> > > > have been fewer concurrent actions than we should realistically
> > > > expect.
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message