esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ethan Jewett <>
Subject Re: Scalability was Re: What are pools? was Re: [VOTE] Approve the release of apache-esme-1.3
Date Wed, 03 Aug 2011 11:56:32 GMT
Hi Jack,

It should be able to run on any Java application server with any
database via JNDI or with any JDBC database by providing JDBC
connection information in a .props file. I believe that on Stax (now
CloudBees) we are using a MySQL database.

Regarding scalability, the design of ESME is such that it doesn't rely
much on the database for scalability. Operations should generally
happen on data in memory, and there shouldn't be access of the
database on most operations. We can do this because the data model has
messages as immutable objects (you can't change or delete them), so
they can be safely cached and we can rest assured that they will never
be out of sync with the database.

I'm fairly sure we did some performance tests a while ago that showed
ESME was able to support thousands of concurrent users on a fairly
small server without trouble, but I can't currently find that
documentation. Maybe someone else knows where it ran off to? Seems
like something we should have on a our website (and a test that we
should redo at some point).


On Tue, Aug 2, 2011 at 7:01 PM, Jack Park <> wrote:
> Thanks!
> Changing subjects again:
> Running on Derby,it seems to me, is not a path to a scalable platform. What
> is known or thought possible in terms of scalability?
> Thanks
> Jack
> On 8/2/2011 5:51 AM, Ethan Jewett wrote:
>> Hi Jack,
>> On the server I am
>> seeing that users in pools with "Write" privilege can read and write.
>> *However* there might be a bug where a user that hasn't logged in
>> recently doesn't receive messages from pools. It is possible the
>> user's mailbox is not being set up properly. We'll need to look into
>> it.
>> Thanks,
>> Ethan
>> On Fri, Jul 29, 2011 at 5:24 PM, Jack Park<>
>>  wrote:
>>> Ethan,
>>> That makes perfect sense.
>>> In 1.2, I am noticing that if I assign "write" to "joe" in a pool, "joe"
>>> can, indeed, write, but not read. Hope that's fixed in the newer
>>> versions.
>>> Thanks
>>> Jack
>>> On 7/28/2011 11:47 PM, Ethan Jewett wrote:
>>>> Hi Jack,
>>>> Just by way of explanation, I think that there have been some
>>>> significant changes to the timeline Javascript files between 1.2, 1.3,
>>>> and the upcoming 1.4. Most likely your browser was trying to use a
>>>> cached 1.3 .js file with the 1.2 backend code-base. That would
>>>> definitely not work as we changed some function and variable names, so
>>>> the backend was spitting out JS variables and functions to the browser
>>>> but the javascript already running in the browser had no idea what to
>>>> do with it.
>>>> Hopefully that explains the problem. Sorry for the apparent flakiness.
>>>> Cheers,
>>>> Ethan
>>>> On Fri, Jul 29, 2011 at 5:42 AM, Jack Park<>
>>>>  wrote:
>>>>> My "virgin" browser was IE whatever comes with Win7. The others are
>>>>> Chrome
>>>>> and FF. Clearing their cache fixed it.  Everything is running well now,
>>>>> with
>>>>> the 1.2 distro.  I have not migrated back to trunk or 1.3 yet. Overall,
>>>>> it
>>>>> was the cache issue, but I did drop the database and start over. I now
>>>>> have
>>>>> three users running in it.
>>>>> Created quite a few messages, some into a shared pool for two of the
>>>>> users.
>>>>> I gave a member Write roll. He can write and the admin sees, but he
>>>>> cannot
>>>>> read.  Not sure if that's a bug?
>>>>> Looking forward to the latest build running well.
>>>>> Thanks
>>>>> Jack
>>>>> On 7/28/2011 8:18 PM, Richard Hirsch wrote:
>>>>>> Did you delete the old db? If you deleted the old db, then the old
>>>>>> users are gone and you have to sign up again.  If it was the first
>>>>>> message after starting then it might take a while for the message
>>>>>> appear.  Create 5 messages over a few minutes time span and see
if the
>>>>>> problem still exists.
>>>>>> What browser are you using? There was an older bug with
>>>>>> InternetExplorer that some messages weren't being displayed.
>>>>>> On Fri, Jul 29, 2011 at 5:09 AM, Jack Park<>
>>>>>>  wrote:
>>>>>>> I did a clean rebuild of the 1.2 distribution and ran mvn jetty:run
>>>>>>> Logging in does not work at all. If I Sign Up and create a user,
>>>>>>> I'm
>>>>>>> in
>>>>>>> and I can send a message with a tag. The tag shows, the timeline
>>>>>>> not.
>>>>>>> But, when clicking users, that user's last message is visible.
>>>>>>>  Something
>>>>>>> fails in painting timelines.
>>>>>>> Next trial will be to attempt to replicate this on a completely
>>>>>>> different
>>>>>>> computer.
>>>>>>> On 7/28/2011 7:55 PM, Jack Park wrote:
>>>>>>>> All trials were with mvn jetty:run.
>>>>>>>> Thanks for the pointer: every database is separate.
>>>>>>>> I just noticed that each, whether I'm using it or not, has
a db.lck
>>>>>>>> I just ran an experiment: removed the db.lck file in the
>>>>>>>> distribution (I don't know that much about Derby) and now
it refuses
>>>>>>>> to
>>>>>>>> respond to the Sign On button.
>>>>>>>> Still fiddling.
>>>>>>>> Jack
>>>>>>>> On 7/28/2011 7:46 PM, Richard Hirsch wrote:
>>>>>>>>> If you are using the integrated derby database, then
you are
>>>>>>>>> probably
>>>>>>>>> using it for multiple versions - which might not be a
good thing. I
>>>>>>>>> haven't tried it yet. Look for a folder called esme_db.
>>>>>>>>> If you are using tomcat, try a clean build and then run
esme with
>>>>>>>>> jetty:run to see if the problem still exists. You may
also want to
>>>>>>>>> delete your browser cookies. That might be a problem
as well.
>>>>>>>>> D.
>>>>>>>>> On Fri, Jul 29, 2011 at 4:39 AM, Jack
>>>>>>>>> Park<>
>>>>>>>>> wrote:
>>>>>>>>>> What I mean is that the timelines stopped working
entirely, but
>>>>>>>>>> the
>>>>>>>>>> tags
>>>>>>>>>> that were visible (this is the 1.2 distribution tarball).
>>>>>>>>>> I shut the system down and built the 1.3 rc-1 distribution
>>>>>>>>>> and ran
>>>>>>>>>> that only to discover that tags were not working
there. I then
>>>>>>>>>> went
>>>>>>>>>> back to
>>>>>>>>>> the original 1.2 distro, booted it, noted my two
users could still
>>>>>>>>>> log in,
>>>>>>>>>> but there are no timelines visible. Sending messages
do not appear
>>>>>>>>>> anywhere.
>>>>>>>>>> I then went and booted the subversion 1.2 rc-1 tag;
no tags, and
>>>>>>>>>> adding a
>>>>>>>>>> user to a pool fails.
>>>>>>>>>> I then went and booted the subversion trunk. Same
>>>>>>>>>> All that to say that I had active timelines in the
>>>>>>>>>> distribution
>>>>>>>>>> until I
>>>>>>>>>> started running other distributions. By way of weak
inference, I
>>>>>>>>>> have
>>>>>>>>>> to
>>>>>>>>>> wonder if there is anything like a shared database
among those
>>>>>>>>>> builds.
>>>>>>>>>> Jack
>>>>>>>>>> On 7/28/2011 7:33 PM, Richard Hirsch wrote:
>>>>>>>>>>> What do you mean by " no messages sent from it
show up anywhere"?
>>>>>>>>>>> There are no messages in the timeline? Tagged
>>>>>>>>>>> D.
>>>>>>>>>>> On Thu, Jul 28, 2011 at 8:21 PM, Jack
>>>>>>>>>>> Park<>
>>>>>>>>>>> wrote:
>>>>>>>>>>>> I returned to 1.2 and found, now, that it
remembers the tags,
>>>>>>>>>>>> but
>>>>>>>>>>>> not any
>>>>>>>>>>>> of
>>>>>>>>>>>> the messages. In fact, no messages sent from
it show up
>>>>>>>>>>>> anywhere.
>>>>>>>>>>>> Time
>>>>>>>>>>>> for
>>>>>>>>>>>> some coffee, I suppose.
>>>>>>>>>>>> On 7/28/2011 11:10 AM, Richard Hirsch wrote:
>>>>>>>>>>>>> I just tried it on the stax instance
and added the user
>>>>>>>>>>>>> "acmebob"
>>>>>>>>>>>>> to a
>>>>>>>>>>>>> pool and it worked.
>>>>>>>>>>>>> Regarding the "Edit" button for pools.
Right now it is not
>>>>>>>>>>>>> correctly
>>>>>>>>>>>>> labeled, because you can just add users.
But we plan to be able
>>>>>>>>>>>>> to
>>>>>>>>>>>>> change other parts of the group (rename,
etc) and then
>>>>>>>>>>>>> hopefully
>>>>>>>>>>>>> the
>>>>>>>>>>>>> "edit" label will be correct.
>>>>>>>>>>>>> D.
>>>>>>>>>>>>> On Thu, Jul 28, 2011 at 8:06 PM, Jack
>>>>>>>>>>>>> Park<>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> Where the messages never stop!
>>>>>>>>>>>>>> Went back to 1.2 with what I learned
playing with 1.3.
>>>>>>>>>>>>>> 1.2 lets me add "joe" as a user to
my pool.
>>>>>>>>>>>>>> So, I suspect 1.3 is broken in terms
of adding users to pools.
>>>>>>>>>>>>>> Jack
>>>>>>>>>>>>>> On 7/28/2011 10:59 AM, Jack Park
>>>>>>>>>>>>>>> Slight update. What I know about
pools came from 1.2. In 1.3,
>>>>>>>>>>>>>>> pools
>>>>>>>>>>>>>>> behave a bit differently, or
so it seems. I found that the
>>>>>>>>>>>>>>> (misnamed?)
>>>>>>>>>>>>>>> Edit button lets me add a member.
I tried adding the other
>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>> ("joe").
>>>>>>>>>>>>>>> Clicking add user does nothing.
"joe" is a real user visible
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> user's list.
>>>>>>>>>>>>>>> I hope to learn what you can
do when you have a pool. Can you
>>>>>>>>>>>>>>> assign
>>>>>>>>>>>>>>> messages to the pool? Can pools
be private?
>>>>>>>>>>>>>>> Jack
>>>>>>>>>>>>>>> On 7/28/2011 10:39 AM, Jack Park
>>>>>>>>>>>>>>>> Now that I've got your attention,
may I ask for links to
>>>>>>>>>>>>>>>> explanations
>>>>>>>>>>>>>>>> about pools and, while we're
at it, tokens? I have some
>>>>>>>>>>>>>>>> hints
>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>> tokens, but absolutely no
clue about pools. I created one,
>>>>>>>>>>>>>>>> thinking
>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>> might be a "group" (a feature
I want), but when I came in as
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>> user, I was not able to "join"
that pool.
>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>> Jack

View raw message