esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jack Park <>
Subject Re: Scalability was Re: What are pools? was Re: [VOTE] Approve the release of apache-esme-1.3
Date Thu, 04 Aug 2011 00:08:18 GMT
Looking in the distribution, I see MongoDB, and CouchDB, plus general db 
paths. Looking through code, I've seen Mongo and Couch Lift projects at 
github, and have code for "vendors" like Derby and H2. I've not seen a 
mysql vendor yet, but guess (without yet trying) that you just insert 
the right class path and URL to use MySQL.

I've not seen any other nosql projects; am interested in graph 
databases, e.g. as driven by tinkerpop DSLs (which may soon migrate to 
scala), or triplestores. Dream on!


On 8/3/2011 4:05 PM, Vladimir Ivanov wrote:
> Ethan,
> Correct me if I'm wrong, but at the moment ESME should be able to run on any
> *relational* database due to all entities are stored/retreived via Lift
> Mapper persistence layer which implies traditional relational store. It's
> absolutly possible to use Lift Record persistence layer though which is
> intended for NoSQL databases, such as MongoDB.
> Vladimir
> 2011/8/3 Jack Park<>
>> Thanks, Ethan.
>> I'd like to know more about using MySQL.
>> Indeed, I'd like to know more about other classes of databases, e.g. graph
>> databases.
>> Still, while hit rate is one aspect of scalability, when you talk about
>> "safely cached", I wonder about the combinatoric aspect of scalability, say,
>> a million users each with timelines in the hundreds to thousands long, and
>> all the entailed message objects. Are you thinking in terms of, say,
>> external memcached solutions, or just large memory models and load
>> balancers?
>> Curious minds...
>> Thanks
>> Jack
>> On 8/3/2011 4:56 AM, Ethan Jewett wrote:
>>> Hi Jack,
>>> It should be able to run on any Java application server with any
>>> database via JNDI or with any JDBC database by providing JDBC
>>> connection information in a .props file. I believe that on Stax (now
>>> CloudBees) we are using a MySQL database.
>>> Regarding scalability, the design of ESME is such that it doesn't rely
>>> much on the database for scalability. Operations should generally
>>> happen on data in memory, and there shouldn't be access of the
>>> database on most operations. We can do this because the data model has
>>> messages as immutable objects (you can't change or delete them), so
>>> they can be safely cached and we can rest assured that they will never
>>> be out of sync with the database.
>>> I'm fairly sure we did some performance tests a while ago that showed
>>> ESME was able to support thousands of concurrent users on a fairly
>>> small server without trouble, but I can't currently find that
>>> documentation. Maybe someone else knows where it ran off to? Seems
>>> like something we should have on a our website (and a test that we
>>> should redo at some point).
>>> Cheers,
>>> Ethan

View raw message