jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Stocker <christian.stoc...@liip.ch>
Subject Re: Add more options to make Jackrabbit more failsafe and/or scale-out
Date Wed, 11 May 2011 10:54:46 GMT


Just one thing to avoid missunderstandings.

On 11.05.11 12:01, Bart van der Schans wrote:
> Hi Christian,

> Does anybody have some pointer why it's trying to write? It seem to
> shutdown just fine anyway.. It doesn't make much sense to me to create
> a ReadOnlyDatabaseFileSystem just to avoid this shutdown issue.
> I think the solution I created is a bit more narrow as it only aims to
> allow to run JR on a database slave node independently. I think
> Christian's idea can be expanded to separate all write database
> operations from read operations in JR, which would allow for scaling
> out in heavy read dependent environments  with multiple db slaves
> while keeping a single db master for handling all writes.

My patch only seperates the read/writes for the cluster journal
operations. It doesn't seperate the "common" read/write request handled
by the PM. This of course only works, if your Jackrabbit instances using
the slaves only do reads and no PM-writes at all. You should handle the
seperation in your application (read from the read-only Jackrabbit, but
write to the write-enabled Jackrabbits), the same as you would do, you
if you would use a traditional master/slave MySQL Setup (with no
Jackrabbit or other CR at all)


> I'm really interested in how other devs dealt with database slaves and
> their opinion about the solutions provided by Christian and me.
> Regards,
> Bart
> On Wed, May 4, 2011 at 3:25 PM, Christian Stocker
> <christian.stocker@liip.ch> wrote:
>> Hi All
>> I made a broader blog post to the topic at
>> http://blog.liip.ch/archive/2011/05/04/how-to-make-jackrabbit-globally-distributable-fail-safe-and-scalable-in-one-go.html
>> How should I proceed if I'd like to get that into Jackrabbit? Just open
>> a ticket, add the patch and hopefully someone takes it from there? Or do
>> you think it doesn't have a chance to get into the jackrabbit-core?
>> greetings
>> chregu
>> On 02.05.11 14:48, Christian Stocker wrote:
>>> On 02.05.11 14:43, Bart van der Schans wrote:
>>>> On Mon, May 2, 2011 at 1:39 PM, Christian Stocker
>>>> <christian.stocker@liip.ch> wrote:
>>>>> Hi all
>>>>> My favourite topic again. Building a fail-safe and/or scalable
>>>>> jackrabbit setup.
>>>>> We had the wish to make our setup datacenter-fail resistant. eg. if one
>>>>> DC goes down, we can still serve pages from a backup jackrabbit
>>>>> instance. We use MySQL as  perstistant store, this is no given, but I
>>>>> guess the problems are everywhere the same.
>>>>> With a traditional setup, if the main DC goes down, your Store goes down
>>>>> with it and the jackrabbit instance in the other DC can't access it
>>>>> anymore as well. That's why we thought about replicating the MySQL DB
>>>>> the 2nd DC and just read from there (we can make sure that nothing
>>>>> writes to the backup jackrabbit instance). This works fine. As we can
>>>>> already point the cluster journal "store" to another place than the PM,
>>>>> we just point the journal store to the central one in the 1st DC and
>>>>> read the data from the PM in the MySQL slave in the 2nd DC. A read-only
>>>>> jackrabbit only has to write to the journal table and nowhere else
>>>>> AFAIK, so that works well even with replicating MySQLs.
>>>>> All fine and good and even if the master MySQL goes down the Jackrabbit
>>>>> instance in the 2nd DC serves its nodes as nothing happened.
>>>>> The one problem which there is left is that there's a replication lag
>>>>> between the master and the slave MySQL (there's one, even if the sit
>>>>> just besides each other). What can happen with this is that a writing
>>>>> jackrabbit writes a new node and the journal entry and then the backup
>>>>> jackrabbit reads from the journal (from the mysql master) but the actual
>>>>> content didn't end up in the mysql slave (where the backup jackrabbit
>>>>> reads its PM data from). This can easily be tested with stopping the
>>>>> mysql replication.
>>>>> The solution I came up with was to read the journal entries also from
>>>>> the MySQL slave (but still write the LOCAL_REVISION to the master). With
>>>>> this we can make sure the jackrabbit in the 2nd DC only reads entries,
>>>>> which are already in its mysql slave. A patch which makes this work is
>>>>> https://gist.github.com/951467
>>>>> The only thing I had to change was to read the "selectRevisionsStmtSQL"
>>>>> from the slave instead of the master, the rest can still go to the master.
>>>>> What do you think of this approach? Would this be worth adding to
>>>>> jackrabbit? Any input for the patch what I could improve?
>>>>> Besides the fail-over scenario you also can easily do scaling with that
>>>>> approach, so you can serve your "read-only" webpages from a totally
>>>>> differnt DC without having too much traffic between the DCs (it's
>>>>> basically just the MySQL replication traffic). That's why I didn't want
>>>>> to read from the Master in the backup jackrabbit and only switch to the
>>>>> replicating slave, when things fail (which would be a solution, too,
>>>>> course)
>>>>> any input is appreciated
>>>> I've played many times with the idea of creating some kind SlaveNode
>>>> next to the ClusterNode which only needs read access to the database
>>>> (slave). I don't think the local revision of the slave isn't much use
>>>> to the master so that could be kept on disk locally with the slave.
>>> AFAICT, the janitor needs to know, where all the cluster-instances are
>>> to safely delete everything it isn't needed anymore. That's why it needs
>>> to be stored in a central place.
>>> chregu

Liip AG  //  Feldstrasse 133 //  CH-8004 Zurich
Tel +41 43 500 39 81 // Mobile +41 76 561 88 60
www.liip.ch // blog.liip.ch // GnuPG 0x0748D5FE

View raw message