cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcelo F. Ochoa" <>
Subject Re: DO NOT REPLY [Bug 6879] - [PATCH] Cache improvement using ESI invalidation protocol (long)
Date Mon, 29 Apr 2002 14:04:25 GMT
Gerhard Froehlich wrote:

>The current Caching process is something like this:
>Cachable request ---> CachingxxxPipeline ---> lookup Store ---> Validate Object
>When I understand everything right you do something like
>                              Http Post Message
>                                     |
>                                    XSP
>                                     |
>                          External Cache Invalidator 
>                                     |
>                                     |
>                        Some Invalidator Server Thread                             
>                                     |
>                                     |
>                                     -
>(Ignore current process ---->) Validate Object
>So this is my current high level understandig of your approach.
>_Please_ correct me, if I understand something wrong here. Draw a 
>picture, that helps!
>Ahem it's the "ignore current process" part, which I don't like ;-)...
>Question: Why is it impossible for you to do it in the current Caching 
   I'll explain how the ExternalValidity Objecs work with a simple 
example. See below.

>>   2) In my test case the background Thread do not impact in the 
>>performance of Cocoon, that is, enabling or not the server through 
>>cocoon.xconf file the average response time of cocoon components is the 
>>same. One alternative is to leave the entry in cocoon.xconf commented 
>>and the user that need the behaviour uncomment it.
>Note: you're not the only thread in Cocoon. Every Thread has an direct impact 
>on the Performance. Multithreading is very complex and I made the expierence
>in Cocoon to implement Threads only when it is neccassary and for a
>generic use (see Berins proposals about event queues...)
 The idea of the threads is for invalidating a lot of pages a the same 
time. For example, ESI Invalidation protocols provides a way to 
invalidate pages with wilcards, for example invalidate the pages that 
starts with /doc/research/*., it mean that the invalidation server need 
to invalidate 1.. thousands of pages depending on how many pages the 
application stores into this path. In the other hand the idea of the 
invalidator server interface is provide the contract for other 
implementations, at this moment the store is a simple HashMap, but I 
have a plan to implement the store outside cocoon in Oracle tables to 
provide better response time for bulk invalidations messages.
Even queues will provides a better approach for this solution, because 
the Invalidator Server comsumes an internal queue of invalitation 
messages to not to block the XSP page that post this message.

>>   3) The logon in the invalidator xsp page uses standard http/https 
>>protocol and is a regular XSP page, that is, the security warkaround 
>>here is equal to any other xsp page. As an aditional security feature 
>>its possible to check the IP number of the sender. (Username/password 
>>and ip checking is the standard security filters implemented by Oracle 
>>Web Cache and I don't know any security problem with this comercial 
>>product). In the other hand this xsp page only parses an xml input 
>>message and signals the Cache Server to invalidate pages.
>Again, when implementing such a feature then IMO it must:
>(a) fit in the current Cocoon pipeline process 
>(b) and more generic, like a system wait security and authorization model
>which supports different protocols...
  I am trying to implement ESI standard ( 
and this protocol only support http/s invalidation messages. If I 
implement my own protocol, applications that where made for Oracle Web 
Cache for example will be hard to port to Cocoon architecture.

>>   4) This functionality bost the final performance of database 
>>intensive calculation. For example in a tradicional portal page like My 
>>Yahoo, the stocks quotes content is viewed by millions of user a day, if 
>>this stock quotes are calculated, obviusly, by an sql query in a target 
>>database, the XSP page that query this table never will be more quickly 
>>than the cached xsp content that only call at the method "isValid" of 
>>the ExternalCacheValidity object.
>>   In this situation the control of the content is inverted, that is, 
>>the origin of the sources signals at the cache instead of the provider 
>>ask for a new content. As a final result instead of to sent a jdbc 
>>message to the database for the content of the quotes in every page hit 
>>only a few messages are sent by the database when the table is updated.
>>  And thats all, if you have any other questions sent me an email,  I 
>>would like to receive positive or negative comment.
>This I believe and I can feel your pain. I think you ideas are valid but
>IMHO they must fit better into the current Cocoon pipeline architecture.
  Righ, I'll try.

>But it would be interesting what other devs. think of it.
>  ~Gerhard
 Supose this simple XSP page:

<?xml version="1.0" encoding="ISO-8859-1"?>
     <title>A Simple XSP Page with External Cache Invalidator Sync</title>

Unlike a regular XSP this page returns an ExternalCacheValidity object. 
This object implements Cocoon's CacheVality interface.
Using Torsten's graph ;) a regular XSP page work as is:

    FileGenerator src="my.xml" ---> TimeStampCacheValidity("my.xml")

An External invalidate XSP page work as is:

    FileGenerator src="my.xml" ---> ExternalCacheValidity("my.xml")  --> CacheStore(..,xx.xml,my.xml,yy.xml,..).

What is Cache Store?
  Cache Store is a simple store (now implemented by a HashMap) that 
identified the enconded url (my.xml and his arguments and cookies) with 
his Cocoon cache key (a java long value), note that the store only 
stores the encoded url and the long key, not the content of the xml page.
  When the event pipeline ask for the validity of the XSP page the 
ExternalCacheValidity object look into the store and return true or 
false of the given page is stored or not respectively. Subsequent 
exececutions of this xsp page work as is:

    FileGenerator src="my.xml" ---> ExternalCacheValidity("my.xml") --> CacheStore(..,xx.xml,my.xml,yy.xml,..).
    ExternalCacheValidity return true, then take the content from the Cocoon cache.

  The taglib <cache:registerPage/> registers the page in this store.
Well, at this moment the functionality is equal to the 
TimeStampCacheVality when the user do not modify the xsp page, because 
editing the page causes the invalidation of the page.
  Now, an External Invalidated page includes information that depend on 
external resources such as an SQL query, for example "select * from 
quotes;", on every hit of this XSP page the SQL query will return the 
same information until a database user insert, update or delete rows 
into the table quotes.
  When the database user modifys the quotes table a trigger attached to 
this table will send an http post message to the External Invalidator 
Server invalidating the page. A this time the invalidate.xsp page will 
be executed, this xsp interprets the ESI invalidation message and 
removes from the store the entry "my.xml". Here the flow:

    ESI invalidation message
    FileGenerator src="invalidate.xsp" -- remove --> CacheStore(..,xx.xml,yy.xml,..)
    ESI Invalidation result

  The next invocation of the my.xml will cause that 
ExternalCacheValidity object do not find his key in the store returning 
false, then the Cocoon Event pipeline will execute the page.

    FileGenerator src="my.xml" ---> ExternalCacheValidity("my.xml") --> CacheStore(..,xx.xml,yy.xml,..)
    ExternalCacheValidity return false, then execute my.xml xsp page again.

  Well, this explanation clarify the idea?
  I looking forward for your reply, best regards Marcelo.

Marcelo F. Ochoa -
Do you Know DB Prism? Look @
More info?
Chapter 21 of the book "Professional XML Databases" (Wrox Press
Chapter 8 of the book "Oracle & Open Source" (O'Reilly
Lab. de Sistemas - Fac. de Cs. Exactas - UNICEN
Paraje Arroyo Seco - Campus Universitario
(7000) Tandil - Bs. AS. - Argentina
Te: +54-2293-444430 Fax: +54-2293-444431

To unsubscribe, e-mail:
For additional commands, email:

View raw message