cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan <alan-coc...@engrm.com>
Subject Re: Momento and Cocoon [was Re: Jisp 3.0 moved to GPL licence]
Date Tue, 24 Feb 2004 00:44:28 GMT
* Geoff Howard <cocoon@leverageweb.com> [2004-02-24 00:31]:
> Upayavira wrote:
> 
> >[changing subject...]
> >
> >Reinhard Poetz wrote:
> >
> >>From: Alan
> >>
> >>>* Geoff Howard <cocoon@leverageweb.com> [2004-02-22 18:47]:
> >>>  
> >>>
> >>>>Alan wrote:
> >>>>    
> >>>>
> >>>>>* Upayavira <uv@upaya.co.uk> [2004-02-22 07:58]:
> >>>>>      
> >>>>>
> >>>>>>I tend to think that Momento isn't suited to this need.     
 
> >>>>>
> >>>>>>However, as an XML data repository, it seems very interesting.
> >>>>>> 
> >>>>>
> >>>>>I've got a better idea of how Jisp is used in Cocoon from reading
> >>>>> all the discussion after my post.
> >>>>> 
> >>>>> I suggested Momento because someone suggested Xindice which led
> >>>>> me to believe Jisp handled an XML persistence task.
> >>>>>
> >>>>> Might not be the best bet, no.   
> >>>>> 
> >>>>
> >>>>Still, I think finding a way to use momento to reduce     
> >>>
> >>>memory overhead   
> >>>
> >>>>in
> >>>>working with large xml datasets has great potential.  No one really 
> >>>>knows how great, but a demo/sample using it would be a     
> >>>
> >>>start...  (hint   
> >>>
> >>>>hint :)  )
> >>>>    
> >>>
> >>>Working on it. As noted, I have JAXP implemented and SAX interface
> >>>   to XUpdate. I have APIs. I am going to start working on services
> >>>   next.
> >>
> 
> JAXP... see below....
> 
> >>>      A Cocoon generator that takes a Momento data source and an XSLT
> >>>   transform would be a start.
> >>>
> >>>   I'm not sure how to get information into Momento via Cocoon. I'm
> >>>   thinking about some sort of Woody binding, but that goes beyond
> >>>   my current understanding of Cocoon.
> >>>  
> >>
> >>speaking without following this thread closly: What about 
> >>implementing a Momento source? 
> >
> 
> I starting to wonder if I'm being dense... wouldn't the easiest first 
> test integratin be to use Memento as the JAXP xslt processor to reduce 
> memory overhead on transformations of large data sets?  Maybe I've 
> misunderstood where/what momento is as a project?   The jaxp processor 
> is declared in cocoon.xconf  (see instructions for switching to saxon 
> for example).

I created a blog entry today:

 Momento Inline
 2004/02/23 12:26:54

 There is a mode of operation for Momento that I've not considered
 at length. Inline operation with XSLT. That is, operation where
 Momento is not a data source, rather it is a transient document
 object model.

 This applies when performing a transform against a document that
 may be too large to fit in system memory. A common use case is XML
 generated from an SQL query. The SQL result set can be streamed as
 a series of SAX events that clogs memory as the XSLT engine tries
 to build a document representing a large data set.

 There is nothing preventing Momento from building a document,
 organized and clustered at the get go. More interesting would be
 for Momento to build the document in memory, writing it to disk
 only when memory runs out.

 Currently, Momento writes its pages as it fills them. Momento might
 delay a page write until a page fills, when it makes sense to do
 so, but it pretty much writes the pages to disk as it writes nodes
 and strings to the pages. It pools pages in memory using weak
 references. When no one is writing to or reading from a page, the
 weak reference will be the only reference, thus it is eligible for
 garbage collection. Momento would have to intercept the garbage
 collector's desire to release the page, and write it out before the
 memory is released.

 This means I need to develop a deeper understanding of weak
 references in Java. If there is no way to hook collectection before
 the fact, I'd have to rethink the paging engine so that it would
 explicitly release pages as part of a MRU cache.

 A hybrid of this could be used to maintain XSLT output as part of a
 cache system. Momento would build the result of an XSLT transform
 organized and clustered, writing it to disk when memory is tight. A
 cache could use this Momento document as a source of SAX events or
 as a W3 DOM document, discarding it when the upstream dependencies
 change. 

So, no Geoff, I'm only getting around to thinking of Momento as an
    transient document object model. I'd designed it as a persistence
    engine, so I'd considered Momento to be a source of data, a
    place where data lives.

(I'll get back to everyone else soon. Great stuff everyone. Thank
    you so very much!)

-- 
Alan / alan@engrm.com / http://engrm.com/
    aim/yim: alanengrm - icq: 228631855 - msn: alanengrm@hotmail.com

Mime
View raw message