avalon-cvs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From cziege...@apache.org
Subject cvs commit: avalon-excalibur/site/xdocs store-janitor.xml store-index.xml store-mru.xml index.xml
Date Mon, 10 Nov 2003 09:07:08 GMT
cziegeler    2003/11/10 01:07:08

  Modified:    site/xdocs index.xml
  Added:       site/xdocs store-janitor.xml store-index.xml store-mru.xml
  Log:
  Restoring store docs, not fully converted yet
  
  Revision  Changes    Path
  1.4       +1 -1      avalon-excalibur/site/xdocs/index.xml
  
  Index: index.xml
  ===================================================================
  RCS file: /home/cvs/avalon-excalibur/site/xdocs/index.xml,v
  retrieving revision 1.3
  retrieving revision 1.4
  diff -u -r1.3 -r1.4
  --- index.xml	29 Oct 2003 17:01:30 -0000	1.3
  +++ index.xml	10 Nov 2003 09:07:08 -0000	1.4
  @@ -126,7 +126,7 @@
           protocols such as myxmldatabase://root/documents/test.xml.
         </p>
                   </td></tr>
  -                <tr><td>Store</td><td>
  +                <tr><td><a href="store-index.html">Store</a></td><td>
         <p>
             The Store Package provides a way to store named objects without 
             taking care about where and how they are stored.
  
  
  
  1.1                  avalon-excalibur/site/xdocs/store-janitor.xml
  
  Index: store-janitor.xml
  ===================================================================
  <?xml version="1.0" encoding="UTF-8"?>
  <document>
      <properties>
          <author email="dev@avalon.apache.org">Avalon Documentation Team</author>
          <title>The Store Janitor</title>
      </properties>
  
      <body>
    <section name="Goal">
      <p>This document describes the usage of the StoreJanitor.</p>
    </section>
    <section name="Description">
    <p>The implementation is quit simple! Every implementation of a Store can register
in the 
    StoreJanitor. It checks in a configurable interval if memory is running low. If low, 
    it greps via Round Robin a victim (Store) and frees xx% of all emlements in this Store.

    After that the StoreJanitor sleeps and waits for the next iteration.</p>
    <p>The StoreJanitor is very useful for web applications that use the store as
     a in-memory cache. The StoreJanitor helps in avoiding OutOfMemory exceptions.</p>
    </section>
    <section name="Configuration">
     <p>The Store Janitor can be configured with a few options:
     </p>
       <ul>
         <li><em>freememory</em>: How many bytes shall be always free in
the JVM (Default: 1mb)</li>
         <li><em>heapsize</em>: Maximum possible size of the JVM memory
consumption (Default: 64mb)</li>
         <li><em>cleanupthreadinterval</em>: How often (sec) shall run the
cleanup thread (Default: 10s)</li>
         <li><em>adaptivethreadinterval</em> (experimental): Enable adaptive
algorithm to determine thread interval
             (Default: false) When true, <code>cleanupthreadinterval</code> defines
the maximum cleanup interval.
             Cleanup interval then is determined based on the memory fill rate: the faster
memory is filled in,
             and the less free memory is left, the shorter is the cleanup time.</li>
         <li><em>threadpriority</em>: priority of the thread (1-10). (Default:
10)</li>
         <li><em>percent_to_free</em>: What fraction of the store to free
when memory is low (1-100). (Default: 10%)</li>
         <li><em>invokegc</em>: Invoke the gc on low memory first (true|false;
default: false)</li>
        </ul>
    <p>The right configuration is very important, because wrong settings can 
      cause a high system load. Let's have a look at a sample configuration.</p>
    <subsection name="Example configuration">
      <ul><li>Tomcat settings in tomcat.sh or tomcat.bat:</li></ul>
      <source><![CDATA[
  %_RUNJAVA% %TOMCAT_OPTS% -Dtomcat.home="%TOMCAT_HOME%" \
    -Xmx200000000 org.apache.tomcat.startup.Tomcat %2 %3 %4 %5 %6 %7 %8 %9
      ]]></source> 
      <ul><li>StoreJanitor settings:</li></ul>
      <p>The freememory and heapsize paramter always depends on the Xmx 
      parameter.</p>
      <source><![CDATA[
    <!--+
        | Store Janitor: the store garbage collector and memory usage controller.
        |
        | Be careful with the heapsize and freememory parameters. Wrong values
        | can cause high cpu usage. Example configuration:
        | Jvm settings:
        |    -Xmx200000000
        | store-janitor settings:
        |    <parameter name="freememory" value="5000000"/>
        |    <parameter name="heapsize" value="196000000"/>
        |
        | It is recommended to have heapsize equal to -Xmx, especially on Sun's
        | JVM which are unable to shrink its heap once it grows above minimum.
        | Freememory should be greater than amount of memory necessary for normal
        | application operation.
        | BUT: The heap size of the memory of the JVM is a little bit less than
        |      the value you specify for -Xmx, so you have to set the heapsize
        |      for the store janitor to a value which is lower (2% less seems
        |      to be a working value).
        +-->
    <store-janitor logger="core.store.janitor">
       <!-- How much free memory shall be available in the jvm -->
       <parameter name="freememory" value="2048000"/>
       <!-- Indicates the limit of the jvm memory consumption. The default max
            heapsize for Sun's JVM is (almost) 64Mb -->
       <parameter name="heapsize" value="66600000"/>
       <!-- How often shall the cleanup thread check memory -->
       <parameter name="cleanupthreadinterval" value="10"/>
       <!-- Experimental adaptive algorithm for cleanup interval
       <parameter name="adaptivethreadinterval" value="true"/>
       -->
       <!-- Indicates the thread priority of the cleanup thread -->
       <parameter name="threadpriority" value="5"/>
       <!-- How much percent of the elements of each registered Store
            shall be removed when low on memory. Default 10% -->
       <parameter name="percent_to_free" value="10"/>
       <!-- Invoke the garbage collector when low memory is reached -->
       <parameter name="invokegc" value="false"/>
    </store-janitor>
      ]]></source>
      <p>It is recommended to have <code>heapsize</code> equal to -Xmx,
especially
      on Sun's JVM which are unable to shrink its heap once it grows above minimum. 
      <code>freememory</code> should be greater than amount of memory necessary
for normal 
      application operation. But the heap size of the memory of the JVM is a little bit less
than
     the value you specify for -Xmx, so you have to set the heapsize
     for the store janitor to a value which is lower (2% less seems
      to be a working value)
      </p>
      <p> The <code>cleanupthreadinterval</code> defines the interval of
the background 
      thread which checks memory in seconds. Also this paramter should configured wisely.

      A to short interval can cause also a high system load. The 
      <code>threadpriority</code> defines the priority of the background thread.

      1 is lowest level and 10 the highest.</p>
      <p>
      The <code>percent_to_free</code> parameter describes, how much percent of
the 
      elements of each registered Store shall be removed when low on memory.
      </p>
    </subsection>
    </section>
    </body>
  </document>
  
  
  
  
  1.1                  avalon-excalibur/site/xdocs/store-index.xml
  
  Index: store-index.xml
  ===================================================================
  <?xml version="1.0"?>
  <document>
      <properties>
          <author email="dev@avalon.apache.org">Avalon Documentation Team</author>
          <title>Excalibur Store - Overview</title>
      </properties>
  
      <body>
  
      <section name="Introduction">
        <p>
            The Store Package provides a way to store named objects without 
            taking care about where and how they are stored.
            You can choose between different implementations that store
            the data e.g. in memory or on the file system etc.
        </p>
        <p>  
          This project provides some general components for dealing with stores.
          It contains different store implementations, like the
          <a href="store-mru.html">MRU Store</a> and a montoring component,
          the <a href="store-janitor.html">Store Janitor</a>.
        </p>
        <p>
          These components have been donated by the <a href="http://cocoon.apache.org">Apache
Cocoon project</a>.
        </p>
      </section>
    </body>
  </document>
  
  
  
  
  1.1                  avalon-excalibur/site/xdocs/store-mru.xml
  
  Index: store-mru.xml
  ===================================================================
  <?xml version="1.0" encoding="UTF-8"?>
  <document>
      <properties>
          <author email="dev@avalon.apache.org">Avalon Documentation Team</author>
          <title>MRUMemoryStore and Swapping</title>
      </properties>
  
      <body>
    <section name="Goal">
      <p>This document explains how the MRUMemoryStore and Swapping executes.</p>
    </section>
    <section name="Overview">
      <p>The MRUMemoryStore was developed to provide a standard algorithm to
         store data in memory. For web-based applications the MRU (Most Recently Used) algorithm
         is very suitable, because the object most frequently accessed is always on "top".
      </p>
      <p>If configured, the objects are also swapped to the filesystem if the
      MRUMemoryStore reached his configured max. object limit.
      </p>
    </section>
    <section name="Implementation">
      <subsection name="MRUMemoryStore">
      <p> 
      The heart of the MRUMemoryStore ist combination of a LinkedList and a HashMap:
      </p>
      <p>
      The LinkedList provides the queue mechanism, and the entries in the LinkedList 
      contain the key to the data in the HashMap. When caching a new entry in to the list,
      the entry is inserted to the front of the list.
      If the list is already full, the oldest data entry is removed from the Cache.
      When requesting a entry, the store returns the object by key and inserts the requested
key 
      on the top of the Cache.
         
      This implementation keeps the most recent used objects in the store and provides the
best
      use of the machines memory.
      </p>
      </subsection>
      <subsection name="Swapping">
      <p>
      When the MRUMemoryStore is full or the JVM is at the heap size limit the 
      objects in the MRUMemoryStore are swapped to the Filesystem. The default 
      directory is "cache-dir" in the work-directory.
      </p>
      <p>
      NOTE: The keys are Strings at the moment. Therefor the
      filenames of the swapped objects can be very long. Especially Windows OS
      flavours have problems with long filenames. Use the JispFilesystemStore to
      get rid of it.
      </p>
      </subsection>
    </section>
    <section name="Configuration of the MRUMemoryStore in the cocoon.xconf">
    <source><![CDATA[
  <store class="org.apache.excalibur.store.impl.MRUMemoryStore"
         logger="core.store">
    <parameter name="maxobjects" value="100"/>
    <parameter name="use-persistent-cache" value="true"/>
  </store>]]>
    </source> 
    <p>Explanation of the paramters:</p>
      <ol> 
        <li><code>&lt;parameter name="maxobjects" value="100"/&gt;</code>:
        Indicates how many objects will be hold in MRUMemoryStore. When the number 
        of maxobjects has been reached, the last object in the MRUMemoryStore will be 
        thrown out.</li>
        <li><code>&lt;parameter name="use-persistent-cache" value="true"/&gt;</code>:
        If this switch is set on true, objects are swapped out to the filesystem,
        if the MRUMemoryStore has reached his maximum object limit, or the JVM 
        memory consumption is over the heap size limit. See StoreJanitor user 
        docs for more information.</li>
      </ol>
    </section>
    </body>
  </document>
  
  
  
  

---------------------------------------------------------------------
To unsubscribe, e-mail: cvs-unsubscribe@avalon.apache.org
For additional commands, e-mail: cvs-help@avalon.apache.org


Mime
View raw message