lucene-solr-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Solr Wiki] Trivial Update of "LotsOfCores" by ErickErickson
Date Thu, 03 Jan 2013 20:52:45 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.

The "LotsOfCores" page has been changed by ErickErickson:
http://wiki.apache.org/solr/LotsOfCores?action=diff&rev1=19&rev2=20

Comment:
Changed "swabbable" to "transient"

  
  There are two new attributes of a <core> tag (defaults in bold) and one new attribute
for <cores>
   * <cores> has a two new attributes:
-   * swappableCacheSize=[NNN]. If this limit is crossed, old cores marked 'swappable="true"'
are removed to make room on an LRU basis. 
+   * transientCacheSize=[NNN]. If this limit is crossed, old cores marked 'transient="true"'
are removed to make room on an LRU basis. 
-    * If this is absent, the default is Integer.MAX_VALUE, an unbounded cache. ''Only'' cores
with "swappable=true" are put in this cache, so specifying this attribute without having any
cores marked as "swappable" has no effect, just wastes a LinkedHashMap<String, SolrCore>
of the specified size which will never be used.
+    * If this is absent, the default is Integer.MAX_VALUE, an unbounded cache. ''Only'' cores
with "transient=true" are put in this cache, so specifying this attribute without having any
cores marked as "transient" has no effect, just wastes a LinkedHashMap<String, SolrCore>
of the specified size which will never be used.
-    * Having this size be less than the number of cores marked 'swappable="true"' AND 'loadOnStartup="true"'
''should'' work, but it's wasteful since a bunch of cores will be loaded on startup then immediately
unloaded after the cache fills up.
+    * Having this size be less than the number of cores marked 'transient="true"' AND 'loadOnStartup="true"'
''should'' work, but it's wasteful since a bunch of cores will be loaded on startup then immediately
unloaded after the cache fills up.
-    * NOTE: when solr.xml is read, the information for all swappable cores (i.e. the CoreDescriptor)
is put in a separate list. So having more swappable cores than the size of the cache will
be handled correctly. The "list of cores" is unbounded.
+    * NOTE: when solr.xml is read, the information for all transient cores (i.e. the CoreDescriptor)
is put in a separate list. So having more transient cores than the size of the cache will
be handled correctly. The "list of cores" is unbounded.
-   * coreDescriptorProvider=<class derived from CoreDescriptorProvider>. (more later,
this isn't done yet). This will be a pluggable component that provides a CoreDescriptor on
demand. Solr core handling will ask this component (if present) for a CoreDescriptor for any
core name it doesn't recognize. If the component returns a CoreDescriptor, it will be added
to the appropriate internal list based on the values of the loadOnStartup and swappable member
variables. The code will ''probably'' load the core no matter what the loadOnStartup parameter
specifies on the theory that there is an immediate request to be satisfied. TBD. 
+   * coreDescriptorProvider=<class derived from CoreDescriptorProvider>. (more later,
this isn't done yet). This will be a pluggable component that provides a CoreDescriptor on
demand. Solr core handling will ask this component (if present) for a CoreDescriptor for any
core name it doesn't recognize. If the component returns a CoreDescriptor, it will be added
to the appropriate internal list based on the values of the loadOnStartup and transient member
variables. The code will ''probably'' load the core no matter what the loadOnStartup parameter
specifies on the theory that there is an immediate request to be satisfied. TBD. 
   * <core> has two new attributes:
    * loadOnStartup=["'''true'''"|"false"]. Whether the core should be completely loaded upon
startup.
-   * swappable=["true"|"'''false'''"]. Whether the core is allowed to be swapped out or not.
+   * transient=["true"|"'''false'''"]. Whether the core is allowed to be swapped out or not.
  
  So the idea is that there's really no reason to tie in "lazy loading" with whether the core
can be swapped out or not, so by splitting up the two options we give the user control over
how these are handled. Use cases below:
-  * loadOnStartup=true swappable=false: Current case. Spend all the time necessary to fully
load the cores on startup.
+  * loadOnStartup=true transient=false: Current case. Spend all the time necessary to fully
load the cores on startup.
-  * loadOnStartup=true swappable=true:  There are some cores you want loaded when the server
first starts up, but that you'll allow to be swapped out. It's wasteful to specify more cores
like this than your swappableCacheSize value.
-  * loadOnStartup=false swappable=false: Probably the least useful combination, but it naturally
falls out of the code. You'd specify this combination if, for some reason, starting Solr up
quickly was more important than the inconvenience of having to wait randomly for cores to
be loaded when a request was made.
+  * loadOnStartup=true transient=true:  There are some cores you want loaded when the server
first starts up, but that you'll allow to be swapped out. It's wasteful to specify more cores
like this than your transientCacheSize value.
+  * loadOnStartup=false transient=false: Probably the least useful combination, but it naturally
falls out of the code. You'd specify this combination if, for some reason, starting Solr up
quickly was more important than the inconvenience of having to wait randomly for cores to
be loaded when a request was made.
-  * loadOnStartup=false swappable=true: This is really the use-case. There are a large number
of cores in your system that are short-duration use. You want Solr to load them as necessary,
but unload them when the cache gets full on an LRU basis.
+  * loadOnStartup=false transient=true: This is really the use-case. There are a large number
of cores in your system that are short-duration use. You want Solr to load them as necessary,
but unload them when the cache gets full on an LRU basis.
  
  
  The following configuration applies to the patch given in [[https://issues.apache.org/jira/browse/SOLR-1293|SOLR-1293]].
@@ -66, +66 @@

  <?xml version='1.0' encoding='UTF-8'?>
  <solr persistent='true'>
    <cores adminPath="/admin/cores"
-           swappableCacheSize="4"
+           transientCacheSize="4"
            adminHandler="org.apache.solr.handler.admin.LotsOfCoresAdminHandler"
            shareSchema="true"
            shareConfig="true">
-     <core name="core0" instanceDir="/opt/solr" loadOnStartup="false" swappable="true"/>
+     <core name="core0" instanceDir="/opt/solr" loadOnStartup="false" transient="true"/>
    </cores>
  </solr>
  }}}

Mime
View raw message