jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jérôme BENOIS <ben...@argia-engineering.fr>
Subject Re: Newbies question
Date Wed, 18 Jan 2006 14:32:19 GMT
Hi,

	Thanks for your response.

	I replaced my PM by DerbyPM and an error occured when i run my test :
	INFO main fr.openmodel.cms.imports.process.Impl_ImportMgtProcess -
inserted count = 18500/50000
ERROR main
org.apache.jackrabbit.core.state.db.SimpleDbPersistenceManager - failed
to write property state: f08c7241-032f-4515-a0cd-c8592649b45e
ERROR 22001: A truncation error was encountered trying to shrink BLOB
'XX-RESOLVE-XX' to length 1048576.
        at
org.apache.derby.iapi.error.StandardException.newException(Unknown
Source)
        at
org.apache.derby.iapi.types.SQLBinary.checkHostVariable(Unknown Source)
        at
org.apache.derby.exe.ac597e80e5x0108xdd95x2506x0000001861b8c.e3(Unknown
Source)
        at
org.apache.derby.impl.services.reflect.DirectCall.invoke(Unknown Source)
        at
org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.doProjection(Unknown Source)
        at
org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown Source)
        at
org.apache.derby.impl.sql.execute.NormalizeResultSet.getNextRowCore(Unknown Source)
        at
org.apache.derby.impl.sql.execute.DMLWriteResultSet.getNextRowCore(Unknown Source)
        at
org.apache.derby.impl.sql.execute.UpdateResultSet.collectAffectedRows(Unknown Source)
        at
org.apache.derby.impl.sql.execute.UpdateResultSet.open(Unknown Source)
        at
org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown
Source)
        at
org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
Source)
        at
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
        at
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown
Source)
        at
org.apache.jackrabbit.core.state.db.SimpleDbPersistenceManager.store(SimpleDbPersistenceManager.java:779)
        at
org.apache.jackrabbit.core.state.AbstractPersistenceManager.store(AbstractPersistenceManager.java:87)
        at
org.apache.jackrabbit.core.state.db.SimpleDbPersistenceManager.store(SimpleDbPersistenceManager.java:446)
        at org.apache.jackrabbit.core.state.SharedItemStateManager
$Update.end(SharedItemStateManager.java:562)
        at
org.apache.jackrabbit.core.state.SharedItemStateManager.update(SharedItemStateManager.java:680)
        at
org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:322)
        at
org.apache.jackrabbit.core.state.XAItemStateManager.update(XAItemStateManager.java:322)
        at
org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:298)
        at
org.apache.jackrabbit.core.state.SessionItemStateManager.update(SessionItemStateManager.java:260)
        at org.apache.jackrabbit.core.ItemImpl.save(ItemImpl.java:1190)
        at
org.apache.jackrabbit.core.SessionImpl.save(SessionImpl.java:758)
        at
fr.openmodel.cms.contentunit.entities.Impl_ContentUnitEntityMgr.saveCUs(Impl_ContentUnitEntityMgr.java:709)

	And i tested with SimpleDBPM and postgres8 database. I created 50000
nodes with 3 properties in 25 minutes and when i launch this simple
query ... it's very long : 

DEBUG main org.apache.jackrabbit.core.query.lucene.Recovery - RedoLog is
empty, no recovery needed.
INFO main org.apache.jackrabbit.core.query.lucene.SearchIndex - Index
initialized: /opt/jackrabbit/repotest/workspaces/default/index
DEBUG main org.apache.jackrabbit.core.query.lucene.QueryImpl - Executing
query: 
+ Root node
+ Select properties: *
  + PathQueryNode
    + LocationStepQueryNode:  NodeTest={} Descendants=false Index=NONE
    + LocationStepQueryNode:  NodeTest=* Descendants=true Index=NONE
      + AndQueryNode
        + NodeTypeQueryNode:
Prop={http://www.jcp.org/jcr/1.0}primaryType
Value={http://www.jcp.org/jcr/nt/1.0}unstructured
        + AndQueryNode
          + RelationQueryNode: Op: LIKE Prop={}email Type=STRING Value=%
jean%

INFO main org.apache.jackrabbit.core.query.lucene.DocNumberCache -
size=64/1024, #accesses=1001, #hits=937, #misses=64, cacheRatio=94%
--------> nb result 1563 in 25.0 seconds

And if i test when another query, it's very good :

DEBUG main org.apache.jackrabbit.core.query.lucene.QueryImpl - Executing
query: 
+ Root node
+ Select properties: *
  + PathQueryNode
    + LocationStepQueryNode:  NodeTest={} Descendants=false Index=NONE
    + LocationStepQueryNode:  NodeTest=* Descendants=true Index=NONE
      + AndQueryNode
        + NodeTypeQueryNode:
Prop={http://www.jcp.org/jcr/1.0}primaryType
Value={http://www.jcp.org/jcr/nt/1.0}unstructured
        + AndQueryNode
          + RelationQueryNode: Op: LIKE Prop={}email Type=STRING Value=%
wanadoo%

--------> nb result 2 in 0.0 seconds


the query time depend of number of results, all results are loadred in
memory ??

Regards,
Jérôme.


Le mercredi 18 janvier 2006 à 10:26 +0100, Stefan Guggisberg a écrit :
> is there a specific reason for using ObjectPersistenceManager?
> 
> if you use jr's 'default' persistence manager (DerbyPersistenceManager)
> you should be experiencing a much better performance.
> 
> there's a sample configuration in svn:
> jackrabbit/src/main/config/repository.xml
> 
> cheers
> stefan
> 
> On 1/18/06, Jérôme BENOIS <benois@argia-engineering.fr> wrote:
> > Hi All,
> >
> >         Thanks for your response.
> >
> >         I carried out some tests with 50000 nodes (small nodes with 3
> > properties), i create this in 25 minutes and my store weigh 2.5Go.
> >
> >         And when i execute a simple query is still long : ~5 minutes.
> >
> >         I applied your suggestion about document order here :
> > <?xml version="1.0" encoding="ISO-8859-1"?>
> > <Repository>
> >     <FileSystem
> > class="org.apache.jackrabbit.core.fs.local.LocalFileSystem">
> >         <param name="path" value="${rep.home}/repository"/>
> >                 <param name="persistent" value="true"/>
> >     </FileSystem>
> >     <Security appName="Jackrabbit">
> >         <AccessManager
> > class="org.apache.jackrabbit.core.security.SimpleAccessManager"/>
> >         <LoginModule
> > class="org.apache.jackrabbit.core.security.SimpleLoginModule">
> >            <param name="anonymousId" value="anonymous"/>
> >         </LoginModule>
> >     </Security>
> >     <Workspaces rootPath="${rep.home}/workspaces"
> > defaultWorkspace="default"/>
> >             <Workspace name="${wsp.name}">
> >                         <FileSystem
> > class="org.apache.jackrabbit.core.fs.local.LocalFileSystem">
> >                         <param name="path" value="${wsp.home}"/>
> >                 </FileSystem>
> >                     <PersistenceManager
> > class="org.apache.jackrabbit.core.state.obj.ObjectPersistenceManager"/>
> >             <SearchIndex
> > class="org.apache.jackrabbit.core.query.lucene.SearchIndex">
> >                     <param name="path"
> > value="${wsp.home}/index"/>
> >                     <param name="autoRepair" value="false"/>
> >                     <param name="respectDocumentOrder" value="false"/>
> >                 </SearchIndex>
> >             </Workspace>
> >             <Versioning rootPath="${rep.home}/version">
> >                         <FileSystem
> > class="org.apache.jackrabbit.core.fs.local.LocalFileSystem">
> >                                 <param name="path" value="${rep.home}/version"/>
> >                         </FileSystem>
> >                 <PersistenceManager
> > class="org.apache.jackrabbit.core.state.obj.ObjectPersistenceManager"/>
> >                 </Versioning>
> > </Repository>
> >
> >         And i use suversion version, i activated debug mode when i launched my
> > simple query :
> >
> > DEBUG main org.apache.jackrabbit.core.query.lucene.QueryImpl - Executing
> > query:
> > + Root node
> > + Select properties: *
> >   + PathQueryNode
> >     + LocationStepQueryNode:  NodeTest={} Descendants=false Index=NONE
> >     + LocationStepQueryNode:  NodeTest=* Descendants=true Index=NONE
> >       + AndQueryNode
> >         + NodeTypeQueryNode:
> > Prop={http://www.jcp.org/jcr/1.0}primaryType
> > Value={http://www.jcp.org/jcr/nt/1.0}unstructured
> >         + AndQueryNode
> >           + RelationQueryNode: Op: LIKE Prop={}email Type=STRING Value=a
> > %
> >
> > DEBUG main org.apache.jackrabbit.core.query.lucene.AbstractIndex -
> > merging segments _0 (1 docs) into _1 (1 docs)
> > DEBUG main org.apache.jackrabbit.core.query.lucene.AbstractIndex -
> > closing IndexWriter.
> > INFO main org.apache.jackrabbit.core.query.lucene.DocNumberCache -
> > size=60/1024, #accesses=1001, #hits=941, #misses=60, cacheRatio=95%
> > DEBUG Timer-2 org.apache.jackrabbit.core.query.lucene.MultiIndex -
> > Flushing index after being idle for 3615 ms.
> > DEBUG Timer-2 org.apache.jackrabbit.core.query.lucene.IndexMerger -
> > index added: name=_ii, numDocs=1
> > DEBUG Timer-2 org.apache.jackrabbit.core.query.lucene.MultiIndex -
> > Committed in-memory index in 2ms.
> > DEBUG IndexMerger org.apache.jackrabbit.core.query.lucene.AbstractIndex
> > - merging segments _0 (8416 docs) into _1 (8416 docs)
> > INFO IndexMerger org.apache.jackrabbit.core.query.lucene.IndexMerger -
> > merged 8416 documents in 4206 ms into _ih.
> > DEBUG IndexMerger org.apache.jackrabbit.core.query.lucene.IndexMerger -
> > replace indexes
> > DEBUG IndexMerger org.apache.jackrabbit.core.query.lucene.AbstractIndex
> > - closing IndexWriter.
> > DEBUG IndexMerger org.apache.jackrabbit.core.query.lucene.IndexMerger -
> > index added: name=_ih, numDocs=8416
> > DEBUG Timer-2 org.apache.jackrabbit.core.query.lucene.MultiIndex -
> > Flushing index after being idle for 3339 ms.
> > DEBUG main
> > org.apache.jackrabbit.core.query.lucene.DocOrderNodeIteratorImpl - 1537
> > node(s) ordered in 331882 ms
> > INFO main fr.openmodel.cms.imports.process.TestImportMgtProcess -
> > testInsert 3 contentUnits.size()=1537
> > INFO Thread-4
> > org.apache.jackrabbit.core.observation.ObservationManagerFactory -
> > Notification of EventListeners stopped.
> > DEBUG Thread-4 org.apache.jackrabbit.core.query.lucene.IndexMerger -
> > dispose IndexMerger
> > INFO IndexMerger org.apache.jackrabbit.core.query.lucene.IndexMerger -
> > IndexMerger terminated
> > DEBUG Thread-4 org.apache.jackrabbit.core.query.lucene.IndexMerger -
> > quit sent
> > DEBUG Thread-4 org.apache.jackrabbit.core.query.lucene.IndexMerger -
> > IndexMerger thread stopped
> > DEBUG Thread-4 org.apache.jackrabbit.core.query.lucene.IndexMerger -
> > merge queue size: 0
> > INFO Thread-4 org.apache.jackrabbit.core.query.lucene.SearchIndex -
> > Index closed: /opt/jackrabbit/repotest/workspaces/default/index
> > DEBUG Thread-4
> > org.apache.jackrabbit.core.observation.ObservationManagerImpl - removing
> > EventListener: org.apache.jackrabbit.core.lock.LockManagerImpl@149a794
> > DEBUG Thread-4
> > org.apache.jackrabbit.core.observation.ObservationManagerImpl - removing
> > EventListener: org.apache.jackrabbit.core.SearchManager@64023c
> > DEBUG Thread-4
> > org.apache.jackrabbit.core.observation.ObservationManagerImpl - removing
> > EventListener: org.apache.jackrabbit.core.RepositoryImpl@190a0d6
> >
> >
> > Can you help me please ?
> >
> > Thanks for your help,
> >
> > Best Regards,
> > Jérôme.
> >
> >
> > Le jeudi 12 janvier 2006 à 12:08 +0100, Marcel Reutegger a écrit :
> > > try disabling document order on query results:
> > >
> > > <SearchIndex
> > > class="org.apache.jackrabbit.core.query.lucene.SearchIndex">
> > >      [...]
> > >      <param name="respectDocumentOrder" value="false"/>
> > >
> > > </SearchIndex>
> > >
> > >
> > > information about document order is not stored in the index, that mean
> > > if you have a large result set, the query handler has to load nodes from
> > > storage, which is expensive compared to index lookups.
> > >
> > > regards
> > >   marcel
> >
> >
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v1.4.2 (GNU/Linux)
> >
> > iD8DBQBDzYJACDKlmlhpwnERAiXUAJ4rYhsCx3S6bYnTfX7fTDHrow76FQCfZsCZ
> > W38VEVjp89GuF6cgWQICP/I=
> > =hvyf
> > -----END PGP SIGNATURE-----
> >
> >
> >
> 

Mime
View raw message