jackrabbit-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nigel Sim <nigel....@gmail.com>
Subject Re: Performance of a large number of small nodes
Date Sat, 15 Aug 2009 04:32:29 GMT
Hi Bertrand,

Thanks for your suggestion. Unfortunately, even in the simplest case of 100
nodes in the root node, the time taken to retrieve is too long. If I could
resolve this fundamental speed issue then I could apply your solution to
help me scale my system.

I think I just need to bite the bullet and admit my use case doesn't really
map on Jackrabbit :)

Thanks
Nigel

2009/8/14 Bertrand Delacretaz <bdelacretaz@apache.org>

> Hi,
>
> On Fri, Aug 14, 2009 at 6:34 AM, Nigel Sim<nigel.sim@gmail.com> wrote:
> > ...I am using Jackrabbit to store a mixture of scientific data, which
> includes
> > files and numerical data. The performance of files are fine, but the
> > numerical data needs to be extracted as datasets based on attributes such
> as
> > observation time, and this appears to be quite slow in comparison to a
> > native DB (obviously). I would really prefer to keep all this related
> data
> > in the same management system, so is there a way to improve the ingestion
> > and retrieval of many small nodes?...
>
> Could you take advantage of paths to express the observation time, and
> use that for "queries"?
>
> Storing data under paths like /data/2009/12/24/23/02/58 would allow
> you to find nodes that belong to a specific day, or hour, by
> navigating paths, which might be much more efficient than queries.
>
> > ...My second question, is there an efficient way to query for the latest
> > observation? I would assume querying for the node type, sorting, and just
> > retrieving the first result?...
>
> Paths would also help here, and you could use observation to keep
> track of the path that corresponds to the most recent data, if needed.
>
> -Bertrand
>



-- 
JCU eResearch Centre
School Of Business (IT)
James Cook University

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message