jackrabbit-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Seidel. Robert" <Robert.Sei...@aeb.de>
Subject AW: Is Jackrabbit suitable for storing lots of large files
Date Mon, 11 Feb 2013 13:28:52 GMT
Hi,

storing is not the problem, cause this is all done by streaming. But you can encounter problems
if you want to index such data, because Lucene holds all tokens for a file in memory (no streaming
here).
The default configuration stores 10K tokens max. per property (see maxFieldLength in http://wiki.apache.org/jackrabbit/Search).
But this can be real frustrating if the 10001. token is searched - it is also not very transparent
for the user.
If you increase this value, you need more memory.

Imho you have to decide to index all tokens (with enough memory) or nothing for this data.

Regards, Robert

-----Urspr√ľngliche Nachricht-----
Von: Bertrand Delacretaz [mailto:bdelacretaz@apache.org]
Gesendet: Montag, 11. Februar 2013 13:59
An: users@jackrabbit.apache.org
Betreff: Re: Is Jackrabbit suitable for storing lots of large files

Hi,

On Mon, Feb 11, 2013 at 1:49 PM, Christoph Läubrich <laeubi@googlemail.com> wrote:
> I read the performance doc here
> http://wiki.apache.org/jackrabbit/Performance but did not find an answer:
> Is Jackrabbit suitable for storing lots of files (around 100GB) with
> each file around 2-200MB?

As usual with performance you'll need to do your own tests, but that shouldn't be a problem
if you use the datastore [1] to store the binary content.

-Bertrand

[1] http://wiki.apache.org/jackrabbit/DataStore
________________________________

Treffen Sie AEB vom 19.-21. Februar 2013 auf der LogiMAT in Stuttgart. Halle 5, Stand 261.
Vereinbaren Sie jetzt einen Termin und Sie erhalten eine Eintrittskarte.
www.aeb.de/logimat
Mime
View raw message