Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of danbsutton@gmail.com
 designates 209.85.216.48 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=G0fMnB+q9thi7R2XvVs/GJesw8XW5j0EzinxceCAFVnUHYSk77OpnB7QMEptkiRWxN
         yMTgPG8nDQTNp5xzELVruOSrJ9xByJwoi36Ttai+6F/2IVMGyXvtbj3sA4KBqAo7Obrd
         dSWzVVA4nJdnunY153r0qyCOKuiZTkF2fsFGM=
MIME-Version: 1.0
Date: Tue, 18 Jan 2011 12:13:05 +0000
Message-ID: <AANLkTi=5BtADgV=b2taxrRH9ZSz9R_Sd1exY2Jf2LAKj@mail.gmail.com>
Subject: Large .frq file
From: dan sutton <danbsutton@gmail.com>
To: java-user <java-user@lucene.apache.org>
Content-Type: text/plain; charset=ISO-8859-1

Hi,

We're trying to create a large index via solr for trends and notice
that we have a large '.frq' file after doing the following:


make all text fields index="true", stored="false",
omitTermFreqAndPositions="true" omitNorms="true" termPositions="false"
termOffsets="false" termVectors="false"

We are using a variation on org.apache.lucene.analysis.cjk and notice
that the .frq is about 4 time larger than, for example, the
WhiteSpaceTokenizer.


Considering that with omitTermFreqAndPositions="true" for the text
fields I'd have thought this should be : "If omitTf were true it would
be this sequence of VInts instead:"
(http://lucene.apache.org/java/2_9_1/fileformats.html#Frequencies)


Can anyone suggest how I can reduce the size of this file?


Many thanks,
Dan

Lucene Specification Version: 2.9.1
Solr Specification Version: 1.4.0.2010.09.10.17.10.36

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org