Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <003201cc1883$11e7d450$35b77cf0$@com>
References: <BANLkTi=z1UyE=fnUv_zk+NamMLB_CitS_A@mail.gmail.com>
	<BANLkTing7Yuksj3vYDtsvLDs3TaGsf09uw@mail.gmail.com>
	<003101cc1876$6dd9f570$498de050$@com>
	<BANLkTi=rn84NO5qA-A5zaXOJeQ4Mtw+oyw@mail.gmail.com>
	<003201cc1883$11e7d450$35b77cf0$@com>
Date: Sun, 22 May 2011 09:35:54 -0400
Message-ID: <BANLkTik7LcDJf0c10u4jd9Fi+FYqWEd08g@mail.gmail.com>
Subject: Re: How to create document objects in our case
From: Michael McCandless <lucene@mikemccandless.com>
To: java-user@lucene.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

You're welcome!

Mike

http://blog.mikemccandless.com

On Sun, May 22, 2011 at 9:20 AM, zhoucheng2008 <zhoucheng2008@gmail.com> wr=
ote:
> Great, thanks Mike.
>
> -----Original Message-----
> From: Michael McCandless [mailto:lucene@mikemccandless.com]
> Sent: Sunday, May 22, 2011 8:09 PM
> To: java-user@lucene.apache.org
> Subject: Re: How to create document objects in our case
>
> Norms is how Lucene records what the apriori boost is for each
> docXfield. =A0This boost is the product of per-field boost, per-doc
> boost (both of which your app would set when it creates the doc), as
> well as the "length normalization" Lucene's default similarity applies
> (shorter docs have higher boost).
>
> It's a quantized float, encoded as 8 bits.
>
> If your fields tend not to vary in length much, and you don't use
> boosting yourself, and you are worried about RAM, you should omit the
> norms. =A0(Field has a method to omit norms).
>
> Not sure when nested docs will be available but I hope soon... even
> so, once it's committed, it won't be released until the next release
> (though you can use it on the trunk/3.x tip as well, once it's
> committed).
>
> Mike
>
> http://blog.mikemccandless.com
>
> On Sun, May 22, 2011 at 7:49 AM, zhoucheng2008 <zhoucheng2008@gmail.com>
> wrote:
>> Mike, thanks for reply.
>>
>> Can you please elaborate a little bit more on " If you don't need norms
>> (don't boost, lengths don't vary much or you
>> don't care to have field length impact scoring) you can omit norms"?
>>
>> When do you expect the handling of nested document will be applicable?
>>
>> Cheng
>>
>>
>> -----Original Message-----
>> From: Michael McCandless [mailto:lucene@mikemccandless.com]
>> Sent: Sunday, May 22, 2011 6:58 PM
>> To: java-user@lucene.apache.org
>> Subject: Re: How to create document objects in our case
>>
>> 30 fields is fine, but if they are all indexed you should watch out
>> for memory usage. =A0Ie, norms require 1 byte per doc per indexed field.
>> =A0If you don't need norms (don't boost, lengths don't vary much or you
>> don't care to have field length impact scoring) you can omit norms.
>>
>> The relationship b/w Group and Subgroup is not something Lucene can do
>> today -- all docs are "independent" in a Lucene index.
>>
>> So.. you can either "denormalize", meaning duplicate all Group fields
>> onto each subgroup, or you can check out
>> https://issues.apache.org/jira/browse/LUCENE-2454 which I think adds
>> exactly what you need. =A0It's not yet committed, but we are finally
>> starting to make progress towards that, eg
>> https://issues.apache.org/jira/browse/LUCENE-3112
>>
>> Mike
>>
>> http://blog.mikemccandless.com
>>
>> On Fri, May 20, 2011 at 8:27 PM, Cheng Zhou <zhoucheng2008@gmail.com>
> wrote:
>>> Hi,
>>>
>>> I have a large number of XML files to be indexed by Lucene. All the fil=
es
>>> share similar structure as below:
>>>
>>> <Group id=3D"abc" member=3D"cde" blah blah ....>
>>> =A0 <Subgroup id=3D"abc1" member =3D"fgh" blah blah ...>
>>> =A0 <Subgroup id=3D"abc2" member =3D"fgh" blah blah ...>
>>> =A0 <Subgroup id=3D"abc3" member =3D"fgh" blah blah ...>
>>> =A0 ......
>>> </Group>
>>>
>>> Things to be noted are:
>>>
>>> The root element of Group has 30 or so attributes, and it usually has
> over
>>> 2000 Subgroup elements, which in turn also have more than 20 attributes=
.
>>>
>>> I want to create one Document object which holds the contents of the
> Group
>>> element, and one Document object which holds all the Subgroup elements.
>>>
>>> Here are my challenges however:
>>>
>>> 1. How many fields are advised for a Document to be indexed by Lucene?
>> Will
>>> over 30 fields (for the Group element) be too many?
>>>
>>> 2. How to create a Document object and fields for holding all the
> Subgroup
>>> elements? Is this a good way to think of?
>>>
>>> 3. How can I link the Document object of the Group element to the
> Document
>>> object of all the Subgroup elements?
>>>
>>> Please note that I intend to use such two Document objects to achieve t=
he
>>> group while I don't know whether it is a good solution or not. I am ope=
n
>> to
>>> using more than two Documents to do the job, but I don't know how to
>> connect
>>> all the objects in Lucene.
>>>
>>> Many thanks!
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org