Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of erickerickson@gmail.com
 designates 209.85.213.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAEY5pxWMfiC_1uyPi_65acNjTVUsu23F1-NTwc0geE50YprE2A@mail.gmail.com>
References: 
 <CALGBXaMwj5H3zNCVgBSaW==58=bSv=M_WeHLZQxqFJWRpG24wA@mail.gmail.com>
	<CAEY5pxWb=gp_f+_LdwCHa8oBBm+Fes0ryDhHT1So2ehW6omghA@mail.gmail.com>
	<CALGBXaOgmcGTsyw4qGdpkqE_3HphCc8GPTpbwZig-+4LXo7EAQ@mail.gmail.com>
	<CAEY5pxWMfiC_1uyPi_65acNjTVUsu23F1-NTwc0geE50YprE2A@mail.gmail.com>
Date: Mon, 14 May 2012 07:30:43 -0400
Message-ID: 
 <CAN4YXveTnxS32ZCOnz5jinsxz1Pnj5RqLm45UHWTkoZ=S8khxQ@mail.gmail.com>
Subject: Re: Getting the frequencies by corresponding order of documents were
 indexed
From: Erick Erickson <erickerickson@gmail.com>
To: java-user@lucene.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

In general you can't rely on anything like this. I admit the merge
stuff isn't my area of expertise, but when segments are merged,
there's no guarantee that they're merged in order. In general
the internal Lucene doc ID should be treated as predictable only
for closed segments.

Your solution of using your own unique ID is much better.

Best
Erick

On Fri, May 11, 2012 at 8:50 AM, Ian Lea <ian.lea@gmail.com> wrote:
> What version of lucene are you using? =A0If not the latest, try that.
> If you really think there is a lucene bug post a small self-contained
> test case that demonstrates the problem.
>
>
> --
> Ian.
>
>
> On Fri, May 11, 2012 at 12:35 PM, Kasun Perera <kasunp@opensource.lk> wro=
te:
>> On Fri, May 11, 2012 at 4:52 PM, Ian Lea <ian.lea@gmail.com> wrote:
>>
>>> Can't spot anything obviously wrong in your code and what you are
>>> trying to do should work. =A0Are you positive that what you think is th=
e
>>> second doc is really being added second? =A0You only show one doc being
>>> added. =A0Are there already 7 docs in the index before you start?
>>>
>>>
>>>
>> Hi Ian
>>
>> yes I'm sure 2nd doc is added second and I use debugger several times to
>> confirm it. If I index 10 documents, I'm getting 10 termFrequncy vectors
>> but their positions are changed. I gave doc #2 as example. =A0#5th
>> termfrequncy vector is correspond to doc and so on.
>>
>> I figured out to overcome this but it may be not efficient. I stored
>> another field at indexing time, base on the content inside new field i'm
>> able to map the doc with its termfrequncy vector. Is there any other
>> efficient way? This may be a bug in Lucene?
>>
>> Thanks
>>
>>> --
>>> Ian.
>>>
>>>
>>> On Fri, May 11, 2012 at 8:58 AM, Kasun Perera <kasunp@opensource.lk>
>>> wrote:
>>> > I have collection of documents (say 10 documents)and i'm indexing the=
m
>>> this
>>> > way, by storing the term vector
>>> >
>>> > StringReader strRdElt =3D new StringReader(content);
>>> >
>>> >
>>> > =A0 =A0Document doc =3D new Document();
>>> >
>>> > =A0 =A0String docname=3DdocNames[docNo];
>>> >
>>> > =A0 =A0doc.add(new Field("doccontent", strRdElt, Field.TermVector.YES=
));
>>> >
>>> > =A0 =A0IndexWriter iW;
>>> > =A0 =A0try {
>>> >
>>> > =A0 =A0 =A0 =A0NIOFSDirectory dir =3D new NIOFSDirectory(new File(pat=
hToIndex)) ;
>>> >
>>> > =A0 =A0 =A0 =A0iW =3D new IndexWriter(dir, new IndexWriterConfig(Vers=
ion.LUCENE_35,
>>> >
>>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0new StandardAnalyzer(Version.LUCENE_35=
)));
>>> >
>>> > =A0 =A0 =A0 =A0iW.addDocument(doc);
>>> > =A0 =A0 =A0 =A0iW.close();
>>> >
>>> > =A0 =A0}
>>> >
>>> > After Index all the documents, i'm getting the term-frequencies of ea=
ch
>>> > document this way
>>> >
>>> >
>>> > IndexReader re =3D IndexReader.open(FSDirectory.open(new
>>> > File(pathToIndex)), true) ;
>>> > TermFreqVector termsFreq[];
>>> > for(int i=3D0;i<noOfDocs;i++){
>>> > =A0 =A0 =A0 =A0termsFreq[i] =3D re.getTermFreqVector(i, "doccontent")=
;
>>> >
>>> > =A0 =A0 =A0}
>>> >
>>> > my problem is i'm not getting the termfreqncy vector correspondingly.=
 Say
>>> > for 2nd document that I have indexed i'm getting it's corresponding
>>> > termfrequncies and terms at "termsFreq[9]"
>>> >
>>> > What is the reason for that?, how can I get the corresponding
>>> > termfrequncies by the order that I have indexed the documents?
>>> >
>>> >
>>> > --
>>> > Regards
>>> >
>>> > Kasun Perera
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>>
>> --
>> Regards
>>
>> Kasun Perera
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org