Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: softfail (nike.apache.org: transitioning domain of
 rnagalin@in.ibm.com does not designate 59.145.155.8 as permitted sender)
To: java-user@lucene.apache.org
MIME-Version: 1.0
Subject: lucene 2.4.1 : document in index but not returned in search
Message-ID: 
 <OFBBCFE1F7.65E5D603-ON65257643.00538592-65257643.00563C2F@in.ibm.com>
From: Rathinapriya Nagalingam <rnagalin@in.ibm.com>
Date: Fri, 2 Oct 2009 21:21:42 +0530
Content-Type: multipart/alternative;
 boundary="=_alternative 00563C2D65257643_="

--=_alternative 00563C2D65257643_=
Content-Type: text/plain; charset="US-ASCII"

Hi,

I created an index of around 45000 documents. I search using Title and 
Abstract field. (Using lucene 2.4.1)
When I look in lukeall, some titles are available in index, but I dont get 
them when I search using title as keyword.
I have copied code snapshots below.

Recently we upgraded from lucene 2.0 to 2.4.1 and I am fairly new to 
lucene. Please let me know what could be the possible issue.

IndexWriter writer = new IndexWriter(directory, new StandardAnalyzer(),
                                                pCreate, 
IndexWriter.MaxFieldLength.UNLIMITED);

........................
SimpleAnalyzer cjkAnalyzer = new SimpleAnalyzer();
doc.add(new Field(LuceneDocument.TITLE_FIELD, pTitle, Field.Store.YES,
                                Field.Index.ANALYZED_NO_NORMS));

doc.add(new Field(LuceneDocument.ABSTRACT_FIELD, pDescription,
                                Field.Store.YES, Field.Index.
ANALYZED_NO_NORMS));
...............
if (localeStr.equals("zh_CN") || localeStr.equals("ko_KR")) { 
        writer.addDocument(doc, cjkAnalyzer);
} else {
        writer.addDocument(doc);
}


While searching the index, I use the analyser as below.

// construct the proper analyzer based on locale
                        if (pLocale.equals("zh_CN") || pLocale.equals(
"ko_KR")) { /* NOI18N */
                                analyzer = new SimpleAnalyzer();
                        } else {
                                analyzer = new StandardAnalyzer();
                        }
                        // analyze the keywords
                        ts = analyzer.tokenStream("abstract", 
//$NON-NLS-1$
                                        new StringReader(sb.toString()));
                        tokens = new ArrayList();
                        try {
                                while (true) {
                                        Token token = ts.next();
                                        if (token == null) {
                                                break;
                                        }
                                        tokens.add(token.termText());
                                }
                        } catch (IOException ex) {
                                Logger.logException(Logger.TYPE_ERR, this,
                                                "analyzeKeywords", ex); 
//$NON-NLS-1$ 
                        }

// Loop through the keywords
                                for (int i = 0; i < tokens.length; i++) {
                                        // Each keyword must be queried 
against the title and abstract.
                                        BooleanQuery bQuery = new 
BooleanQuery();
                                        TermQuery titleTerm = new 
TermQuery(
                                                        new Term("title", 
tokens[i])); //$NON-NLS-1$
                                        TermQuery abstractTerm = new 
TermQuery(new Term(
                                                        "abstract", 
tokens[i])); //$NON-NLS-1$
                                        if (keywordStatusListSize > i
                                                        && ((Boolean) 
keywordStatusList.get(i)).booleanValue()) {
                                                bQuery.add(titleTerm, 
BooleanClause.Occur.MUST);
                                                bQuery.add(abstractTerm, 
BooleanClause.Occur.MUST);
                                        } else {
                                                bQuery.add(titleTerm, 
BooleanClause.Occur.SHOULD);
                                                bQuery.add(abstractTerm, 
BooleanClause.Occur.SHOULD);
                                        }
                                        if (flag || ((Boolean) 
keywordStatusList.get(i)).booleanValue()) {
                                                keyQuery.add(bQuery, 
BooleanClause.Occur.MUST);
                                        } else {
                                                keyQuery.add(bQuery, 
BooleanClause.Occur.SHOULD);
                                        }
                                }

................................
CachingWrapperFilter cf;
Searcher searcher;
HitCollector collector;
................. some assignments...............
searcher.search(keyQuery, cf, collector);


Thanks & Regards,
Priya
PT-7A-012
Residency Road
Bangalore
India
Mob: 99011 22033

--=_alternative 00563C2D65257643_=--