lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phil Herold <Phil.Her...@d-wise.com>
Subject RE: FieldType.tokenized not the same after query
Date Fri, 17 Jan 2014 11:43:13 GMT
Uwe,

Thanks for the response. This is what I expected, and it's unfortunate (for us). Our software
puts an abstraction layer above Lucene (for historical reasons), and expects to be able to
pull the same "type" of term (here's the abstraction) from the index that it puts in. It doesn't
appear as if that is going to work now in Lucene 4 (of course it worked in Lucene 3.x). We'll
try to come up some other way of doing this.

--
Phil

________________________________________
From: Uwe Schindler [uwe@thetaphi.de]
Sent: Friday, January 17, 2014 3:19 AM
To: java-user@lucene.apache.org
Subject: RE: FieldType.tokenized not the same after query

Hi,

The indexing settings of FieldTypes are not available in the index. The FieldType information
is only used during indexing. IndexReader.document() only returns stored fields, nothing more.

This is one reason why Lucene 5.x (currently trunk), no longer shares the same "Document"
/ "Field" API for indexing and returning stored fields. In Lucene 5, the methods of IndexReader
return StoredDocument / StoredField instances, while IndexWriter takes IndexDocuments.

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Phil Herold [mailto:Phil.Herold@d-wise.com]
> Sent: Friday, January 17, 2014 2:37 AM
> To: java-user@lucene.apache.org
> Subject: FieldType.tokenized not the same after query
>
> The last line in the test program below fails. I'm trying to store a keyword, not
> tokenized, and get the same "type" of field back after query. But it doesn't
> work, it comes back as "tokenized". Is this a known problem, or am I missing
> something?
>
>
>
> Thanks.
>
>
>
> --
>
> Phil
>
>
>
> import org.apache.lucene.analysis.Analyzer;
>
> import org.apache.lucene.analysis.standard.StandardAnalyzer;
>
> import org.apache.lucene.document.Document;
>
> import org.apache.lucene.document.Field;
>
> import org.apache.lucene.document.Field.Store;
>
> import org.apache.lucene.document.FieldType;
>
> import org.apache.lucene.document.StringField;
>
> import org.apache.lucene.index.DirectoryReader;
>
> import org.apache.lucene.index.IndexReader;
>
> import org.apache.lucene.index.IndexWriter;
>
> import org.apache.lucene.index.IndexWriterConfig;
>
> import org.apache.lucene.index.IndexableField;
>
> import org.apache.lucene.search.IndexSearcher;
>
> import org.apache.lucene.search.Query;
>
> import org.apache.lucene.search.TopScoreDocCollector;
>
> import org.apache.lucene.store.Directory;
>
> import org.apache.lucene.store.RAMDirectory;
>
> import org.apache.lucene.util.QueryBuilder;
>
> import org.apache.lucene.util.Version;
>
> import org.junit.Assert;
>
> import org.junit.Test;
>
>
>
> public class TestField {
>
>
>
>    @Test
>
>    public void testIt() throws Exception {
>
>       Document doc = new Document();
>
>       boolean useStringField = false;
>
>       IndexableField field;
>
>       if (useStringField) {
>
>          field = new StringField("foo", "bar", Store.YES);
>
>       }
>
>       else {
>
>          FieldType fieldType = new FieldType();
>
>          fieldType.setStored(true);
>
>          fieldType.setTokenized(false);
>
>          fieldType.setIndexed(true);
>
>          field = new Field("foo", "bar", fieldType);
>
>       }
>
>       doc.add(field);
>
>
>
>       Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_46);
>
>       Directory directory = new RAMDirectory();
>
>
>
>       IndexWriterConfig iwc = new IndexWriterConfig(Version.LUCENE_46,
> analyzer);
>
>       iwc.setOpenMode(IndexWriterConfig.OpenMode.CREATE);
>
>       IndexWriter iwriter = new IndexWriter(directory, iwc);
>
>       iwriter.addDocument(doc);
>
>       iwriter.close();
>
>
>
>       IndexReader reader = DirectoryReader.open(directory);
>
>       IndexSearcher searcher = new IndexSearcher(reader);
>
>
>
>       Query q = new QueryBuilder(analyzer).createPhraseQuery("foo", "bar");
>
>       TopScoreDocCollector collector = TopScoreDocCollector.create(5, true);
>
>       searcher.search(q, collector);
>
>       Assert.assertTrue(collector.getTotalHits() == 1);
>
>       doc = searcher.doc(collector.topDocs().scoreDocs[0].doc);
>
>       Assert.assertTrue(doc != null);
>
>       field = doc.getField("foo");
>
>       Assert.assertTrue(field != null);
>
>       if (useStringField) {
>
>          Assert.assertTrue(field instanceof StringField);
>
>       }
>
>       else {
>
>          Assert.assertTrue(field instanceof Field);
>
>       }
>
>       Assert.assertTrue(!field.fieldType().tokenized());
>
>
>
>    }
>
>
>
> }


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message