lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nischal reddy <>
Subject Retrieving attributes of terms in lucene
Date Wed, 11 Sep 2013 13:24:55 GMT

I have written a custom Tokenizer which will split my input text into
tokens, i have overridden the incrementToken method and setting
chartermAttribute, offsetAttribute, typeAttribute (Please find the method

    final public boolean incrementToken() throws IOException {
        if(reader == null){
            reader = input;
        TokenType myObj = null;
        if((myObj = next()) != null){
            offsetAttribute.setOffset(myObj.startOffset, myObj.endOffset);
            return true;
            return false;

now when i search for a text in my index i want to retrieve the type,
offset and charTermAttribute of the matched tokens.

to achieve this i am using the matched documents to retrieve the
DocsAndPositionsEnum object and then calling the method startOffset() and
endOffset() to retrieve the offsets and getPayload() to get the payload.
but i am not able to retrieve the type and charTermAttribute values of the
matched terms. Below is the method where i am doing all the stuff to
retrieve the offsets.

private void showHits(TermQuery query, TopDocs hits)
            throws CorruptIndexException, IOException {
                .debug("Found " + hits.totalHits
                        + " document(s) that matched query '"
                        + query.toString() + "':");
        for (ScoreDoc scoreDoc : hits.scoreDocs) {
            //Get the document
            Document doc = iSearcher.doc(scoreDoc.doc);
            ProgressSearchEngine.debug("File Name:: "
                    + doc.get(FIELD_FILE_PATH));
            //Get the terms of that document
            Terms termsVector = iReader.getTermVector(scoreDoc.doc, query

            if (termsVector != null) {
                TermsEnum termsEnum = null;
                termsEnum = termsVector.iterator(termsEnum);
                //seek to the exact position of the matched term
                if (termsEnum.seekExact(new
                        false)) {

                    DocsAndPositionsEnum dpEnum = null;
                    dpEnum = termsEnum.docsAndPositions(null, dpEnum);

                    if (dpEnum != null) {

                         if (dpEnum.nextDoc() == 0) { // you need to call
nextDoc() to have the enum positioned

                             int freq = dpEnum.freq();

                                for(int i=0;i < freq; ++i){
                                    int position = dpEnum.nextPosition();
                                    if(position != -1){
                                        String filePath =
                                        System.out.println("file path
                                        System.out.println("Start offset "
                                                + dpEnum.startOffset() + "
End offset "
                                                + dpEnum.endOffset());



                                        "Not able to find the offsets for
the file: "+ doc.get(FIELD_FILE_PATH));




Can someone please help me how to get all the attributes that we set in the
incrementToken method.

And can we add our own attribute apart from already available ones? if yes

Nischal Y

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message