Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <50EE2408.6050002@getrailo.org>
Date: Wed, 09 Jan 2013 18:14:32 -0800
From: "Igal @ getRailo.org" <igal@getrailo.org>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
 rv:17.0) Gecko/17.0 Thunderbird/17.0
MIME-Version: 1.0
To: java-user@lucene.apache.org
Subject: Re: NPE when adding a Document to an IndexWriter
References: <50EE1110.7010900@getrailo.org>
 <alpine.DEB.2.02.1301091717130.28372@frisbee>
In-Reply-To: <alpine.DEB.2.02.1301091717130.28372@frisbee>
Content-Type: multipart/mixed;
 boundary="------------070409060303050309080808"

--------------070409060303050309080808
Content-Type: multipart/alternative;
 boundary="------------030406020505000104070406"


--------------030406020505000104070406
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

thanks for your reply.  please see attached.  I tried to maintain the 
structure of the code that I need to use in the library I'm building.  I 
think it should work for you as long as you remove the package 
declaration at the top.

when I run the attached file I get the following output:

debug:
Exception in thread "main" java.lang.NullPointerException
     at 
org.apache.lucene.analysis.util.CharacterUtils$Java5CharacterUtils.fill(CharacterUtils.java:191)
     at 
org.apache.lucene.analysis.util.CharTokenizer.incrementToken(CharTokenizer.java:153)
     at 
org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:102)
     at 
org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:307)
     at 
org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:244)
     at 
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:373)
     at 
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1445)
     at 
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1124)
     at 
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1105)
     at s21waf.text.lucene4.TestNPE.testIndexWriter(TestNPE.java:47)
     at s21waf.text.lucene4.TestNPE.main(TestNPE.java:111)
Java Result: 1
BUILD SUCCESSFUL (total time: 0 seconds)

thanks,

Igal


On 1/9/2013 5:23 PM, Chris Hostetter wrote:
> : I keep getting an NPE when trying to add a Doc to an IndexWriter. I've
> : minimized my code to very basic code.  what am I doing wrong? pseudo-code:
>
> can you post a full test that other people can run to try and reproduce?
>
> it doesn't even have to be a junit test -- just some complete javacode
> people paste into a main method and compile would be enough (right now we
> have no idea what IndexWriterConfig you are using (could easily affect
> things) or what directory you are using (less likeley, but still)
>
>
> -Hoss
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>


--------------030406020505000104070406
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    thanks for your reply.&nbsp; please see attached.&nbsp; I tried to maintain
    the structure of the code that I need to use in the library I'm
    building.&nbsp; I think it should work for you as long as you remove the
    package declaration at the top.<br>
    <br>
    when I run the attached file I get the following output:<br>
    <font color="#6600cc"><br>
      debug:<br>
      Exception in thread "main" java.lang.NullPointerException<br>
      &nbsp;&nbsp;&nbsp; at
org.apache.lucene.analysis.util.CharacterUtils$Java5CharacterUtils.fill(CharacterUtils.java:191)<br>
      &nbsp;&nbsp;&nbsp; at
org.apache.lucene.analysis.util.CharTokenizer.incrementToken(CharTokenizer.java:153)<br>
      &nbsp;&nbsp;&nbsp; at
org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:102)<br>
      &nbsp;&nbsp;&nbsp; at
org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:307)<br>
      &nbsp;&nbsp;&nbsp; at
org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:244)<br>
      &nbsp;&nbsp;&nbsp; at
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:373)<br>
      &nbsp;&nbsp;&nbsp; at
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1445)<br>
      &nbsp;&nbsp;&nbsp; at
      org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1124)<br>
      &nbsp;&nbsp;&nbsp; at
      org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1105)<br>
      &nbsp;&nbsp;&nbsp; at
      s21waf.text.lucene4.TestNPE.testIndexWriter(TestNPE.java:47)<br>
      &nbsp;&nbsp;&nbsp; at s21waf.text.lucene4.TestNPE.main(TestNPE.java:111)<br>
      Java Result: 1<br>
      BUILD SUCCESSFUL (total time: 0 seconds)</font><br>
    <br>
    thanks,<br>
    <br>
    Igal<br>
    <br>
    <br>
    <div class="moz-cite-prefix">On 1/9/2013 5:23 PM, Chris Hostetter
      wrote:<br>
    </div>
    <blockquote cite="mid:alpine.DEB.2.02.1301091717130.28372@frisbee"
      type="cite">
      <pre wrap="">
: I keep getting an NPE when trying to add a Doc to an IndexWriter. I've
: minimized my code to very basic code.  what am I doing wrong? pseudo-code:

can you post a full test that other people can run to try and reproduce?  

it doesn't even have to be a junit test -- just some complete javacode 
people paste into a main method and compile would be enough (right now we 
have no idea what IndexWriterConfig you are using (could easily affect 
things) or what directory you are using (less likeley, but still)


-Hoss

---------------------------------------------------------------------
To unsubscribe, e-mail: <a class="moz-txt-link-abbreviated" href="mailto:java-user-unsubscribe@lucene.apache.org">java-user-unsubscribe@lucene.apache.org</a>
For additional commands, e-mail: <a class="moz-txt-link-abbreviated" href="mailto:java-user-help@lucene.apache.org">java-user-help@lucene.apache.org</a>

</pre>
    </blockquote>
    <br>
  </body>
</html>

--------------030406020505000104070406--

--------------070409060303050309080808
Content-Type: text/plain; charset=windows-1252;
 name="TestNPE.java"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="TestNPE.java"

package s21waf.text.lucene4;

import java.io.IOException;
import java.io.Reader;
import java.io.StringReader;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.TokenStream;
import org.apache.lucene.analysis.Tokenizer;
import org.apache.lucene.analysis.core.WhitespaceTokenizer;
import org.apache.lucene.analysis.standard.StandardTokenizer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.document.TextField;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.FSDirectory;
import org.apache.lucene.util.Version;


public class TestNPE {
 
    
    public static void testIndexWriter() throws IOException {
        
        Directory dir = FSDirectory.open( new java.io.File( "F:/Test/Lucene/dir1" ) );
        
        Document doc = new Document();
                
        TextField ft;
        
        ft = new TextField( "desc1", "word1", Field.Store.YES );        
        doc.add( ft );
        
        ft = new TextField( "desc2", "word2", Field.Store.YES );        
        doc.add( ft );
                
        Analyzer analyzer = createAnalyzer( TokenizerConfig.DEFAULT );
        
        IndexWriterConfig iwc = new IndexWriterConfig( Version.LUCENE_40, analyzer );
        
        IndexWriter iw = new IndexWriter( dir, iwc);
        
        iw.addDocument(doc);
        
        iw.close();
    }
    
    
    /** returns a WhitespaceTokenizerExt Tokenizer that strips html and replaces commas with comma-space */
    public static Tokenizer getCharTokenizer( Reader input ) {
        
//        Tokenizer result = new WhitespaceTokenizer( Version.LUCENE_40, getCharFilter( input ) );
        
        Tokenizer result = new WhitespaceTokenizer( Version.LUCENE_40, input );
        
//        Tokenizer result = new StandardTokenizer( Version.LUCENE_40, input );
        
        return result;
    }

    
    /** return getTokenizer( new StringReader( input ) ); */
    public static Tokenizer getCharTokenizer( String input ) {
        
        return getCharTokenizer( new StringReader( input ) );
    }
    
    
    public static Analyzer createAnalyzer( final TokenizerConfig config ) {
        
        Analyzer result = new Analyzer() {

            @Override
            protected Analyzer.TokenStreamComponents createComponents( String fieldName, Reader reader ) {

                Analyzer.TokenStreamComponents tsc = new Analyzer.TokenStreamComponents( 
                        
                          getCharTokenizer( reader )
                        , getTokenFilterChain( reader, config ) 
                );
                
                return tsc;
            }
            
        };
        
        return result;
    }
       
    
    public static TokenStream getTokenFilterChain( String input, TokenizerConfig config ) {
        
        return getTokenFilterChain( new StringReader( input ), config );
    }
    
    
    public static TokenStream getTokenFilterChain( Reader input, TokenizerConfig config ) {
        
        TokenStream result = getCharTokenizer( input );

        return result;
    }
    
    
    public static void main(String[] args) throws IOException {
    
        testIndexWriter();
        
    }
    
    
    public static class TokenizerConfig {
        
        private int shingleSize = 3;
        private Map replaceList = Collections.EMPTY_MAP;
        private Map typeList = Collections.EMPTY_MAP;
        private List synonymList = Collections.EMPTY_LIST;
        
        
        public int getShingleSize() {
            
            return this.shingleSize;
        }
        
        public void setShingleSize( int value ) {
            
            this.shingleSize = value;
        }
        
        
        public void setReplaceList( Map replaceList ) {
            
            this.replaceList = replaceList;
        }        
        
        public Map getReplaceList() {
            
            return this.replaceList;
        }
        
        
        public void setTypeList( Map typeList ) {
            
            this.typeList = typeList;
        }        
        
        public Map getTypeList() {
            
            return this.typeList;
        }
        
        
        public void setSynonyms( List<String> synonymList ) {
            
            this.synonymList = synonymList;
        }
        
        public List getSynonyms() {
            
            return synonymList;
        }
        
        public static final TokenizerConfig DEFAULT = new TokenizerConfig();
        
    }
    
}


--------------070409060303050309080808
Content-Type: text/plain; charset=us-ascii


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
--------------070409060303050309080808--