Mailing-List: contact lucene-user-help@jakarta.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Lucene Users List" <lucene-user@jakarta.apache.org>
From: "Josh Guice" <Joshua.Guice@nssl.noaa.gov>
To: "'Lucene Users List'" <lucene-user@jakarta.apache.org>
Subject: RE: Manually creating a RAMDirectory from pre-existing index files.
Date: Fri, 5 Apr 2002 13:39:50 -0600
Message-ID: <000801c1dcd9$a8f81890$e11310ac@ernie>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Importance: Normal
In-Reply-To: <000701c1dcd5$5e530b50$e11310ac@ernie>

I'll be answering my own question here...sort of:

Apparently all my back-tracking through code has put me in a mode to
approach *other* things backwards as well.  After just discovering the
archive search engine, I located this:

______________________________________________________________________

        //Variable to store the names of the indexfiles in   =20
        Stack stackFileNames =3D new Stack();

        //open the file with the filenames
        //you need this, because you cannot search a directory for files
an
an URL
      URL source =3D new URL(getCodeBase(), "filenames.txt");
      BufferedReader in =3D new BufferedReader(new
InputStreamReader(source.openStream()));
      while(true)
      {
        String s =3D in.readLine();
        if(s=3D=3Dnull)
          break;
        stackFileNames.push(s);
      }
      in.close();

      //open the index
      while(!stackFileNames.empty())
      {
        String sFileName =3D (String)stackFileNames.pop();
        //streaming the file to the RAMDirectory
        org.apache.lucene.store.OutputStream os =3D
ramDir.createFile(sFileName);
        java.io.InputStream is =3D new URL(getCodeBase(),
"repository/"+sFileName).openStream();

        byte[] baBuffer =3D new byte[1024];
        while(true)
        {
          synchronized(baBuffer)
          {
            int iBytesRead =3D is.read(baBuffer);
            if (iBytesRead =3D=3D -1)
              break;
            os.writeBytes(baBuffer, iBytesRead);
          }
        }
        is.close();
        os.close();
      }

      //IndexReader
      IndexReader ir =3D IndexReader.open(ramDir);
With this I can perform searchen in the Applet. Work=B4s fine.


Have fun,=20

Christoph Breidert=20
www.sitewaerts.de

______________________________________________________________________

This code works perfectly fine and appears to only be different from
mine in that it uses InputStream instead of an InputStreamReader and a
byte buffer as opposed to single bytes...?...offhand I don't see why it
works and my attempt failed, but chasing rabbits is a leisure-time
activity :)

Thanks to Christoph and anyone else who was considering my problem...

Josh

> -----Original Message-----
> From: Josh Guice [mailto:Joshua.Guice@nssl.noaa.gov]
> Sent: Friday, April 05, 2002 1:09 PM
> To: lucene-user@jakarta.apache.org
> Subject: FW: Manually creating a RAMDirectory from pre-existing index
> files.
>=20
> **Apologies for sending this to wrong list (for those of you on both),
I
> entered the mail archive from the rear-end and didn't realize there
was
> a user list as well as a developer list...again...sorry :)
>=20
> -----Original Message-----
> From: Josh Guice [mailto:Joshua.Guice@nssl.noaa.gov]
> Sent: Friday, April 05, 2002 1:01 PM
> To: 'lucene-dev@jakarta.apache.org'
> Subject: Manually creating a RAMDirectory from pre-existing index
files.
>=20
> (Using version 1.2rc4 of canned Lucene)
>=20
> I am working on a document navigation/browsing applet and want to
> implement a client-side search that is completely independent of any
> server mechanism, yet is still fast (i.e. not a "crawler").  It will
be
> LAN-based, so moving large files is not an issue.
>=20
> Currently I create an index of the files to be searched in the
> traditional Lucene manner and store them on a web server.  From the
> applet I open each of these pre-existing files and do a byte->byte
copy
> to a RAMDirectory (RD) in the client applet's memory.  I am then able
to
> access the files from the RD in the applet, list their sizes (which
> match up with the originals) and read back their contents (which also
> match up with the origs, byte-for-byte).  However, when I attempt to
> create an IndexSearcher by passing my RAMDirectory, I get the
following
> Exception:
>=20
> Error creating IndexSearcher!
>=20
> java.lang.ArrayIndexOutOfBoundsException: 116 >=3D 7
> 	at java.util.Vector.elementAt(Unknown Source)
> 	at org.apache.lucene.index.FieldInfos.fieldInfo(Unknown Source)
> 	at org.apache.lucene.index.FieldInfos.fieldName(Unknown Source)
> 	at org.apache.lucene.index.SegmentTermEnum.readTerm(Unknown
> Source)
> 	at org.apache.lucene.index.SegmentTermEnum.next(Unknown Source)
> 	at org.apache.lucene.index.TermInfosReader.readIndex(Unknown
> Source)
> 	at org.apache.lucene.index.TermInfosReader.<init>(Unknown
> Source)
> 	at org.apache.lucene.index.SegmentReader.<init>(Unknown Source)
> 	at org.apache.lucene.index.SegmentReader.<init>(Unknown Source)
> 	at org.apache.lucene.index.IndexReader$1.doBody(Unknown Source)
> 	at org.apache.lucene.store.Lock$With.run(Unknown Source)
> 	at org.apache.lucene.index.IndexReader.open(Unknown Source)
> 	at org.apache.lucene.search.IndexSearcher.<init>(Unknown Source)
> 	at Search.finishedLoad(Search.java:200)
> 	at fileLoader.run(fileLoader.java:98)
>=20
> I can successfully pass the physical index directory to an IndexReader
> and everything is fine, so I know the index files are ok.  Here is the
> code I use to get and copy each file to the client applets memory:
>=20
> 	public void addItem(String dataLine) // dataLine is the real
> filename
// of an
> index component
> 	{
> 		// Get file
> 		try
> 		{
> 			InputStreamReader fileIn;
> 			org.apache.lucene.store.OutputStream outFile;
>=20
> 			URL url;
> 			URLConnection urlConn;
> 			int EOF;
>=20
> 			// Process file
> 			url =3D new URL(
> 				_applet.getCodeBase().toString()
> 				+ "index/" + dataLine);
> 			urlConn =3D url.openConnection();
> 			urlConn.setDoInput(true);
> 			urlConn.setUseCaches(false);
>=20
> 			fileIn =3D new
> InputStreamReader(urlConn.getInputStream());
>=20
> 			System.out.println("File: " + dataLine +
> 							   ", Length: "
> + urlConn.getContentLength());
>=20
> 			// indices is a pre-defined RAMDirectory
> 			outFile =3D indices.createFile(dataLine);
>=20
> 			EOF =3D fileIn.read();
>=20
> 			while (EOF !=3D -1)
> 			{
> 				outFile.writeByte((byte)EOF);
> 				EOF =3D fileIn.read();
> 			}
>=20
> 			outFile.close();
> 		}
> 		catch (MalformedURLException mue)
> 		{
> 			System.out.println("!!Error Bad data file link "
> +
> 			_applet.getCodeBase().toString() + "/" +
> dataLine);
> 		}
> 		catch (IOException ioe)
> 		{
> 			System.out.println("!!Error Reading " +
> 			_applet.getCodeBase().toString() + "/" +
> dataLine);
> 		}
> 	}
>=20
> And here is the code I use to create the IndexSearcher:
>=20
> 		indices.close();
>=20
> 		try
> 		{
> 			searcher =3D new IndexSearcher(indices);
> 		}
> 		catch (Exception ioe)
> 		{
> 			System.out.println("Error creating
> IndexSearcher!");
> 			ioe.printStackTrace();
> 		}
>=20
> This is when the exception gets thrown...
>=20
> Any ideas on why this wouldn't be working?
>=20
> Thanks,
> Josh
>=20
>=20
> --
> To unsubscribe, e-mail:   <mailto:lucene-user-
> unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: <mailto:lucene-user-
> help@jakarta.apache.org>


--
To unsubscribe, e-mail:   <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>