Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of anshulnirvana@gmail.com
 designates 209.85.218.15 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:date:x-google-sender-auth:message-id:subject
         :from:to:content-type;
        b=jhq8MTuxyHZzicp8hDH9Fb75j9rYokH+aGvdn3ZI61NNZtcYDT1YP+4TJXMEglNa/u
         YiDsEG9puczX+midPDNgtDYxBdzWPuYzrLNgId25kQMtP3cYE1qz5MOPBn47BJVSOa7+
         YLxQLcKlnV/9SeShLFr/1e4H8I81sQkb8x+4I=
MIME-Version: 1.0
Sender: anshulnirvana@gmail.com
Date: Wed, 21 Jan 2009 14:39:39 +0100
Message-ID: <b8bb47ad0901210539j56edb59wfe63517322673503@mail.gmail.com>
Subject: Lucene Performance issue
From: Anshul jain <anshul.jain@epfl.ch>
To: java-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=001636c5ac8c9a0a070460fe4924

--001636c5ac8c9a0a070460fe4924
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Hi,

I've indexed around half a million XML documents. Here is the document
sample:

 <a:attribute>

<a:name>cogito:Name</a:name>

<a:value>Alexander the Great</a:value>

</a:attribute>


<a:attribute>

<a:name>cogito:domain</a:name>

<a:value>ancient history</a:value>

</a:attribute>


<a:attribute>

<a:name>cogito:first_sentence</a:name>

<a:value>

Alexander the Great (Greek: or Megas Alexandros; July 20 356 BC June 10 323
BC), also known as Alexander III, was an ancient Greek king (basileus) of
Macedon (336-323 BC).

</a:value>

</a:attribute>


Average size of documents is around 4KB.

There are a few performance issues I need help with. When I index documents,
in a structured manner, using field information like:
name: alexander the great
domain: ancient history
first_sentence: Alexander the Great (Greek: or Megas Alexandros; July 20 356
BC June 10 323 BC), also known as Alexander III, was an ancient Greek king
(basileus) of Macedon (336-323 BC).
bagOfWords: alexander the great ancient history Alexander the Great (Greek:
or Megas Alexandros; July 20 356 BC June 10 323 BC), also known as Alexander
III, was an ancient Greek king (basileus) of Macedon (336-323 BC).

bagOfWords is the field with all the text appended to it.

I get the index size of 4.5 GB, but if I just append the text and store in
one field
like:
value: alexander the great ancient history Alexander the Great (Greek: or
Megas Alexandros; July 20 356 BC June 10 323 BC), also known as Alexander
III, was an ancient Greek king (basileus) of Macedon (336-323 BC).

 the index size is only 700 MB.. why is this happening?


Also the query execution time of MultiFieldQueries is very slow, it is 20
times slower than single field query. Is it normal,  what could be the
reason for that?

Thanks,
Cheers,
Anshul

-- 
Anshul Jain

--001636c5ac8c9a0a070460fe4924--