Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of goksron@gmail.com designates
 209.85.221.191 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=u2/qrPPOzQ5r+7CcOwslUIiPuaBWS0xORHM8RCjKfGSneDUZeH9tCRNgaxrRFMI06j
         J8rpE/Vy2FkJbksL1QASWHTSF31uqFn+i+b9xAHxmXHw4sibTap+BLJHdXAZ73ID8w6O
         YS60oukOFNubjH+PkwByJxIbY1jvkTRSLd9Uc=
MIME-Version: 1.0
In-Reply-To: <4BE5A14C.6040108@gmail.com>
References: <796818.71994.qm@web29007.mail.ird.yahoo.com>
	 <4BE51387.7050304@getopt.org>
	 <6F4496E2-F0F2-4732-A4C3-788C71BC0C53@yahoo.co.uk>
	 <4BE5A14C.6040108@gmail.com>
Date: Sat, 8 May 2010 16:32:50 -0700
Message-ID: <x2x87c998321005081632i81abf87fy8a1368b3413c9e1@mail.gmail.com>
Subject: Re: Adding another dimension to Lucene searches
From: Lance Norskog <goksron@gmail.com>
To: dev@lucene.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

There are two separate problems that I know of in indexing parts of
PDFs in an overlapping way:

1) block-structured documents of
   a) the entire PDF file
   b) chapters
   c) sections of chapters
   d.....z)
2)   Tracking the set of pages that each document contains.

As I understand this, LUCENE-2324 handles the first case but not the
second. True?

On Sat, May 8, 2010 at 10:37 AM, Michael Busch <buschmic@gmail.com> wrote:
> On 5/8/10 3:10 AM, Mark Harwood wrote:
>>
>> The downside is the need to maintain sequences of related docs in the sa=
me
>> segment - something Lucene currently doesn't make easy with its limited
>> control over when segments are flushed. I suspect we'll need some discus=
sion
>> on how best to support this.
>>
>
> LUCENE-2324 should help to make this work even when you add documents wit=
h
> multiple threads. =C2=A0There will be one DocumentsWriter per thread (DWP=
T), and
> the different DWPTs will write to their own segments. =C2=A0We will also =
have an
> extension point to control thread binding. =C2=A0Then you can make sure t=
hat all
> parts of your compound document end up sequentially in the same segment.
>
> One thing we have to make sure though is that a DWPT doesn't flush "betwe=
en"
> different parts of your compound doc. =C2=A0Hmm, we might have to add a "=
flush
> policy" to our growing family of policies.
>
> =C2=A0Michael
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>


--=20
Lance Norskog
goksron@gmail.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org