Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of pgwillia@uwaterloo.ca
 designates 129.97.152.18 as permitted sender)
Message-ID: <473F3A32.9000202@uwaterloo.ca>
Date: Sat, 17 Nov 2007 12:00:02 -0700
From: Tricia Williams <pgwillia@uwaterloo.ca>
User-Agent: Thunderbird 2.0.0.6 (X11/20071022)
MIME-Version: 1.0
To: java-user@lucene.apache.org
Subject: Re: Payloads, Tokenizers, and Filters.  Oh My!
References: <473E21AC.5000702@uwaterloo.ca>
 <70CC07CE-5F57-4884-8E23-35F541FCE1CD@apache.org>
In-Reply-To: <70CC07CE-5F57-4884-8E23-35F541FCE1CD@apache.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Hi Grant,

    Thanks for your response!

    Taking a closer look at the TokenFilter(s) that causes my problem 
with the Payload are all from org.apache.solr.analysis rather than 
org.apache.lucene.analysis.  I had originally thought that all the 
TokenFilters available through Solr's TokenFilterFactory(s) were part of 
Lucene.  But I guess there are TokenFilters specific to Solr, such as 
the WordDelimiterFilter, that aren't aware of Payloads.  Thanks for 
saying exactly the right thing to make me realize that.
> I guess you just want to be careful about how big your payloads get.
    Erik Hatcher suggested storing the bulky XPath strings in a table of 
contents field and just storing a smaller representation of the 
information at each token with the intention of doing a lookup to get 
the bulky stuff at query time. 
>
> One of the original use cases for payloads was for doing XPath queries.
>
    Has anyone actually completed anything with XPath queries and Payloads?
> Also, the only thing experimental about Payloads is the actual 
> signature of the methods, not the need for them.  If anything, I think 
> you will see an expansion of payload capability in the future.  Also 
> note, that you will probably be interested in adding more Payload 
> querying capability.  And also note, I am in the process of adding the 
> ability to get payloads from Spans, but I am not sure if this gets 
> into 2.3 or not.
>
    I look forward to seeing more of Payloads!  I can already see how 
they can be extremely useful.

Thanks,
Tricia

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org