lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (JIRA)" <j...@apache.org>
Subject [jira] Commented: (SOLR-1954) Highlighter component should expose snippet character offsets and the score.
Date Fri, 18 Jun 2010 23:10:36 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880360#action_12880360
] 

Robert Muir commented on SOLR-1954:
-----------------------------------

bq. so the only possible unit you can use to convey position information that can be ignorant
of the remote client is bytes.

I think bytes are no better, for example:

𩬅 is 6-bytes in UTF-16, 4-bytes in UTF-8, 2 chars in java.

in other programming languages maybe its 1 char and 4-bytes, in java its 2 chars and 6 bytes.

So this is why any offset numbers will be problematic, as they will invite bugs in client
applications.

> Highlighter component should expose snippet character offsets and the score.
> ----------------------------------------------------------------------------
>
>                 Key: SOLR-1954
>                 URL: https://issues.apache.org/jira/browse/SOLR-1954
>             Project: Solr
>          Issue Type: New Feature
>          Components: highlighter
>            Reporter: David Smiley
>            Priority: Minor
>         Attachments: SOLR-1954_start_and_end_offsets.patch
>
>
> The Highlighter Component does not currently expose the snippet character offsets nor
the score.  There is a TODO in DefaultSolrHighlighter indicating the intention to add this
eventually.  This information is needed when doing highlighting on external content.  The
data is there so its pretty easy to output it in some way.  The challenge is deciding on the
output and its ramifications on backwards compatibility.  The current highlighter component
response structure doesn't lend itself to adding any new data, unfortunately.  I wish the
original implementer had some foresight.  Unfortunately all the highlighting tests assume
this structure.  Here is a snippet of the current response structure in Solr's sample data
searching for "sdram" for reference:
> {code:xml}
> <lst name="highlighting">
>  <lst name="VS1GB400C3">
>   <arr name="text">
> 	<str>CORSAIR ValueSelect 1GB 184-Pin DDR &lt;em&gt;SDRAM&lt;/em&gt;
Unbuffered DDR 400 (PC 3200) System Memory - Retail</str>
>   </arr>
>  </lst>
> </lst>
> {code}
> Perhaps as a little hack, we introduce a pseudo field called text_startCharOffset which
is the concatenation of the matching field and "_startCharOffset".  This would be an array
of ints.  Likewise, there would be another array for endCharOffset and score.
> Thoughts?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message