Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
MIME-Version: 1.0
In-Reply-To: <55CB5F65.5000806@tnstaafl.net>
References: <566901b4e7234461866e02b03dec0589@SJCPEXCH05.corp.avanade.org>
	<alpine.DEB.2.11.1508101404350.4296@tray>
	<389050a871f24eff92a79b4eadcaf3a3@SJCPEXCH05.corp.avanade.org>
	<55CA46B6.9000107@tnstaafl.net>
	<3BA01A6CF46D6640B83AC1D1B5D16532FCBAAB@EXMBX01L-CRP-03.webmdhealth.net>
	<55CA9B2E.60102@tnstaafl.net>
	<79487E45-F2F7-4CF7-A436-3CF222AB8ACA@gmail.com>
	<55CB3E81.40901@tnstaafl.net>
	<55CB5627.8030808@tnstaafl.net>
	<55CB5F65.5000806@tnstaafl.net>
Date: Wed, 12 Aug 2015 08:27:19 -0700
Message-ID: 
 <CAN4YXvfWQDDrnAnF1XCpqK6xF6enzTjYNwR8WA2xTFMHh64Sow@mail.gmail.com>
Subject: Re: Highlighting, all matches show empty {}
From: Erick Erickson <erickerickson@gmail.com>
To: solr-user@lucene.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Well, the example you just showed shouldn't show any highlighting. Your que=
ry is
q=3Dconcord
so it's trying to highlight "concord" which isn't in any of your
documents. hl.q can be
used to highlight something other than your q parameter.

I did notice in some of your other examples that you seemed to be searching=
 for
terms that were in the fields so I suspect this isn't really your root
problem though.

do note that fields _must_ be stored to have highlighting work. Is it
possible that your
matches are on fields that aren't stored?

Let's build it up slowly though, try searching on one term in one
field that you _know_
is stored and see if you get anything back. While the query with
hl.fl=3D* and fl=3Dfield1, field2,
should be fine, let's start as simply as possible and work up maybe?

Best,
Erick

On Wed, Aug 12, 2015 at 7:59 AM, Scott Derrick <scott@tnstaafl.net> wrote:
> I think the highlighter is actually running, but I'm not getting the
> results??
>
> with this request
>
> http://localhost:8983/solr/mbepp/select?q=3Dconcord&fl=3Daccession%2C+tit=
le%2C+author%2C+date&wt=3Djson&indent=3Dtrue&hl=3Dtrue&hl.fl=3D*
>
>
> I get this response
>
> {
>   "responseHeader":{
>     "status":0,
>     "QTime":3,
>     "params":{
>       "q":"concord",
>       "hl":"true",
>       "indent":"true",
>       "fl":"accession, title, author, date",
>       "hl.fl":"*",
>       "wt":"json"}},
>   "response":{"numFound":3,"start":0,"docs":[
>       {
>         "date":"1890-02-26",
>         "author":"Mary Baker Eddy",
>         "accession":"L13943",
>         "title":["Mary Baker Eddy to Joseph E. Adams,"]},
>       {
>         "date":"1896-01-13",
>         "author":"Mary Baker Eddy",
>         "accession":"L03453",
>         "title":["Mary Baker Eddy to Ira O. Knapp,"]},
>       {
>         "date":"1902-06-15",
>         "author":"Mary Baker Eddy",
>         "accession":"A10145",
>         "title":["Message of the Pastor Emeritus to The First Church of
> Christ, Scientist, Boston, Mass., June 15, 1902"]}]
>   },
>   "highlighting":{
>
> "/home/scott/workspace/mbel-work/tei2html/build/web/L13943/L13943.html":{=
},
>
> "/home/scott/workspace/mbel-work/tei2html/build/web/L03453/L03453.html":{=
},
>
> "/home/scott/workspace/mbel-work/tei2html/build/web/A10145/A10145.html":{=
}}}
>
> When I ran the request.
> In the admin plubins/Stats I set "Watch Changes" before processing the
> request.  Highlighting showed 2 changes, the gapFragmenter and HTMLFormat=
ter
>
> here are the reported changes
>
> org.apache.solr.highlight.GapFragmenter
>     class: org.apache.solr.highlight.GapFragmenter
>     version: 5.2.1
>     description: GapFragmenter
>     stats: requests: Was: 117, Now: 156, Delta: 39
>
> org.apache.solr.highlight.HtmlFormatter
>     class: org.apache.solr.highlight.HtmlFormatter
>     version:5.2.1
>     description:HtmlFormatter
>     stats: requests: Was: 117, Now: 156, Delta: 39
>
> Looks to me like there were 39 fragments or something processed, yet you =
can
> see above the highlights are empty {}???
>
> though all the the other libraries in the highlighter showed no changes.
>
> which are these...
>
>     org.apache.solr.highlight.BreakIteratorBoundaryScanner
>     org.apache.solr.highlight.HtmlEncoder
>     org.apache.solr.highlight.RegexFragmenter
>     org.apache.solr.highlight.ScoreOrderFragmentsBuilder
>     org.apache.solr.highlight.SimpleBoundaryScanner
>     org.apache.solr.highlight.SimpleFragListBuilder
>     org.apache.solr.highlight.SingleFragListBuilder
>     org.apache.solr.highlight.WeightedFragListBuilder
>
>
> Scott
>
> -------- Original Message --------
> Subject: Highlighting, all matches show empty {}
> From: Scott Derrick <scott@tnstaafl.net>
> To: solr-user@lucene.apache.org
> Date: 08/12/2015 08:20 AM
>
>> Tried submitting a filed for hl.fl still empty {}
>>
>> here are the query terms
>>
>> "responseHeader": {
>>      "status": 0,
>>      "QTime": 8,
>>      "params": {
>>        "q": "mary or calvin",
>>        "hl": "true",
>>        "hl.simple.post": "</em>",
>>        "indent": "true",
>>        "fl": "accession, title, author, date",
>>        "hl.fl": "*",
>>        "wt": "json",
>>        "hl.simple.pre": "<em>",
>>        "_": "1439388969240"
>>      }
>>
>> here is one of the responses, there were 135
>>
>> {
>>          "date": "1886-07-06",
>>          "author": "Mary Baker Eddy",
>>          "accession": "L02634",
>>          "title": [
>>            "Mary Baker Eddy to Josephine C. Woodbury, July 6, 1886"
>>          ]
>> },
>>
>> here is the highlight section listing the first 10 matches, still empty =
{}
>>
>> "highlighting": {
>>
>> "/home/scott/workspace/mbel-work/tei2html/build/web/./L02634/L02634.html=
":
>> {},
>>
>> "/home/scott/workspace/mbel-work/tei2html/build/web/./A10720/A10720.html=
":
>> {},
>>
>> "/home/scott/workspace/mbel-work/tei2html/build/web/./L07894/L07894.html=
":
>> {},
>>
>> "/home/scott/workspace/mbel-work/tei2html/build/web/./L09828/L09828.html=
":
>> {},
>>
>>
>> "/home/scott/workspace/mbel-work/tei2html/build/web/./A10636D/A10636D.ht=
ml":
>> {},
>>
>> "/home/scott/workspace/mbel-work/tei2html/build/web/./L13943/L13943.html=
":
>> {},
>>
>> "/home/scott/workspace/mbel-work/tei2html/build/web/./A10594/A10594.html=
":
>> {},
>>
>>
>> "/home/scott/workspace/mbel-work/tei2html/build/web/./A10385B/A10385B.ht=
ml":
>> {},
>>
>> "/home/scott/workspace/mbel-work/tei2html/build/web/./A10879/A10879.html=
":
>> {},
>>
>> "/home/scott/workspace/mbel-work/tei2html/build/web/./L00003/L00003.html=
":
>> {}
>>    }
>>
>>
>> -------- Original Message --------
>> Subject: Re: Highlighting
>> From: Scott Derrick <scott@tnstaafl.net>
>> To: solr-user@lucene.apache.org
>> Date: 08/12/2015 06:39 AM
>>
>>> I was pretty sure I tried that, though I thought if you don't specify i=
t
>>> just uses the search terms?
>>>
>>> If I just search for "calvin" and don't specify a field, what do I
>>> assign hl.fl?
>>>
>>> Scott
>>>
>>> On 8/11/2015 7:27 PM, Erik Hatcher wrote:
>>>>
>>>> Scott - doesn=E2=80=99t look you=E2=80=99ve specified hl.fl specifying=
 which field(s)
>>>> to highlight.
>>>>
>>>> p.s. Erick Erickson surely likes your e-mail domain :)
>>>>
>>>>
>>>> =E2=80=94
>>>> Erik Hatcher, Senior Solutions Architect
>>>> http://www.lucidworks.com <http://www.lucidworks.com/>
>>>>
>>>>
>>>>
>>>>
>>>>> On Aug 11, 2015, at 9:02 PM, Scott Derrick <scott@tnstaafl.net> wrote=
:
>>>>>
>>>>> I guess I really don't get Highlighting in Solr.
>>>>>
>>>>> We are transitioning from Google Custom Search which generally sucks,
>>>>> but does return nicely formatted highlighted fragment.
>>>>>
>>>>> I turn highlighting on hl=3Dtrue in the query and I get a highlightin=
g
>>>>> section returned at the bottom of the page, each identified by the
>>>>> document file name with a empty {} .  It doesn't matter what I search
>>>>> for, plain text, a field, I get a list of documents followed by an
>>>>> empty brace?
>>>>>
>>>>> "highlighting": {
>>>>>
>>>>> "/home/scott/workspace/mbel-work/tei2html/build/web/./A10385B/A10385B=
.html":
>>>>>
>>>>> {},
>>>>>
>>>>> "/home/scott/workspace/mbel-work/tei2html/build/web/./A10089/A10089.h=
tml":
>>>>>
>>>>> {},
>>>>>
>>>>> "/home/scott/workspace/mbel-work/tei2html/build/web/./L00003/L00003.h=
tml":
>>>>>
>>>>> {},
>>>>>
>>>>> "/home/scott/workspace/mbel-work/tei2html/build/web/./A10646/A10646.h=
tml":
>>>>>
>>>>> {},
>>>>>
>>>>> "/home/scott/workspace/mbel-work/tei2html/build/web/./V03482/V03482.h=
tml":
>>>>>
>>>>> {},
>>>>>
>>>>> "/home/scott/workspace/mbel-work/tei2html/build/web/./A10594/A10594.h=
tml":
>>>>>
>>>>> {},
>>>>>
>>>>> "/home/scott/workspace/mbel-work/tei2html/build/web/./645A.66.043/645=
A.66.043.html":
>>>>>
>>>>> {},
>>>>>
>>>>> "/home/scott/workspace/mbel-work/tei2html/build/web/./352.48.001/352.=
48.001.html":
>>>>>
>>>>> {},
>>>>>
>>>>> "/home/scott/workspace/mbel-work/tei2html/build/web/./144.23.001/144.=
23.001.html":
>>>>>
>>>>> {},
>>>>>
>>>>> "/home/scott/workspace/mbel-work/tei2html/build/web/./L18512/L18512.h=
tml":
>>>>>
>>>>> {}
>>>>>   }
>>>>>
>>>>> I haven't made any changes to the default settings
>>>>>
>>>>>    <highlighting>
>>>>>       <!-- Configure the standard fragmenter -->
>>>>>       <!-- This could most likely be commented out in the "default"
>>>>> case -->
>>>>>       <fragmenter name=3D"gap"
>>>>>                   default=3D"true"
>>>>>                   class=3D"solr.highlight.GapFragmenter">
>>>>>         <lst name=3D"defaults">
>>>>>           <int name=3D"hl.fragsize">100</int>
>>>>>         </lst>
>>>>>       </fragmenter>
>>>>>
>>>>>       <!-- A regular-expression-based fragmenter
>>>>>            (for sentence extraction)
>>>>>         -->
>>>>>       <fragmenter name=3D"regex"
>>>>>                   class=3D"solr.highlight.RegexFragmenter">
>>>>>         <lst name=3D"defaults">
>>>>>           <!-- slightly smaller fragsizes work better because of slop
>>>>> -->
>>>>>           <int name=3D"hl.fragsize">70</int>
>>>>>           <!-- allow 50% slop on fragment sizes -->
>>>>>           <float name=3D"hl.regex.slop">0.5</float>
>>>>>           <!-- a basic sentence pattern -->
>>>>>           <str name=3D"hl.regex.pattern">[-\w
>>>>> ,/\n\&quot;&apos;]{20,200}</str>
>>>>>         </lst>
>>>>>       </fragmenter>
>>>>>
>>>>>       <!-- Configure the standard formatter -->
>>>>>       <formatter name=3D"html"
>>>>>                  default=3D"true"
>>>>>                  class=3D"solr.highlight.HtmlFormatter">
>>>>>         <lst name=3D"defaults">
>>>>>           <str name=3D"hl.simple.pre"><![CDATA[<em>]]></str>
>>>>>           <str name=3D"hl.simple.post"><![CDATA[</em>]]></str>
>>>>>         </lst>
>>>>>       </formatter>
>>>>>
>>>>>       <!-- Configure the standard encoder -->
>>>>>       <encoder name=3D"html"
>>>>>                class=3D"solr.highlight.HtmlEncoder" />
>>>>>
>>>>>       <!-- Configure the standard fragListBuilder -->
>>>>>       <fragListBuilder name=3D"simple"
>>>>>                        class=3D"solr.highlight.SimpleFragListBuilder"=
/>
>>>>>
>>>>>       <!-- Configure the single fragListBuilder -->
>>>>>       <fragListBuilder name=3D"single"
>>>>>                        class=3D"solr.highlight.SingleFragListBuilder"=
/>
>>>>>
>>>>>       <!-- Configure the weighted fragListBuilder -->
>>>>>       <fragListBuilder name=3D"weighted"
>>>>>                        default=3D"true"
>>>>>                        class=3D"solr.highlight.WeightedFragListBuilde=
r"/>
>>>>>
>>>>>       <!-- default tag FragmentsBuilder -->
>>>>>       <fragmentsBuilder name=3D"default"
>>>>>                         default=3D"true"
>>>>>
>>>>> class=3D"solr.highlight.ScoreOrderFragmentsBuilder">
>>>>>         <!--
>>>>>         <lst name=3D"defaults">
>>>>>           <str name=3D"hl.multiValuedSeparatorChar">/</str>
>>>>>         </lst>
>>>>>         -->
>>>>>       </fragmentsBuilder>
>>>>>
>>>>>       <!-- multi-colored tag FragmentsBuilder -->
>>>>>       <fragmentsBuilder name=3D"colored"
>>>>>
>>>>> class=3D"solr.highlight.ScoreOrderFragmentsBuilder">
>>>>>         <lst name=3D"defaults">
>>>>>           <str name=3D"hl.tag.pre"><![CDATA[
>>>>>                <b style=3D"background:yellow">,<b
>>>>> style=3D"background:lawgreen">,
>>>>>                <b style=3D"background:aquamarine">,<b
>>>>> style=3D"background:magenta">,
>>>>>                <b style=3D"background:palegreen">,<b
>>>>> style=3D"background:coral">,
>>>>>                <b style=3D"background:wheat">,<b
>>>>> style=3D"background:khaki">,
>>>>>                <b style=3D"background:lime">,<b
>>>>> style=3D"background:deepskyblue">]]></str>
>>>>>           <str name=3D"hl.tag.post"><![CDATA[</b>]]></str>
>>>>>         </lst>
>>>>>       </fragmentsBuilder>
>>>>>
>>>>>       <boundaryScanner name=3D"default"
>>>>>                        default=3D"true"
>>>>>                        class=3D"solr.highlight.SimpleBoundaryScanner"=
>
>>>>>         <lst name=3D"defaults">
>>>>>           <str name=3D"hl.bs.maxScan">10</str>
>>>>>           <str name=3D"hl.bs.chars">.,!? &#9;&#10;&#13;</str>
>>>>>         </lst>
>>>>>       </boundaryScanner>
>>>>>
>>>>>       <boundaryScanner name=3D"breakIterator"
>>>>>
>>>>> class=3D"solr.highlight.BreakIteratorBoundaryScanner">
>>>>>         <lst name=3D"defaults">
>>>>>           <!-- type should be one of CHARACTER, WORD(default), LINE
>>>>> and SENTENCE -->
>>>>>           <str name=3D"hl.bs.type">WORD</str>
>>>>>           <!-- language and country are used when constructing Locale
>>>>> object.  -->
>>>>>           <!-- And the Locale object will be used when getting
>>>>> instance of BreakIterator -->
>>>>>           <str name=3D"hl.bs.language">en</str>
>>>>>           <str name=3D"hl.bs.country">US</str>
>>>>>         </lst>
>>>>>       </boundaryScanner>
>>>>>     </highlighting>
>>>>
>>>>
>>>
>>>
>>> ---
>>> This email has been checked for viruses by Avast antivirus software.
>>> https://www.avast.com/antivirus
>>>
>>>
>>
>
> --
> One man's "magic" is another man's engineering. "Supernatural" is a null
> word.=E2=80=9D
> Robert A. Heinlein
>