lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Pugh <>
Subject Should analysis.jsp honor maxFieldLength
Date Tue, 24 Aug 2010 16:03:17 GMT
Hi all,

I have maxFieldLength set to 10000 in solrconfig.xml, but was playing around with really large
document (The King James Bible) in analysis.jsp.   I hacked analysis.jsp to show me the number
of terms at each filter, and the headers, but without turning everything on by checkboxing

My results shown at this screenshot:
seem to confirm that maxFieldLength is NOT honored by the analysis.jsp.   

But it seems to me that folks using analysis.jsp would expect the process to be exactly like
what happens during a document being indexed??   In my specific case, it took me a while to
realize that the reason my indexing results differed from analysis.jsp results was because
indexing only looked at the first 10000 tokens, but analysis looked at all 101561. A horizontal
table of 10,000 cells kind of looks like a horizontal field of 101,561 cells!

Would it make sense to parse the text through the DocInverterPerField in analysis.jsp?  Or
to maybe just modify the getTokens method in analysis.jsp to only parse maxFieldLength tokens?
 I think I can do it via looking up the SolrCore, and doing core.getSolrConfig().mainIndexConfig.maxFieldLength


Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 |
Co-Author: Solr 1.4 Enterprise Search Server available from

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message