lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel Bernstein <joels...@gmail.com>
Subject Re: igain query parser generating invalid output
Date Fri, 11 Oct 2019 15:34:16 GMT
This sounds like a great patch. I can help with the review and commit after
the jira is created.

Thanks!

Joel


On Fri, Oct 11, 2019 at 1:06 AM Peter Davie <
Peter.Davie@convergentsolutions.com.au> wrote:

> Hi,
>
> I apologise in advance for the length of this email, but I want to share
> my discovery steps to make sure that I haven't missed anything during my
> investigation...
>
> I am working on a classification project and will be using the
> classify(model()) stream function to classify documents.  I have noticed
> that models generated include many noise terms from the (lexically)
> early part of the term list.  To test, I have used the /BBC articles
> fulltext and category //dataset from Kaggle/
> (https://www.kaggle.com/yufengdev/bbc-fulltext-and-category). I have
> indexed the data into a Solr collection (news_categories) and am
> performing the following operation to generate a model for documents
> categorised as "BUSINESS" (only keeping the 100th iteration):
>
> having(
>      train(
>          news_categories,
>          features(
>              news_categories,
>              zkHost="localhost:9983",
>              q="*:*",
>              fq="role:train",
>              fq="category:BUSINESS",
>              featureSet="business",
>              field="body",
>              outcome="positive",
>              numTerms=500
>          ),
>          fq="role:train",
>          fq="category:BUSINESS",
>          zkHost="localhost:9983",
>          name="business_model",
>          field="body",
>          outcome="positive",
>          maxIterations=100
>      ),
>      eq(iteration_i, 100)
> )
>
> The output generated includes "noise" terms, such as the following
> "1,011.15", "10.3m", "01", "02", "03", "10.50", "04", "05", "06", "07",
> "09", and these terms all have the same value for idfs_ds ("-Infinity").
>
> Investigating the "features()" output, it seems that the issue is that
> the noise terms are being returned with NaN for the score_f field:
>
>      "docs": [
>        {
>          "featureSet_s": "business",
>          "score_f": "NaN",
>          "term_s": "1,011.15",
>          "idf_d": "-Infinity",
>          "index_i": 1,
>          "id": "business_1"
>        },
>        {
>          "featureSet_s": "business",
>          "score_f": "NaN",
>          "term_s": "10.3m",
>          "idf_d": "-Infinity",
>          "index_i": 2,
>          "id": "business_2"
>        },
>        {
>          "featureSet_s": "business",
>          "score_f": "NaN",
>          "term_s": "01",
>          "idf_d": "-Infinity",
>          "index_i": 3,
>          "id": "business_3"
>        },
>        {
>          "featureSet_s": "business",
>          "score_f": "NaN",
>          "term_s": "02",
>          "idf_d": "-Infinity",
>          "index_i": 4,
>          "id": "business_4"
>        },...
>
> I have examined the code within
> org/apache/solr/client/solrj/io/streamFeatureSelectionStream.java and
> see that the scores being returned by {!igain} include NaN values, as
> follows:
>
> {
>    "responseHeader":{
>      "zkConnected":true,
>      "status":0,
>      "QTime":20,
>      "params":{
>        "q":"*:*",
>        "distrib":"false",
>        "positiveLabel":"1",
>        "field":"body",
>        "numTerms":"300",
>        "fq":["category:BUSINESS",
>          "role:train",
>          "{!igain}"],
>        "version":"2",
>        "wt":"json",
>        "outcome":"positive",
>        "_":"1569982496170"}},
>    "featuredTerms":[
>      "0","NaN",
>      "0.0051","NaN",
>      "0.01","NaN",
>      "0.02","NaN",
>      "0.03","NaN",
>
> Looking intoorg/apache/solr/search/IGainTermsQParserPlugin.java, it
> seems that when a term is not included in the positive or negative
> documents, the docFreq calculation (docFreq = xc + nc) is 0, which means
> that subsequent calculations result in NaN (division by 0) which
> generates these meaningless values for the computed score.
>
> I have patched a local version of Solr to skip terms for which docFreq
> is 0 in the finish() method of IGainTermsQParserPlugin and this is now
> the result:
>
> {
>    "responseHeader":{
>      "zkConnected":true,
>      "status":0,
>      "QTime":260,
>      "params":{
>        "q":"*:*",
>        "distrib":"false",
>        "positiveLabel":"1",
>        "field":"body",
>        "numTerms":"300",
>        "fq":["category:BUSINESS",
>          "role:train",
>          "{!igain}"],
>        "version":"2",
>        "wt":"json",
>        "outcome":"positive",
>        "_":"1569983546342"}},
>    "featuredTerms":[
>      "3",-0.0173133558644304,
>      "authority",-0.0173133558644304,
>      "brand",-0.0173133558644304,
>      "commission",-0.0173133558644304,
>      "compared",-0.0173133558644304,
>      "condition",-0.0173133558644304,
>      "continuing",-0.0173133558644304,
>      "deficit",-0.0173133558644304,
>      "expectation",-0.0173133558644304,
>
> To my (admittedly inexpert) eye, it seems like this is producing more
> reasonable results.
>
> With this change in place, train() now produces:
>
>      "idfs_ds": [
>            0.6212826193303013,
>            0.6434237452075148,
>            0.7169578292536639,
>            0.741349282377823,
>            0.86843471069652,
>            1.0140549006400466,
>            1.0639267306802198,
>            1.0753554265038423,...
>
> |"terms_ss": [ "รข", "company", "market", "firm", "month", "analyst",
> "chief", "time",|||...| I am not sure if I have missed anything, but this
> seems like it's
> producing better outcomes. I would appreciate any input on whether I
> have missed anything here before I proceed further (JIRA and submit a
> patch). Kind Regards, Peter |
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message