Mailing-List: contact solr-commits-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-dev@lucene.apache.org
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
From: Apache Wiki <wikidiffs@apache.org>
To: Apache Wiki <wikidiffs@apache.org>
Date: Fri, 26 Feb 2010 04:40:48 -0000
Message-ID: <20100226044048.9088.31800@eos.apache.org>
Subject: 
 =?utf-8?q?=5BSolr_Wiki=5D_Update_of_=22FieldAliasesAndGlobsInParams=22_by?=
 =?utf-8?q?_FrankWesemann?=

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Solr Wiki" for chan=
ge notification.

The "FieldAliasesAndGlobsInParams" page has been changed by FrankWesemann.
http://wiki.apache.org/solr/FieldAliasesAndGlobsInParams?action=3Ddiff&rev1=
=3D6&rev2=3D7

--------------------------------------------------

- This was prompted by some ideas put forth  in [[https://issues.apache.org=
/jira/browse/SOLR-247|SOLR-247]] and in the mailing list threads linked to =
from that issue.  See also: [[https://issues.apache.org/jira/browse/SOLR-45=
6|SOLR-456]] =

+ This was prompted by some ideas put forth  in [[https://issues.apache.org=
/jira/browse/SOLR-247|SOLR-247]] and in the mailing list threads linked to =
from that issue.  See also: [[https://issues.apache.org/jira/browse/SOLR-45=
6|SOLR-456]]
  =

  For now this is a brainstorming page, if/when any of this gets implemente=
d it can be reworked into a documentation page for users.
  =

  =3D Background =3D
- =

  Currently the [[CommonQueryParameters#head-db2785986af2355759faaaca53dc8f=
d0b012d1ab|fl]] param supports two "special" field names: "*" which means "=
any stored field", and "score" which not only means "include the score in t=
he response", but also informs the request handler that scores should be co=
mputed.  the fl param is split on the regex Pattern ",| ".
  =

  The splitting happens in !SolrPluginUtils.setReturnFields, which parses (=
one and only one) string "fl" param, and sets the field list on the !SolrQu=
eryResponse, as well as returning info about whether or not the list contai=
ned "score" so the handler has that info to work with.
  =

  Small problems with this (that most people have never cared about)...
+ =

-    * it makes it hard to use field names with spaces (or "|" or ",") ...n=
o other code in Solr cares what chars are in field names.
+  * it makes it hard to use field names with spaces (or "|" or ",") ...no =
other code in Solr cares what chars are in field names.
-    * you can't have a field named "score"
+  * you can't have a field named "score"
  =

  Some people expressed a desire to have "*" work for the facet.field param=
 as well ... see SOLR-247 for reasons why this is probably a bad idea, but =
having more generic glob syntax support (in both the "fl" and "facet.field"=
 params) would be handy.
  =

  =3D related issues =3D
- =

-    * "fl" can't be used as a multi-value param
+  * "fl" can't be used as a multi-value param
-    * no way to prevent certain users from getting certain fields
+  * no way to prevent certain users from getting certain fields
-    * no way to prevent faceting on certain fields
+  * no way to prevent faceting on certain fields
-    * in most cases, searching and sorting on the same logical field requi=
res clients to know two differenet field names (ie: "q=3Dname:foo&sort=3Dna=
me_sortable+asc")
+  * in most cases, searching and sorting on the same logical field require=
s clients to know two differenet field names (ie: "q=3Dname:foo&sort=3Dname=
_sortable+asc")
  =

  =3D Broad Idea =3D
- =

  Add robust support for letting solr admins configure what special syntax =
or aliases can be used *at query type* to refer to fields based on context =
(sorting, returned fields, search fields, facet fields, etc...)
  =

  new syntax in solrconfig.xml -- most of which should be on a per handler =
basis (probably via a new Component) -- that let's Solr administrators say =
things like:
  =

-    * "this is the regex pattern to be used when processing fieldname real=
ated params"
+  * "this is the regex pattern to be used when processing fieldname realat=
ed params"
-       * "fl" becomes a multivalued field
+   * "fl" becomes a multivalued field
-       * default: "|, "
+   * default: "|, "
-       * if not specificed, then fieldname params like fl and facet.field =
are taken literally (what do do about"sort" ?)
+   * if not specificed, then fieldname params like fl and facet.field are =
taken literally (what do do about"sort" ?)
-    * "for this param, alias this string to this real field"
+  * "for this param, alias this string to this real field"
-       * ie: "sort=3Dname+asc" ultimately sorts on "name_sortable"
+   * ie: "sort=3Dname+asc" ultimately sorts on "name_sortable"
-    * "for this param, alias this string to the documents score"
+  * "for this param, alias this string to the documents score"
-       * defaults to "score" for "fl" and "sort"
+   * defaults to "score" for "fl" and "sort"
-       * ie: "fl=3Dname,importance" ... importance might be the score fiel=
ds
+   * ie: "fl=3Dname,importance" ... importance might be the score fields
-    * "for this param, take any string that looks like a fieldname and app=
end/prepend this string to it"
+  * "for this param, take any string that looks like a fieldname and appen=
d/prepend this string to it"
-       * ie: any field name specified in the sort param can have "_sort" a=
ppended to it.
+   * ie: any field name specified in the sort param can have "_sort" appen=
ded to it.
-    * "for this param, alias this string to a regex or glob"
+  * "for this param, alias this string to a regex or glob"
-       * ie: "fl=3DstockFields&fl=3DpriceFields&facet.field=3DcatFields" m=
ight mean return all fields matching two configured regexes and facet on al=
l fields related to categorization.
+   * ie: "fl=3DstockFields&fl=3DpriceFields&facet.field=3DcatFields" might=
 mean return all fields matching two configured regexes and facet on all fi=
elds related to categorization.
-    * "allow users to specify globs for this param" or "allow users to spe=
cify regexes for this param"
+  * "allow users to specify globs for this param" or "allow users to speci=
fy regexes for this param"
-       * ie: if globing is turned on for facet.field, then "facet.field=3D=
facet_*" is legal
+   * ie: if globing is turned on for facet.field, then "facet.field=3Dface=
t_*" is legal
-       * ie: if regexes are turned on for "fl" then, then "fl=3Dname&fl=3D=
.*text" is legal
+   * ie: if regexes are turned on for "fl" then, then "fl=3Dname&fl=3D.*te=
xt" is legal
-    * "fields (not)-matching this glob or regex pattern are to be treated =
as if they didn't exist when using dealing with this param"
+  * "fields (not)-matching this glob or regex pattern are to be treated as=
 if they didn't exist when using dealing with this param"
-       * allows fields to be hidden in various contexts, even if the user =
guesses/knows they exist
+   * allows fields to be hidden in various contexts, even if the user gues=
ses/knows they exist
-       * ie: only allow the "sort" param to contain something matching the=
 glob "*_sort"
+   * ie: only allow the "sort" param to contain something matching the glo=
b "*_sort"
-       * ie: only return fields matching the regex "name|.*price.*|(short|=
long)summary" ... even if the users uses a glob "fl" param (return the inte=
rsection of fields matching the regex and the glob)
+   * ie: only return fields matching the regex "name|.*price.*|(short|long=
)summary" ... even if the users uses a glob "fl" param (return the intersec=
tion of fields matching the regex and the glob)
-    * "for this param, ignore field names that aren't recognized or allowe=
d by the configured rules"
+  * "for this param, ignore field names that aren't recognized or allowed =
by the configured rules"
-    * "for this param, error if a field name isn't recognized or allowed b=
y the configured rules"
+  * "for this param, error if a field name isn't recognized or allowed by =
the configured rules"
  =

  ...all of these things should be combinable in an order specified by the =
solr admin, they can say things like "when dealing with the facet.field par=
am, let users specify regexes to identify the fields to facet on, and map t=
he string "price" to the field "price_dollars_facet" but ultimately ignore =
any field that doesn't match the glob "*_facet"
  =

  =3D Implementation =3D
- =

  The best way to do this may be to have a Component which can be configure=
d with all of these rules (and reused by multiple handlers).  The component=
 would parse the input params, error if neccessary, and construct an object=
 put into the request context that subsequent Components can call methods o=
n to get field name Sets (or iterators) based on the param name being proce=
ssed, the schema, the rules defined, the context of operation (ie: dealing =
with stored fields, dealing with indexed indexed fields, a specific documen=
t for returned fields, etc...)
  =

- It should not be too difficult if one uses the "new" queryParser mechanis=
m from Lucene contrib. The Processor/Builder chain is suited for these chan=
ges.
+ It should not be too difficult if one uses the "new" queryParser mechanis=
m from Lucene contrib. The Processor/Builder chain is suited for these chan=
ges. All this aliasing can be configured by configfile or on a per request =
basis.
 =20