lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (JIRA)" <j...@apache.org>
Subject [jira] Commented: (SOLR-217) schema option to ignore unused fields
Date Fri, 27 Apr 2007 18:15:15 GMT

    [ https://issues.apache.org/jira/browse/SOLR-217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12492357
] 

Hoss Man commented on SOLR-217:
-------------------------------

whatever mechanism we may add for supporting something like this, the default if unspecified
should definitely be an error ... if Solr is asked to index data it doesn't know what to do
with it should complain, rather then silently ignoring it ... this will help people with typos
in their schema or indexing code find their problems faster.

As for the proposed solutions: my initial reaction to reading the comments so far was to agree
with Will: having an explicit true/false option makes it much cleraer to people reading the
schema what's going on ... but in thinking about the possible use cases I prefer yonik's approach:
leveraging the existing field/dynamcField syntax will allow people to not only say "any unknown
field should be ignored" but also "field XXXX should be ignored" and "any unknown field that
starts with S_* should be ignored"

(there's also the question as to hwat should happen if i did have a stored="true" dynamicField
of "*" and i set ignoreUnnamedFields="true")


For the example config, we might want to do something like this to make it more obvious what's
going on, and to serve as a recommended config style...

   <!-- since fields of this type are by default not stored or indexed, any data added
to 
         them will be ignored outright
     -->
   <fieldtype name="ignored" stored="false" indexed="false" class="solr.StrField" />
   ...
   <!-- ignore any fields that don't already match an existing field name or dynamic field
-->
   <dynamicField name="*" type="ignored" />



> schema option to ignore unused fields
> -------------------------------------
>
>                 Key: SOLR-217
>                 URL: https://issues.apache.org/jira/browse/SOLR-217
>             Project: Solr
>          Issue Type: Improvement
>          Components: update
>    Affects Versions: 1.2
>            Reporter: Will Johnson
>            Priority: Minor
>             Fix For: 1.2
>
>         Attachments: ignoreUnnamedFields.patch
>
>
> One thing that causes problems for me (and i assume others) is that Solr is schema-strict
in that unknown fields cause solr to throw exceptions and there is no way to relax this constraint.
 this can cause all sorts of serious problems if you have automated feeding applications that
do things like SELECT * FROM table1 or where you want to add other fields to the document
for processing purposes before sending them to solr but don't want to deal with 'cleanup'

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message