Mailing-List: contact solr-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-dev@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: domain of vengroff@gmail.com designates
 64.233.184.228 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:from:to:subject:date:mime-version:content-type:content-transfer-encoding:x-mailer:in-reply-to:x-mimeole:thread-index:message-id;
        b=aWzOPXulaE9Cd0yhJdlbdSSNMJCKLcHE1tUx8ytW4/f+wVVVD6Fdlqkw5KtmGWLqzla3TbDiUTNzxEbHSjQG49Gx8VBT7sauj1N+phiSh/uFlGglC7LfMdv03nfZZjHAtTxo3QF1YZHkZEnElthx79dtcx7AiI3bR98g2iR1IIs=
From: "Darren Vengroff" <vengroff@gmail.com>
To: <solr-dev@lucene.apache.org>
Subject: RE: dynamicField + copyField = dynamicCopyField?
Date: Tue, 6 Jun 2006 20:04:45 -0700
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit
In-Reply-To: <c68e39170606061825r29f14506le7a5bb0689df6976@mail.gmail.com>
Thread-Index: AcaJ0VmJb5Y4gfvLRZWVlrbpGDVqrQADb5Eg
Message-ID: <44864253.00c4b18e.49fa.fffffed4@mx.gmail.com>

Sounds good, I will check it out.

-D

-----Original Message-----
From: Yonik Seeley [mailto:yseeley@gmail.com] 
Sent: Tuesday, June 06, 2006 6:26 PM
To: solr-dev@lucene.apache.org
Subject: Re: dynamicField + copyField = dynamicCopyField?

Darren, I'm not sure how familiar you are with Lucene, but if you are
using dynamicFields (or a lot of indexed fields), check out the
omitNorms attribute.

Even if your indexed field values are sparse (contained only on a few
documents), Lucene keeps a 1 byte norm for each document in the index
for each indexed field by default.  That can really add up depending
on what you are doing.

I've considered making it the default for non-text fields in the
schema, but I'm worried that some people might try index-time boosts,
and they won't work w/o norms.

-Yonik

On 6/5/06, Darren Vengroff <vengroff@gmail.com> wrote:
> Dear Solrians,
>
> I've been looking through the IndexSchema and DocumentBuilder code, hoping
> to find something that is essentially a combination of what dynamicField
and
> copyField do.  Specifically, I'd like to say that any field that matches a
> given pattern should be copied to a specific multivalued destination
field.
> For example, my config might contain something like:
>
>     <!-- All text fields get thrown in here for default indexing. -->
>     <field name="text" type="text" indexed="true" stored="false"
> multiValued="true"/>
>
>     <!-- Search text by default so we get all text fields. -->
>     <defaultSearchField>text</defaultSearchField>
>
>     <!--
>         Any field ending in _t is assumed to be a text field and is thrown
> into text,
>         despite the fact that we may not have known of its specific
> existence when
>         we created the config file.
>     -->
>     <dynamicCopyField source="*_t" dest="text" />
>
> The advantage of this approach over pure copyField is that we don't have
to
> know the full set of text fields up front when we are writing the config
> file.  If a field name matches both a dynamicField and a dynamicCopyField,
> then both behaviors should probably occur, although config files will not
> typically be written this way.
>
> Has anything like this been discussed or implemented before?
>
> Looking forward to your feedback.
>
> Thanks,
>
> -D
>
>
>


-- 
-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server