Return-Path: 
 <solr-dev-return-2714-apmail-lucene-solr-dev-archive=lucene.apache.org@lucene.apache.org>
Delivered-To: apmail-lucene-solr-dev-archive@locus.apache.org
Received: (qmail 35746 invoked from network); 1 Feb 2007 21:09:18 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2)
  by minotaur.apache.org with SMTP; 1 Feb 2007 21:09:18 -0000
Received: (qmail 68041 invoked by uid 500); 1 Feb 2007 21:09:24 -0000
Delivered-To: apmail-lucene-solr-dev-archive@lucene.apache.org
Received: (qmail 67903 invoked by uid 500); 1 Feb 2007 21:09:24 -0000
Mailing-List: contact solr-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:solr-dev-help@lucene.apache.org>
List-Unsubscribe: <mailto:solr-dev-unsubscribe@lucene.apache.org>
List-Post: <mailto:solr-dev@lucene.apache.org>
List-Id: <solr-dev.lucene.apache.org>
Reply-To: solr-dev@lucene.apache.org
Delivered-To: mailing list solr-dev@lucene.apache.org
Received: (qmail 67894 invoked by uid 99); 1 Feb 2007 21:09:24 -0000
Received: from herse.apache.org (HELO herse.apache.org) (140.211.11.133)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Feb 2007 13:09:24 -0800
X-ASF-Spam-Status: No, hits=-0.0 required=10.0
	tests=SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (herse.apache.org: domain of yseeley@gmail.com designates
 64.233.182.186 as permitted sender)
Received: from [64.233.182.186] (HELO nf-out-0910.google.com) (64.233.182.186)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Feb 2007 13:09:15 -0800
Received: by nf-out-0910.google.com with SMTP id a27so947552nfc
        for <solr-dev@lucene.apache.org>;
 Thu, 01 Feb 2007 13:08:53 -0800 (PST)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:sender:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth;
        b=gSyxF29BLZnrerENnHXLU6cRjEYUaAMbU+GawNNqcyUThORsj5MEddRjlb3agDK5V9Mz7UGoySYctnGzBLqkp4VDxsQSqPOL5LwgjqZfs16R5xcJgjGI6FhA7iXclWtt4y27s5ucbxxr3W6ro/E3vohg86SSMUSqIqIPzfksq/4=
Received: by 10.82.107.15 with SMTP id f15mr855272buc.1170364133229;
        Thu, 01 Feb 2007 13:08:53 -0800 (PST)
Received: by 10.82.126.8 with HTTP; Thu, 1 Feb 2007 13:08:53 -0800 (PST)
Message-ID: <c68e39170702011308h5d06dc64we6df86866d801c86@mail.gmail.com>
Date: Thu, 1 Feb 2007 16:08:53 -0500
From: "Yonik Seeley" <yonik@apache.org>
Sender: yseeley@gmail.com
To: solr-dev@lucene.apache.org
Subject: Re: loading many documents by ID
In-Reply-To: <176776ee0702011246r2960e12ewdcb7e59ad91086a5@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <Pine.LNX.4.58.0701311528510.16454@hal.rescomp.berkeley.edu>
	 <C1E6B39E.4C0E%wunderwood@netflix.com>
	 <176776ee0701312105k3b33db02n7398c5d3039d1a22@mail.gmail.com>
	 <8A5B8E8F-B717-4086-831F-8617E2C11494@ehatchersolutions.com>
	 <176776ee0702011055j47a0126ck3f27201a87fde18@mail.gmail.com>
	 <c68e39170702011118i7858c11ck258dcb9ad93b2851@mail.gmail.com>
	 <176776ee0702011246r2960e12ewdcb7e59ad91086a5@mail.gmail.com>
X-Google-Sender-Auth: 9ac35ed43fd4dbe2
X-Virus-Checked: Checked by ClamAV on apache.org

On 2/1/07, Ryan McKinley <ryantxu@gmail.com> wrote:
> I am (was?) using DISTINCT to say, only add the unique fields.  As
> implemented, it keeps a Collection<String> for each field name.  If
> the 'mode' is 'DISTINCT' the collection is Set<String>, otherwise
> List<String>

Ah, OK... that does seem useful.

> How would you feel about an interface like this:

Not sure... depends on how update handlers will use it...
One thing we might not want to get rid of though is streaming
(constructing and adding a document, then discarding it).  People are
starting to add a lot of documents in a single XML request, and this
will be much larger for CVS/SQL.

For that reason, I'm not sure how often the "Collection" part will be utilized.

I like the it OK on the conceptual level though.

-Yonik

> public class IndexDocumentsCommand
> {
>   public enum MODE {
>     APPEND,    // add the fields to existing fields
>     OVERWRITE, // overwrite existing fields
>     INCREMENT, // increment existing field
>     DISTINCT   // same as APPEND, but make sure there are distinct values
>   };
>
>   // optional id in "internal" indexed form... if it is needed and not supplied,
>   // it will be obtained from the doc.
>   public String indexedId;
>
>   public Collection<SolrDocument> docs;
>   public boolean allowDups;
>   public boolean overwrite;
>   public SimpleOrderedMap<MODE> modifyFieldMode; // What to do for
> each field.  We should support *
>   public int commitMaxTime = -1; // make sure these documents are
> committed within this much time
> }