Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: domain of yseeley@gmail.com designates
 64.233.166.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
        b=DmHqjvBRPlC8+NIWBTU40ST37KSZ+yCi/pobuOCqL5HEYjo+1nwD0uq8rUtnoszqX84jeVB1FKmT7SK+Cg2W656yYYtGtaEnHVU/ou3yR8cjkRdtFesJgCBXSte/rscG7pHX4NyOIUqIWA6Gnkig8eVqvrJqy9fgwVnSebCMeac=
Message-ID: <c68e39170607150851n4db12836wcf67e565fb2ff1c2@mail.gmail.com>
Date: Sat, 15 Jul 2006 11:51:55 -0400
From: "Yonik Seeley" <yseeley@gmail.com>
To: java-user@lucene.apache.org
Subject: Re: MissingStringLastComparatorSource and MultiSearcher
In-Reply-To: <20060715151024.2A27E187BE@mail.seseit.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <c68e39170607141359ta85cf24kfeb268e56d82d27c@mail.gmail.com>
	 <20060715151024.2A27E187BE@mail.seseit.com>

On 7/15/06, Rob Staveley (Tom) <rstaveley@seseit.com> wrote:
> Incidentally, Yonik, is the logic for
> org.apache.solr.search.Sorting#getStringSortField flawed? I don't see how
> the same comparator can be used for forward and reverse sorts for the
> non-null values.

The logic does cause a double-take at first glance :-)
It's the combination of the normal lucene string sort and
MissingStringLastComparatorSource that let you do all the
permutations.

Keep in mind that "reverse" in the Solr schema sense is not quite the
same as "reverse" to a Lucene SortField.

> Methinks it is broken for reverse sorts where you specify nulls to appear
> first.

Let's take this case specifically...

First the semantics:
   * @param nullFirst   true if null should come first, regardless of sort order

So if the sort is reversed, we want [null,null,Z,Y,...]

MissingStringLastComparatorSource orders null at the end in an
ascending sort.  So if the sort is completely reversed (in the lucene
sense), nulls will be at the beginning like we specified.

> I added this test to TestMissingStringLastComparatorSource.java from
> http://issues.apache.org/jira/browse/LUCENE-406, which confirms my
> suspicions ...or confirms that I'd never get a job in QA:

:-)

I haven't tried to run that test, but the non-null elements in your
test are ordered a bit funny for a reverse sort ;-)

-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server


> --------8<--------
> public void testSortingReverseNullFirst() throws Exception {
>
>         Hits result;
>
>         boolean reverse = true;
>         boolean nullLast = false;
>         boolean nullFirst = true;
>
>         Sort order = new Sort(Sorting.getStringSortField("data", reverse,
> nullLast, nullFirst));
>
>         result = s.search(ALL, order);
>
>         assertEquals("didn't get all", data.length, result.length());
>
>         assertEquals("wrong order 2", "0", result.doc(2).get("id"));
>         assertEquals("wrong order 3", "3", result.doc(3).get("id"));
>         assertEquals("wrong order 4", "6", result.doc(4).get("id"));
>         assertEquals("wrong order 5", "7", result.doc(5).get("id"));
>         assertEquals("wrong order 6", "4", result.doc(6).get("id"));
>         assertEquals("wrong order 7", "1", result.doc(7).get("id"));
> }
> --------8<--------
>
> -----Original Message-----
> From: Yonik Seeley [mailto:yseeley@gmail.com]
> Sent: 14 July 2006 21:59
> To: java-user@lucene.apache.org
> Subject: Re: MissingStringLastComparatorSource and MultiSearcher
>
> On 7/14/06, Rob Staveley (Tom) <rstaveley@seseit.com> wrote:
> > I was wanting to apply this to a field, which sorts on INT.
>
> The problem with int is that the FieldCache stores the values as an int[],
> and you can't tell when a value is missing.
>
> > Specifically I'm
> > trying to achieve reverse chronological sorting on a timestamp field,
> > which stores YYMMDDHHI (i.e. resolves to 10 minutes and doesn't handle
> centuries).
> > Missing timestamps are assumed to be "old" (i.e. should appear at the
> end).
> >
> > I could get this to sort on String and use
> > MissingStringLastComparatorSource, but would this not be less
> > efficient than sorting in INT??
>
> String sorting takes more memory, but the speed is the same.  Local sorting
> with the FieldCache for strings is done via the ordinal value (no string
> compare is done, just int comparisons).
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org