Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of erickerickson@gmail.com
 designates 209.85.214.179 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <085942DC39DB5B40AB3F0195C8E5DC8DEA96F9@USASHEXMB02.LYV.LiveNation.com>
References: 
 <085942DC39DB5B40AB3F0195C8E5DC8DEA95AB@USASHEXMB02.LYV.LiveNation.com>
	<CAN4YXvc8nkF2q9GvaKKa1eu8yC5oSdg09hSDt10Ec_EG9z36JA@mail.gmail.com>
	<085942DC39DB5B40AB3F0195C8E5DC8DEA96F9@USASHEXMB02.LYV.LiveNation.com>
Date: Fri, 2 May 2014 09:07:08 -0700
Message-ID: 
 <CAN4YXvdOaTSHLtO0BPVkhw7VgzV=Oh1bwqA7vym1Hxs32MZQkA@mail.gmail.com>
Subject: Re: RE : Shards don't return documents in same order
From: Erick Erickson <erickerickson@gmail.com>
To: solr-user@lucene.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Francois:

Yes, there are several means to examine the raw terms in the index.
> The admin/schema-browser page
> TermsComponent: https://cwiki.apache.org/confluence/display/solr/The+Term=
s+Component
> Luke

the  schema-browser is all set up for you, it's easiest. The
TermsComponent should be directly usable too, I believe it's
configured by default in solrconfig.xml Luke takes a bit of setup but
is a great tool.

Did you re-index from scratch on all shards? I presume your ordering
is still not the same on all shards... the order I'd expect would be:
mb20140410a
mb20140410anew
mb20140411a

Best,
Erick


On Thu, May 1, 2014 at 8:27 AM, Francois Perron
<Francois.Perron@ticketmaster.com> wrote:
> Hi Erick,
>
>   thank you for your response.  You are right, I changed alphaOnlySort to=
 keep lettres and numbers and to remove some acticles (a, an, the).
>
> This is the filetype definition :
>
>     <fieldType name=3D"alphaOnlySort" class=3D"solr.TextField" sortMissin=
gLast=3D"true" omitNorms=3D"true">
>       <analyzer>
>         <tokenizer class=3D"solr.KeywordTokenizerFactory"/>
>         <filter class=3D"solr.LowerCaseFilterFactory"/>
>         <filter class=3D"solr.TrimFilterFactory"/>
>         <filter class=3D"solr.PatternReplaceFilterFactory" replace=3D"all=
" replacement=3D"" pattern=3D"(\b(a|an|the)\b|[^a-z,0-9])"/>
>       </analyzer>
>     </fieldType>
>
>
> Then, I tested each name with admin ui on each server and this is the res=
ults :
>
> server1
>
> MB20140410A =3D mb20140410a
> MB20140411A =3D mb20140411a
> MB20140410A-New =3D mb20140410anew
>
> server2
>
> MB20140410A =3D mb20140410a
> MB20140411A =3D mb20140411a
> MB20140410A-New =3D mb20140410anew
>
> server3
>
> MB20140410A =3D mb20140410a
> MB20140411A =3D mb20140411a
> MB20140410A-New =3D mb20140410anew
>
> "Unfortunately", all results are identical so is there a mean to view dat=
a real indexed in these documents ?  Can be a problem with a particular ser=
ver ?  All configs are in zookeeper so all cores shouldhave the same config=
, right ?  Is there any way to force a replicat to resynchronize ?
>
> Regards,
>
> Francois.
>
> ________________________________________
> De : Erick Erickson [erickerickson@gmail.com]
> Envoy=C3=A9 : 30 avril 2014 16:36
> =C3=80 : solr-user@lucene.apache.org
> Objet : Re: Shards don't return documents in same order
>
> Hmmm, take a look at the admin/analysis page for these inputs for
> alphaOnlySort. If you're using the stock Solr distro, you're probably
> not considering the effects patternReplaceFilterFactory which is
> removing all non-letters. So these three terms reduce to
>
> mba
> mba
> mbanew
>
> You can look at the actual indexed terms by the admin/schema-browser as w=
ell.
>
> That said, unless you transposed the order because you were
> concentrating on the numeric part, the doc with MB20140410A-New should
> always be sorting last.
>
> All of which is irrelevant if you're doing something else with
> "alphaOnlySort", so please paste in the fieldType definition if you've
> changed it.
>
> What gets returned in the doc for _stored_ data is a verbatim copy,
> NOT the output of the analysis chain, which can be confusing.
>
> Oh, and Solr uses the internal lucene doc ID to break ties, and docs
> on different replicas can have different internal Lucene doc IDs
> relative to each other as a result of merging so that's something else
> to watch out for.
>
> Best,
> Erick
>
> On Wed, Apr 30, 2014 at 1:06 PM, Francois Perron
> <Francois.Perron@ticketmaster.com> wrote:
>> Hi guys,
>>
>>   I have a small SolrCloud setup (3 servers, 1 collection with 1 shard a=
nd 3 replicat).  In my schema, I have a alphaOnlySort field with a copyfiel=
d.
>>
>> This is a part of my managed-schema :
>>
>>     <field name=3D"_root_" type=3D"string" indexed=3D"true" stored=3D"fa=
lse"/>
>>     <field name=3D"_uid" type=3D"string" multiValued=3D"false" indexed=
=3D"true" required=3D"true" stored=3D"true"/>
>>     <field name=3D"_version_" type=3D"long" indexed=3D"true" stored=3D"t=
rue"/>
>>     <field name=3D"event_id" type=3D"string" indexed=3D"true" stored=3D"=
true"/>
>>     <field name=3D"event_name" type=3D"text_general" indexed=3D"true" st=
ored=3D"true"/>
>>     <field name=3D"event_name_sort" type=3D"alphaOnlySort"/>
>>
>> with the copyfield
>>
>>   <copyField source=3D"event_name" dest=3D"event_name_sort"/>
>>
>>
>> The problem is : I query my collection with a sort on my alphasort field=
 but on one of my servers, the sort order is not the same.
>>
>> On server 1 and 2, I have this result :
>>
>> <doc>
>> <str name=3D"event_name">MB20140410A</str>
>> </doc>
>> <doc>
>> <str name=3D"event_name">MB20140410A-New</str>
>> </doc>
>> <doc>
>> <str name=3D"event_name">MB20140411A</str>
>> </doc>
>>
>>
>>
>> and on the third one, this :
>>
>> <str name=3D"event_name">MB20140410A</str>
>> </doc>
>> <doc>
>> <str name=3D"event_name">MB20140411A</str>
>> </doc>
>> <doc>
>> <str name=3D"event_name">MB20140410A-New</str>
>> </doc>
>>
>>
>> The doc named "MB20140411A" should be at the end ...
>>
>> Any idea ?
>>
>> Regards