Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Message-ID: <1418651850.2730.70.camel@te-prime>
Subject: Re: Determining NumericType for a field
From: Toke Eskildsen <te@statsbiblioteket.dk>
Reply-To: te@statsbiblioteket.dk
To: "dev@lucene.apache.org" <dev@lucene.apache.org>
CC: "david.w.smiley@gmail.com" <david.w.smiley@gmail.com>
Date: Mon, 15 Dec 2014 14:57:30 +0100
In-Reply-To: 
 <CABEwPvHQbGSUdS=DuQY63PQBFjViVjPKjDb_wmAm5Z9c_j=Wbw@mail.gmail.com>
References: <1418220744.4160.187.camel@te-prime>
	 <CAL8PwkawhqEWqDO3cKQZu94o30Tp0KtaohNxu+JWmNFv1NpviQ@mail.gmail.com>
	 <1418637225.2730.30.camel@te-prime>
	 <CAL8PwkYJim6dTud_vAT+OpEPKgkwo-wQ+BY65gcMnLraRU1dyQ@mail.gmail.com>
	 <1418648442.2730.53.camel@te-prime>
	 <CABEwPvHQbGSUdS=DuQY63PQBFjViVjPKjDb_wmAm5Z9c_j=Wbw@mail.gmail.com>
Organization: State and University Library, Denmark
Content-Type: text/plain; charset="UTF-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

On Mon, 2014-12-15 at 14:23 +0100, david.w.smiley@gmail.com wrote:

Toke:
>         Down to practicalities, we need Lucene 4.8 as our DocValues
>         are Disk
>         based and that support was removed in 4.9.

> I assume you’re referring to the “Disk” DV format/Codec?  The standard
> format has the data on disk too, it’s just that there’s some
> “small” (relative to the disk data) lookup references in heap/memory
> whereas the codec you’re using doesn’t.  Are you sure the standard
> codec isn’t sufficient?

As we have not tried anything else than "Disk" for our Net Archive
index, we have no comparison with "standard" (or whatever it is called).
We have no real preference and our next shards will be build with
"standard". Only reason for "Disk" is that it seemed like a good idea at
the time and now we have 20TB of index with it.

We would like to convert away from "Disk" too, but we would like to
avoid having to do a two-pass upgrade ("Disk" -> "standard" followed by
"non-DV" -> "DV"), so the DVEnabling code should preferably support
"Disk" for reading and do it all as single-pass.

>   If your use-case shows that there’s a need for the disk codec, I
> think it could be brought back, perhaps into the codecs module.

I think the removal of "Disk" during a minor version increase was not in
line with the backwards compatibility spirit of Solr. But I am sure it
was marked "Experimental" somewhere in the code and that the removal
obeyed the stated rules.

Anyway, done is done and as we have no future need for "Disk". But
thanks for the suggested fix.

>   You could copy the code too to use newer Lucene versions…

We looked at that sometime back and the code tentacles reached too far
for us to dare grapple with.

Regards,
Toke Eskildsen, State and University Library, Denmark


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org