lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Male <gento...@gmail.com>
Subject Re: [ANNOUNCE] Apache Lucene 4.0 released.
Date Fri, 12 Oct 2012 08:58:30 GMT
A great day.

On Fri, Oct 12, 2012 at 9:34 PM, Uwe Schindler <uwe@thetaphi.de> wrote:

> Thanks Robert for doing the hard work of managing this release!
>

Absolutely.  Thanks Robert.


>
> I am happy that the release finally came out, after a long time of
> development, code refactoring, and lots of non-finite beer-automatons!
>
> Uwe
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
> > -----Original Message-----
> > From: Robert Muir [mailto:rmuir@apache.org]
> > Sent: Friday, October 12, 2012 10:10 AM
> > To: dev@lucene.apache.org; Lucene mailing list; java-user; announce
> > Subject: [ANNOUNCE] Apache Lucene 4.0 released.
> >
> > October 12 2012, Apache Luceneā€š 4.0 available.
> > The Lucene PMC is pleased to announce the release of Apache Lucene 4.0
> >
> > Apache Lucene is a high-performance, full-featured text search engine
> library
> > written entirely in Java. It is a technology suitable for nearly any
> application
> > that requires full-text search, especially cross-platform.
> >
> > This release contains numerous bug fixes, optimizations, and
> improvements,
> > some of which are highlighted below.  The release is available for
> immediate
> > download at:
> >    http://lucene.apache.org/core/mirrors-core-latest-redir.html
> >
> > See the CHANGES.txt file included with the release for a full list of
> details.
> >
> > Lucene 4.0 Release Highlights:
> >
> >  * The index formats for terms, postings lists, stored fields, term
> vectors, etc
> > are pluggable via the Codec api. You can select from the provided
> > implementations or customize the index format with your own Codec to meet
> > your needs.
> >
> >  * Similarity has been decoupled from the vector space model (TF/IDF).
> > Additional models such as BM25, Divergence from Randomness, Language
> > Models, and Information-based models are provided (see
> >
> http://www.lucidimagination.com/blog/2011/09/12/flexible-ranking-in-lucene-
> > 4).
> >
> >  * The new doc values feature stores typed values per-document.  It can
> be
> > used for custom scoring factors (accessible via Similarity), for
> pre-sorted Sort
> > values, and more.
> >
> >  * IndexWriter now flushes segments to disk concurrently, when the
> application
> > uses multiple threads for indexing, resulting in substantial performance
> > improvements (see http://blog.mikemccandless.com/2011/05/265-indexing-
> > speedup-with-lucenes.html).
> >
> >  * Per-document normalization factors ("norms") are no longer limited to
> a
> > single byte. Similarity implementations can use any DocValues type to
> store
> > norms.
> >
> >  * New index statistics have been added, including the number of tokens
> for a
> > term or field, number of postings for a field, and number of documents
> with a
> > posting for a field.  These support additional scoring models (see
> > http://blog.mikemccandless.com/2012/03/new-index-statistics-in-lucene-
> > 40.html).
> >
> >  * A new default term dictionary/index (BlockTree) indexes shared
> prefixes
> > instead of every n'th term. This is not only more time- and
> > space- efficient, but can avoid going to disk at all for terms that do
> not exist in
> > certain cases. Alternative term dictionary implementions are provided and
> > pluggable via the Codec api.
> >
> >  * Indexed terms are no longer limited to UTF-16 char sequences; they
> can now
> > be any binary value encoded as byte arrays. By default, text terms are
> encoded
> > as UTF-8 bytes. Sort order of terms is defined by their binary value,
> which is
> > identical to UTF-8 (Unicode code point) sort order.
> >
> >  * Substantially faster performance when using a Filter during searching.
> >
> >  * File-system based directories can rate-limit the IO (MB/sec) of merge
> > threads, to reduce IO contention between merging and searching threads.
> >
> >  * A number of alternative Codecs and components have been added:
> > "Appending" works with append-only filesystems (such as Hadoop DFS),
> > "Memory" writes the entire terms+postings as an FST read into RAM (see
> >
> http://blog.mikemccandless.com/2011/06/primary-key-lookups-are-28x-faster-
> > with.html),
> > "Pulsing" inlines the postings for low-frequency terms into the term
> dictionary
> > (see http://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-
> > primary-key.html),
> > "SimpleText" writes all files in plain-text for easy
> debugging/transparency (see
> > http://blog.mikemccandless.com/2010/10/lucenes-simpletext-codec.html),
> > "Bloom" uses a bloom filter to sometimes avoid disk seeks when looking up
> > terms, "Direct" holds all postings as simple byte[] and int[] for very
> fast
> > performance at the cost of very high RAM consumption, "Block" use a new
> > index layout and compression scheme for improved performance, among
> > others.
> >
> >  * Term offsets can be optionally encoded into the postings lists and
> retrieved
> > per-position.
> >
> >  * A new AutomatonQuery returns all documents containing any term
> matching
> > a provided finite-state automaton (see
> http://www.slideshare.net/otisg/finite-
> > state-queries-in-lucene).
> >
> >  * FuzzyQuery is 100-200 times faster than in past releases (see
> > http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-
> > faster.html).
> >
> >  * A new spell checker, DirectSpellChecker, finds possible corrections
> directly
> > against the main search index without requiring a separate index.
> >
> >  * Various in-memory data structures such as the term dictionary and
> > FieldCache are represented more efficiently with less object overhead
> (see
> > http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-
> > searching.html).
> >
> >  * All search logic is now required to work per segment, IndexReader was
> > therefore refactored to differentiate between atomic and composite
> readers
> > (see
> http://blog.thetaphi.de/2012/02/is-your-indexreader-atomic-major.html).
> >
> >  * Lucene 4.0 provides a modular API, consolidating components such as
> > Analyzers and Queries that were previously scattered across Lucene core,
> > contrib, and Solr. These modules also include additional functionality
> such as
> > UIMA analyzer integration and a completely reworked spatial search
> > implementation.
> >
> > Noteworthy changes since 4.0-BETA:
> >
> >  * A new "Block" PostingsFormat offering improved search performance and
> > index compression. This will likely become the default format in a future
> > release. (see http://blog.mikemccandless.com/2012/08/lucenes-new-
> > blockpostingsformat-thanks.html).
> >
> >  * All non-default codec implementations were moved to a separated codecs
> > module. Just add lucene-codecs-4.0.0.jar to your classpath to test these
> out.
> >
> >  * Payloads can be optionally stored on the term vectors.
> >
> >  * Many bugfixes and optimizations.
> >
> > Please read CHANGES.txt and MIGRATE.txt for a full list of new features
> and
> > notes on upgrading. Particularly, the new apis are not compatible with
> previous
> > versions of Lucene, however, file format backwards compatibility is
> provided
> > for indexes from the 3.0 series and the 4.0-alpha and -beta releases.
> >
> > Please report any feedback to the mailing lists
> > (http://lucene.apache.org/core/discussion.html)
> >
> > Note: The Apache Software Foundation uses an extensive mirroring network
> for
> > distributing releases.  It is possible that the mirror you are using may
> not have
> > replicated the release yet.  If that is the case, please try another
> mirror.  This
> > also goes for Maven access.
> >
> > Happy searching,
> >
> > Apache Lucene/Solr Developers
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>


-- 
Chris Male | Open Source Search Developer | elasticsearch |
www.e<http://www.dutchworks.nl>
lasticsearch.com

Mime
View raw message