Return-Path: X-Original-To: apmail-lucene-commits-archive@www.apache.org Delivered-To: apmail-lucene-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8E73E17F19 for ; Wed, 27 May 2015 19:26:02 +0000 (UTC) Received: (qmail 89136 invoked by uid 500); 27 May 2015 19:26:02 -0000 Mailing-List: contact commits-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list commits@lucene.apache.org Received: (qmail 88894 invoked by uid 99); 27 May 2015 19:26:02 -0000 Received: from eris.apache.org (HELO hades.apache.org) (140.211.11.105) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 May 2015 19:26:02 +0000 Received: from hades.apache.org (localhost [127.0.0.1]) by hades.apache.org (ASF Mail Server at hades.apache.org) with ESMTP id E30FBAC0B9C for ; Wed, 27 May 2015 19:26:01 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: svn commit: r9151 [4/21] - in /dev/lucene/lucene-solr-5.2.0-RC1-rev1682085: ./ lucene/ lucene/changes/ lucene/maven/ lucene/maven/org/ lucene/maven/org/apache/ lucene/maven/org/apache/lucene/ lucene/maven/org/apache/lucene/lucene-analyzers-common/ luce... Date: Wed, 27 May 2015 19:25:50 -0000 To: commits@lucene.apache.org From: anshum@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20150527192601.E30FBAC0B9C@hades.apache.org> Added: dev/lucene/lucene-solr-5.2.0-RC1-rev1682085/lucene/changes/Changes.html ============================================================================== --- dev/lucene/lucene-solr-5.2.0-RC1-rev1682085/lucene/changes/Changes.html (added) +++ dev/lucene/lucene-solr-5.2.0-RC1-rev1682085/lucene/changes/Changes.html Wed May 27 19:25:47 2015 @@ -0,0 +1,12395 @@ + + + + Lucene Change Log + + + + + + + + +

Lucene Change Log

+ +
+ +

For more information on past and future Lucene versions, please see: +http://s.apache.org/luceneversions +

+

Release 5.2.0

+
    +
  • New Features   (16) +
      +
    1. LUCENE-6308, LUCENE-6385, LUCENE-6391: Span queries now share +document conjunction/intersection +code with boolean queries, and use two-phased iterators for +faster intersection by avoiding loading positions in certain cases. +
      (Paul Elschot, Terry Smith, Robert Muir via Mike McCandless)
      2. +
    2. LUCENE-6393: Add two-phase support to SpanPositionCheckQuery +and its subclasses: SpanPositionRangeQuery, SpanPayloadCheckQuery, +SpanNearPayloadCheckQuery, SpanFirstQuery. +
      (Paul Elschot, Robert Muir)
      3. +
    3. LUCENE-6394: Add two-phase support to SpanNotQuery and refactor +FilterSpans to just have an accept(Spans candidate) method for +subclasses. +
      (Robert Muir)
      4. +
    4. LUCENE-6373: SpanOrQuery shares disjunction logic with boolean +queries, and supports two-phased iterators to avoid loading +positions when possible. +
      (Paul Elschot via Robert Muir)
      5. +
    5. LUCENE-6352, LUCENE-6472: Added a new query time join to the join module +that uses global ordinals, which is faster for subsequent joins between +reopens. +
      (Martijn van Groningen, Adrien Grand)
      6. +
    6. LUCENE-5879: Added experimental auto-prefix terms to BlockTree terms +dictionary, exposed as AutoPrefixPostingsFormat +
      (Adrien Grand, +Uwe Schindler, Robert Muir, Mike McCandless)
      7. +
    7. LUCENE-5579: New CompositeSpatialStrategy combines speed of RPT with +accuracy of SDV. Includes optimized Intersect predicate to avoid many +geometry checks. Uses TwoPhaseIterator. +
      (David Smiley)
      8. +
    8. LUCENE-5989: Allow passing BytesRef to StringField to make it easier +to index arbitrary binary tokens, and change the experimental +StoredFieldVisitor.stringField API to take UTF-8 byte[] instead of +String +
      (Mike McCandless)
      9. +
    9. LUCENE-6389: Added ScoreMode.Min that aggregates the lowest child score +to the parent hit. +
      (Martijn van Groningen, Adrien Grand)
      10. +
    10. LUCENE-6423: New LimitTokenOffsetFilter that limits tokens to those before +a configured maximum start offset. +
      (David Smiley)
      11. +
    11. LUCENE-6422: New spatial PackedQuadPrefixTree, a generally more efficient +choice than QuadPrefixTree, especially for high precision shapes. +When used, you should typically disable RPT's pruneLeafyBranches option. +
      (Nick Knize, David Smiley)
      12. +
    12. LUCENE-6451: Expressions now support bindings keys that look like +zero arg functions +
      (Jack Conradson via Ryan Ernst)
      13. +
    13. LUCENE-6083: Add SpanWithinQuery and SpanContainingQuery that return +spans inside of / containing another spans. +
      (Paul Elschot via Robert Muir)
      14. +
    14. LUCENE-6454: Added distinction between member variable and method in +expression helper VariableContext +
      (Jack Conradson via Ryan Ernst)
      15. +
    15. LUCENE-6196: New Spatial "Geo3d" API with partial Spatial4j integration. +It is a set of shapes implemented using 3D planar geometry for calculating +spatial relations on the surface of a sphere. Shapes include Point, BBox, +Circle, Path (buffered line string), and Polygon. +
      (Karl Wright via David Smiley)
      16. +
    16. LUCENE-6464: Add a new expert lookup method to +AnalyzingInfixSuggester to accept an arbitrary BooleanQuery to +express how contexts should be filtered. +
      (Arcadius Ahouansou via +Mike McCandless)
      17. +
    +
    • +
  • Optimizations   (10) +
      +
    1. LUCENE-6379: IndexWriter.deleteDocuments(Query...) now detects if +one of the queries is MatchAllDocsQuery and just invokes the much +faster IndexWriter.deleteAll in that case +
      (Robert Muir, Adrien +Grand, Mike McCandless)
      2. +
    2. LUCENE-6388: Optimize SpanNearQuery when payloads are not present. +
      (Robert Muir)
      3. +
    3. LUCENE-6421: Defer reading of positions in MultiPhraseQuery until +they are needed. +
      (Robert Muir)
      4. +
    4. LUCENE-6392: Highligher- reduce memory of tokens in +TokenStreamFromTermVector, and add maxStartOffset limit. +
      (David Smiley)
      5. +
    5. LUCENE-6456: Queries that generate doc id sets that are too large for the +query cache are not cached instead of evicting everything. +
      (Adrien Grand)
      6. +
    6. LUCENE-6455: Require a minimum index size to enable query caching in order +not to cache eg. on MemoryIndex. +
      (Adrien Grand)
      7. +
    7. LUCENE-6330: BooleanScorer (used for top-level disjunctions) does not decode +norms when not necessary anymore. +
      (Adrien Grand)
      8. +
    8. LUCENE-6350: TermsQuery is now compressed with PrefixCodedTerms. +
      (Robert Muir, Mike McCandless, Adrien Grand)
      9. +
    9. LUCENE-6458: Multi-term queries matching few terms per segment now execute +like a disjunction. +
      (Adrien Grand)
      10. +
    10. LUCENE-6360: TermsQuery rewrites to a disjunction when there are 16 matching +terms or less. +
      (Adrien Grand)
      11. +
    +
    • +
  • Bug Fixes   (14) +
      +
    1. LUCENE-329: Fix FuzzyQuery defaults to rank exact matches highest. +
      (Mark Harwood, Adrien Grand)
      2. +
    2. LUCENE-6378: Fix all RuntimeExceptions to throw the underlying root cause. +
      (Varun Thacker, Adrien Grand, Mike McCandless)
      3. +
    3. LUCENE-6415: TermsQuery.extractTerms is a no-op (used to throw an +UnsupportedOperationException). +
      (Adrien Grand)
      4. +
    4. LUCENE-6416: BooleanQuery.extractTerms now only extracts terms from scoring +clauses. +
      (Adrien Grand)
      5. +
    5. LUCENE-6409: Fixed integer overflow in LongBitSet.ensureCapacity. +
      (Luc Vanlerberghe via Adrien Grand)
      6. +
    6. LUCENE-6424, LUCENE-6430: Fix many bugs with mockfs filesystems in the +test-framework: always consistently wrap Path, fix buggy behavior for +globs, implement equals/hashcode for filtered Paths, etc. +
      (Ryan Ernst, Simon Willnauer, Robert Muir)
      7. +
    7. LUCENE-6426: Fix FieldType's copy constructor to also copy over the numeric +precision step. +
      (Adrien Grand)
      8. +
    8. LUCENE-6345: Null check terms/fields in Lucene queries +
      (Lee +Hinman via Mike McCandless)
      9. +
    9. LUCENE-6400: SolrSynonymParser should preserve original token instead +of replacing it with a synonym, when expand=true and there is no +explicit mapping +
      (Ian Ribas, Robert Muir, Mike McCandless)
      10. +
    10. LUCENE-6449: Don't throw NullPointerException if some segments are +missing the field being highlighted, in PostingsHighlighter +
      (Roman +Khmelichek via Mike McCandless)
      11. +
    11. LUCENE-6427: Added assertion about the presence of ghost bits in +(Fixed|Long)BitSet. +
      (Luc Vanlerberghe via Adrien Grand)
      12. +
    12. LUCENE-6468: Fixed NPE with empty Kuromoji user dictionary. +
      (Jun Ohtani via Christian Moen)
      13. +
    13. LUCENE-6483: Ensure core closed listeners are called on the same cache key as +the reader which has been used to register the listener. +
      (Adrien Grand)
      14. +
    14. LUCENE-6486 DocumentDictionary iterator no longer skips +documents with no payloads and now returns an empty BytesRef instead +
      (Marius Grama via Michael McCandless)
      15. +
    +
    • +
  • API Changes   (7) +
      +
    1. LUCENE-6377: SearcherFactory#newSearcher now accepts the previous reader +to simplify warming logic during opening new searchers. +
      (Simon Willnauer)
      2. +
    2. LUCENE-6410: Removed unused "reuse" parameter to +Terms.iterator. +
      (Robert Muir, Mike McCandless)
      3. +
    3. LUCENE-6425: Replaced Query.extractTerms with Weight.extractTerms. +
      (Adrien Grand)
      4. +
    4. LUCENE-6446: Simplified Explanation API. +
      (Adrien Grand)
      5. +
    5. LUCENE-6445: Two new methods in Highlighter's TokenSources; the existing +methods are now marked deprecated. +
      (David Smiley)
      6. +
    6. LUCENE-6484: Removed EliasFanoDocIdSet, which was unused. +
      (Paul Elschot via Adrien Grand)
      7. +
    7. LUCENE-6497: Allow subclasses of FieldType to check frozen state +
      (Ryan Ernst)
      8. +
    +
    • +
  • Other   (6) +
      +
    1. LUCENE-6413: Test runner should report the number of suites completed/ +remaining. +
      (Dawid Weiss)
      2. +
    2. LUCENE-5439: Add 'ant jacoco' build target. +
      (Robert Muir)
      3. +
    3. LUCENE-6315: Simplify the private iterator Lucene uses internally +when resolving deleted terms to matched docids. +
      (Robert Muir, Adrien +Grand, Mike McCandless)
      4. +
    4. LUCENE-6399: Benchmark module's QueryMaker.resetInputs should call setConfig +so queries can react to property changes in new rounds. +
      (David Smiley)
      5. +
    5. LUCENE-6382: Lucene now enforces that positions never exceed the +maximum value IndexWriter.MAX_POSITION. +
      (Robert Muir, Mike McCandless)
      6. +
    6. LUCENE-6372: Simplified and improved equals/hashcode of span queries. +
      (Paul Elschot via Adrien Grand)
      7. +
    +
    • +
  • Build   (1) +
      +
    1. LUCENE-6420: Update forbiddenapis to v1.8 +
      (Uwe Schindler)
      2. +
    +
    • +
  • Test Framework   (2) +
      +
    1. LUCENE-6419: Added two-phase iteration assertions to AssertingQuery. +
      (Adrien Grand)
      2. +
    2. LUCENE-6437: Randomly set CPU core count and spins, derived from +test's master seed, used by ConcurrentMergeScheduler to set dynamic +defaults, for better test randomization and to help tests reproduce +
      (Robert Muir, Mike McCandless)
      3. +
    +
    • +
+

Release 5.1.0 [2015-04-14]

+
    +
  • New Features   (9) +
      +
    1. LUCENE-6066: Added DiversifiedTopDocsCollector to misc for collecting no more +than a given number of results under a choice of key. Introduces new remove +method to core's PriorityQueue. +
      (Mark Harwood)
      2. +
    2. LUCENE-6191: New spatial 2D heatmap faceting for PrefixTreeStrategy. +
      (David Smiley)
      3. +
    3. LUCENE-6227: Added BooleanClause.Occur.FILTER to filter documents without +participating in scoring (on the contrary to MUST). +
      (Adrien Grand)
      4. +
    4. LUCENE-6294: Added oal.search.CollectorManager to allow for parallelization +of the document collection process on IndexSearcher. +
      (Adrien Grand)
      5. +
    5. LUCENE-6303: Added filter caching baked into IndexSearcher, disabled by +default. +
      (Adrien Grand)
      6. +
    6. LUCENE-6304: Added a new MatchNoDocsQuery that matches no documents. +
      (Lee Hinman via Adrien Grand)
      7. +
    7. LUCENE-6341: Add a -fast option to CheckIndex. +
      (Robert Muir)
      8. +
    8. LUCENE-6355: IndexWriter's infoStream now also logs time to write FieldInfos +during merge +
      (Lee Hinman via Mike McCandless)
      9. +
    9. LUCENE-6339: Added Near-real time Document Suggester via custom postings format +
      (Areek Zillur, Mike McCandless, Simon Willnauer)
      10. +
    +
    • +
  • Bug Fixes   (11) +
      +
    1. LUCENE-6368: FST.save can truncate output (BufferedOutputStream may be closed +after the underlying stream). +
      (Ippei Matsushima via Dawid Weiss)
      2. +
    2. LUCENE-6249: StandardQueryParser doesn't support pure negative clauses. +
      (Dawid Weiss)
      3. +
    3. LUCENE-6190: Spatial pointsOnly flag on PrefixTreeStrategy shouldn't switch all predicates to +Intersects. +
      (David Smiley)
      4. +
    4. LUCENE-6242: Ram usage estimation was incorrect for SparseFixedBitSet when +object alignment was different from 8. +
      (Uwe Schindler, Adrien Grand)
      5. +
    5. LUCENE-6293: Fixed TimSorter bug. +
      (Adrien Grand)
      6. +
    6. LUCENE-6001: DrillSideways hits NullPointerException for certain +BooleanQuery searches. +
      (Dragan Jotannovic, jane chang via Mike +McCandless)
      7. +
    7. LUCENE-6311: Fix NIOFSDirectory and SimpleFSDirectory so that the +toString method of IndexInputs confess when they are from a compound +file. +
      (Robert Muir, Mike McCandless)
      8. +
    8. LUCENE-6381: Add defensive wait time limit in +DocumentsWriterStallControl to prevent hangs during indexing if we +miss a .notify/All somewhere +
      (Mike McCandless)
      9. +
    9. LUCENE-6386: Correct IndexWriter.forceMerge documentation to state +that up to 3X (X = current index size) spare disk space may be needed +to complete forceMerge(1). +
      (Robert Muir, Shai Erera, Mike McCandless)
      10. +
    10. LUCENE-6395: Seeking by term ordinal was failing to set the term's +bytes in MemoryIndex +
      (Mike McCandless)
      11. +
    11. LUCENE-6429: Removed the TermQuery(Term,int) constructor which could lead to +inconsistent term statistics. +
      (Adrien Grand, Robert Muir)
      12. +
    +
    • +
  • Optimizations   (16) +
      +
    1. LUCENE-6183, LUCENE-5647: Avoid recompressing stored fields +and term vectors when merging segments without deletions. +Lucene50Codec's BEST_COMPRESSION mode uses a higher deflate +level for more compact storage. +
      (Robert Muir)
      2. +
    2. LUCENE-6184: Make BooleanScorer only score windows that contain +matches. +
      (Adrien Grand)
      3. +
    3. LUCENE-6161: Speed up resolving of deleted terms to docIDs by doing +a combined merge sort between deleted terms and segment terms +instead of a separate merge sort for each segment. In delete-heavy +use cases this can be a sizable speedup. +
      (Mike McCandless)
      4. +
    4. LUCENE-6201: BooleanScorer can now deal with values of minShouldMatch that +are greater than one and is used when queries produce dense result sets. +
      (Adrien Grand)
      5. +
    5. LUCENE-6218: Don't decode frequencies or match all positions when scoring +is not needed. +
      (Robert Muir)
      6. +
    6. LUCENE-6233 Speed up CheckIndex when the index has term vectors +
      (Robert Muir, Mike McCandless)
      7. +
    7. LUCENE-6198: Added the TwoPhaseIterator API, exposed on scorers which +is for now only used on phrase queries and conjunctions in order to check +positions lazily if the phrase query is in a conjunction with other queries. +
      (Robert Muir, Adrien Grand, David Smiley)
      8. +
    8. LUCENE-6244, LUCENE-6251: All boolean queries but those that have a +minShouldMatch > 1 now either propagate or take advantage of the two-phase +iteration capabilities added in LUCENE-6198. +
      (Adrien Grand, Robert Muir)
      9. +
    9. LUCENE-6241: FSDirectory.listAll() doesnt filter out subdirectories anymore, +for faster performance. Subdirectories don't matter to Lucene. If you need to +filter out non-index files with some custom usage, you may want to look at +the IndexFileNames class. +
      (Robert Muir)
      10. +
    10. LUCENE-6262: ConstantScoreQuery does not wrap the inner weight anymore when +scores are not required. +
      (Adrien Grand)
      11. +
    11. LUCENE-6263: MultiCollector automatically caches scores when several +collectors need them. +
      (Adrien Grand)
      12. +
    12. LUCENE-6275: SloppyPhraseScorer now uses the same logic as ConjunctionScorer +in order to advance doc IDs, which takes advantage of the cost() API. +
      (Adrien Grand)
      13. +
    13. LUCENE-6290: QueryWrapperFilter propagates approximations and FilteredQuery +rewrites to a BooleanQuery when the filter is a QueryWrapperFilter in order +to leverage approximations. +
      (Adrien Grand)
      14. +
    14. LUCENE-6318: Reduce RAM usage of FieldInfos when there are many fields. +
      (Mike McCandless, Robert Muir)
      15. +
    15. LUCENE-6320: Speed up CheckIndex. +
      (Robert Muir)
      16. +
    16. LUCENE-4942: Optimized the encoding of PrefixTreeStrategy indexes for +non-point data: 33% smaller index, 68% faster indexing, and 44% faster +searching. YMMV +
      (David Smiley)
      17. +
    +
    • +
  • API Changes   (21) +
      +
    1. LUCENE-6204, LUCENE-6208: Simplify CompoundFormat: remove files() +and remove files parameter to write(). +
      (Robert Muir)
      2. +
    2. LUCENE-6217: Add IndexWriter.isOpen and getTragicException. +
      (Simon +Willnauer, Mike McCandless)
      3. +
    3. LUCENE-6218, LUCENE-6220: Add Collector.needsScores() and needsScores +parameter to Query.createWeight(). +
      (Robert Muir, Adrien Grand)
      4. +
    4. LUCENE-4524, LUCENE-6246, LUCENE-6256, LUCENE-6271: Merge DocsEnum and DocsAndPositionsEnum +into a single PostingsEnum iterator. TermsEnum.docs() and TermsEnum.docsAndPositions() +are replaced by TermsEnum.postings(). +
      (Alan Woodward, Simon Willnauer, Robert Muir, Ryan Ernst)
      5. +
    5. LUCENE-6222: Removed TermFilter, use a QueryWrapperFilter(TermQuery) +instead. This will be as efficient now that queries can opt out from +scoring. +
      (Adrien Grand)
      6. +
    6. LUCENE-6269: Removed BooleanFilter, use a QueryWrapperFilter(BooleanQuery) +instead. +
      (Adrien Grand)
      7. +
    7. LUCENE-6270: Replaced TermsFilter with TermsQuery, use a +QueryWrapperFilter(TermsQuery) instead. +
      (Adrien Grand)
      8. +
    8. LUCENE-6223: Move BooleanQuery.BooleanWeight to BooleanWeight. +
      (Robert Muir)
      9. +
    9. LUCENE-1518: Make Filter extend Query and return 0 as score. +
      (Uwe Schindler, Adrien Grand)
      10. +
    10. LUCENE-6245: Force Filter subclasses to implement toString API from Query. +
      (Ryan Ernst)
      11. +
    11. LUCENE-6268: Replace FieldValueFilter and DocValuesRangeFilter with equivalent +queries that support approximations. +
      (Adrien Grand)
      12. +
    12. LUCENE-6289: Replace DocValuesRangeFilter with DocValuesRangeQuery which +supports approximations. +
      (Adrien Grand)
      13. +
    13. LUCENE-6266: Remove unnecessary Directory params from SegmentInfo.toString, +SegmentInfos.files/toString, and SegmentCommitInfo.toString. +
      (Robert Muir)
      14. +
    14. LUCENE-6272: Scorer extends DocSetIdIterator rather than DocsEnum +
      (Alan +Woodward)
      15. +
    15. LUCENE-6281: Removed support for slow collations from lucene/sandbox. Better +performance would be achieved through CollationKeyAnalyzer or +ICUCollationKeyAnalyzer. +
      (Adrien Grand)
      16. +
    16. LUCENE-6286: Removed IndexSearcher methods that take a Filter object. +A BooleanQuery with a filter clause must be used instead. +
      (Adrien Grand)
      17. +
    17. LUCENE-6300: PrefixFilter, TermRangeFilter and NumericRangeFilter have been +removed. Use PrefixQuery, TermRangeQuery and NumericRangeQuery instead. +
      (Adrien Grand)
      18. +
    18. LUCENE-6303: Replaced FilterCache with QueryCache and CachingWrapperFilter +with CachingWrapperQuery. +
      (Adrien Grand)
      19. +
    19. LUCENE-6317: Deprecate DataOutput.writeStringSet and writeStringStringMap. +Use writeSetOfStrings/Maps instead. +
      (Mike McCandless, Robert Muir)
      20. +
    20. LUCENE-6307: Rename SegmentInfo.getDocCount -> .maxDoc, +SegmentInfos.totalDocCount -> .totalMaxDoc, MergeInfo.totalDocCount +
        +
      • +> .totalMaxDoc and MergePolicy.OneMerge.totalDocCount -> +
        • +
      +.totalMaxDoc +
      (Adrien Grand, Robert Muir, Mike McCandless)
      21. +
    21. LUCENE-6367: PrefixQuery now subclasses AutomatonQuery, removing the +specialized PrefixTermsEnum. +
      (Robert Muir, Mike McCandless)
      22. +
    +
    • +
  • Other   (6) +
      +
    1. LUCENE-6248: Remove unused odd constants from StandardSyntaxParser.jj +
      (Dawid Weiss)
      2. +
    2. LUCENE-6193: Collapse identical catch branches in try-catch statements. +
      (shalin)
      3. +
    3. LUCENE-6239: Removed RAMUsageEstimator's sun.misc.Unsafe calls. +
      (Robert Muir, Dawid Weiss, Uwe Schindler)
      4. +
    4. LUCENE-6292: Seed StringHelper better. +
      (Robert Muir)
      5. +
    5. LUCENE-6333: Refactored queries to delegate their equals and hashcode +impls to the super class. +
      (Lee Hinman via Adrien Grand)
      6. +
    6. LUCENE-6343: DefaultSimilarity javadocs had the wrong float value to +demonstrate precision of encoded norms +
      (András Péteri via Mike McCandless)
      7. +
    +
    • +
  • Changes in Runtime Behavior   (2) +
      +
    1. LUCENE-6255: PhraseQuery now ignores leading holes and requires that +positions are positive and added in order. +
      (Adrien Grand)
      2. +
    2. LUCENE-6298: SimpleQueryParser returns an empty query rather than +null, if e.g. the terms were all stopwords. +
      (Lee Hinman via Robert Muir)
      3. +
    +
    • +
+

Older Releases

+
    +

    Release 5.0.0 [2015-02-20]

    +
      +
    • New Features   (32) +
        +
      1. LUCENE-5945: All file handling converted to NIO.2 apis. +
        (Robert Muir)
        2. +
      2. LUCENE-5946: SimpleFSDirectory now uses Files.newByteChannel, for +portability with custom FileSystemProviders. If you want the old +non-interruptible behavior of RandomAccessFile, use RAFDirectory +in the misc/ module. +
        (Uwe Schindler, Robert Muir)
        3. +
      3. SOLR-3359: Added analyzer attribute/property to SynonymFilterFactory. +
        (Ryo Onodera via Koji Sekiguchi)
        4. +
      4. LUCENE-5648: Index and search date ranges, particularly multi-valued ones. It's +implemented in the spatial module as DateRangePrefixTree used with +NumberRangePrefixTreeStrategy. +
        (David Smiley)
        5. +
      5. LUCENE-5895: Lucene now stores a unique id per-segment and per-commit to aid +in accurate replication of index files +
        (Robert Muir, Mike McCandless)
        6. +
      6. LUCENE-5889: Add commit method to AnalyzingInfixSuggester, and allow just using .add +to build up the suggester. +
        (Varun Thacker via Mike McCandless)
        7. +
      7. LUCENE-5123: Add a "pull" option to the postings writing API, so +that a PostingsFormat now receives a Fields instance and it is +responsible for iterating through all fields, terms, documents and +positions. +
        (Robert Muir, Mike McCandless)
        8. +
      8. LUCENE-5268: Full cutover of all postings formats to the "pull" +FieldsConsumer API, removing PushFieldsConsumer. Added new +PushPostingsWriterBase for single-pass push of docs/positions to the +postings format. +
        (Mike McCandless)
        9. +
      9. LUCENE-5906: Use Files.delete everywhere instead of File.delete, so that +when things go wrong, you get a real exception message why. +
        (Uwe Schindler, Robert Muir)
        10. +
      10. LUCENE-5933: Added FilterSpans for easier wrapping of Spans instance. +
        (Shai Erera)
        11. +
      11. LUCENE-5925: Remove fallback logic from opening commits, instead use +Directory.renameFile so that in-progress commits are never visible. +
        (Robert Muir)
        12. +
      12. LUCENE-5820: SuggestStopFilter should have a factory. +
        (Varun Thacker via Steve Rowe)
        13. +
      13. LUCENE-5949: Add Accountable.getChildResources(). +
        (Robert Muir)
        14. +
      14. SOLR-5986: Added ExitableDirectoryReader that extends FilterDirectoryReader and enables +exiting requests that take too long to enumerate over terms. +
        (Anshum Gupta, Steve Rowe, +Robert Muir)
        15. +
      15. LUCENE-5911: Add MemoryIndex.freeze() to allow thread-safe searching over a +MemoryIndex. +
        (Alan Woodward, David Smiley, Robert Muir)
        16. +
      16. LUCENE-5969: Lucene 5.0 has a new index format with mismatched file detection, +improved exception handling, and indirect norms encoding for sparse fields. +
        (Mike McCandless, Ryan Ernst, Robert Muir)
        17. +
      17. LUCENE-6053: Add Serbian analyzer. +
        (Nikola Smolenski via Robert Muir, Mike McCandless)
        18. +
      18. LUCENE-4400: Add support for new NYSIIS Apache commons phonetic +codec +
        (Thomas Neidhart via Mike McCandless)
        19. +
      19. LUCENE-6059: Add Daitch-Mokotoff Soundex phonetic Apache commons +phonetic codec, and upgrade to Apache commons codec 1.10. +
        (Thomas +Neidhart via Mike McCandless)
        20. +
      20. LUCENE-6058: With the upgrade to Apache commons codec 1.10, the +experimental BeiderMorseFilter has changed its behavior, so any +index using it will need to be rebuilt. +
        (Thomas +Neidhart via Mike McCandless)
        21. +
      21. LUCENE-6050: Accept MUST and MUST_NOT (in addition to SHOULD) for +each context passed to Analyzing/BlendedInfixSuggester +
        (Arcadius +Ahouansou, jane chang via Mike McCandless)
        22. +
      22. LUCENE-5929: Also extract terms to highlight from block join +queries. +
        (Julie Tibshirani via Mike McCandless)
        23. +
      23. LUCENE-6063: Allow overriding whether/how ConcurrentMergeScheduler +stalls incoming threads when merges are falling behind +
        (Mike +McCandless)
        24. +
      24. LUCENE-5833: DocumentDictionary now enumerates each value separately +in a multi-valued field (not just the first value), so you can build +suggesters from multi-valued fields. +
        (Varun Thacker via Mike +McCandless)
        25. +
      25. LUCENE-6077: Added a filter cache. +
        (Adrien Grand, Robert Muir)
        26. +
      26. LUCENE-6088: TermsFilter implements Accountable. +
        (Adrien Grand)
        27. +
      27. LUCENE-6034: The default highlighter when used with QueryScorer will highlight payload-sensitive +queries provided that term vectors with positions, offsets, and payloads are present. This is the +only highlighter that can highlight such queries accurately. +
        (David Smiley)
        28. +
      28. LUCENE-5914: Add an option to Lucene50Codec to support either BEST_SPEED +or BEST_COMPRESSION for stored fields. +
        (Adrien Grand, Robert Muir)
        29. +
      29. LUCENE-6119: Add auto-IO-throttling to ConcurrentMergeScheduler, to +rate limit IO writes for each merge depending on incoming merge +rate. +
        (Mike McCandless)
        30. +
      30. LUCENE-6155: Add payload support to MemoryIndex. The default highlighter's +QueryScorer and WeighedSpanTermExtractor now have setUsePayloads(bool). +
        (David Smiley)
        31. +
      31. LUCENE-6166: Deletions (alone) can now trigger new merges. +
        (Mike McCandless)
        32. +
      32. LUCENE-6177: Add CustomAnalyzer that allows to configure analyzers +like you do in Solr's index schema. This class has a builder API to configure +Tokenizers, TokenFilters, and CharFilters based on their SPI names +and parameters as documented by the corresponding factories. +
        (Uwe Schindler)
        33. +
      +
      • +
    • Optimizations   (18) +
        +
      1. LUCENE-5960: Use a more efficient bitset, not a Set<Integer>, to +track visited states. +
        (Markus Heiden via Mike McCandless)
        2. +
      2. LUCENE-5959: Don't allocate excess memory when building automaton in +finish. +
        (Markus Heiden via Mike McCandless)
        3. +
      3. LUCENE-5963: Reduce memory allocations in +AnalyzingSuggester. +
        (Markus Heiden via Mike McCandless)
        4. +
      4. LUCENE-5938: MultiTermQuery.CONSTANT_SCORE_FILTER_REWRITE is now faster on +queries that match few documents by using a sparse bit set implementation. +
        (Adrien Grand)
        5. +
      5. LUCENE-5969: Refactor merging to be more efficient, checksum calculation is +per-segment/per-producer, and norms and doc values merging no longer cause +RAM spikes for latent fields. +
        (Mike McCandless, Robert Muir)
        6. +
      6. LUCENE-5983: CachingWrapperFilter now uses a new DocIdSet implementation +called RoaringDocIdSet instead of WAH8DocIdSet. +
        (Adrien Grand)
        7. +
      7. LUCENE-6022: DocValuesDocIdSet checks live docs before doc values. +
        (Adrien Grand)
        8. +
      8. LUCENE-6030: Add norms patched compression for a small number of common values +
        (Ryan Ernst)
        9. +
      9. LUCENE-6040: Speed up EliasFanoDocIdSet through broadword bit selection. +
        (Paul Elschot)
        10. +
      10. LUCENE-6033: CachingTokenFilter now uses ArrayList not LinkedList, and has new +isCached() method. +
        (David Smiley)
        11. +
      11. LUCENE-6031: TokenSources (in the default highlighter) converts term vectors into a +TokenStream much faster in linear time (not N*log(N) using less memory, and with reset() +implemented. Only one of offsets or positions are required of the term vector. +
        (David Smiley)
        12. +
      12. LUCENE-6089, LUCENE-6090: Tune CompressionMode.HIGH_COMPRESSION for +better compression and less cpu usage. +
        (Adrien Grand, Robert Muir)
        13. +
      13. LUCENE-6034: QueryScorer, used by the default highlighter, needn't re-index the provided +TokenStream with MemoryIndex when it comes from TokenSources (term vectors) with offsets and +positions. +
        (David Smiley)
        14. +
      14. LUCENE-5951: ConcurrentMergeScheduler detects whether the index is on SSD or not +and does a better job defaulting its settings. This only works on Linux for now; +other OS's will continue to use the previous defaults (tuned for spinning disks). +
        (Robert Muir, Uwe Schindler, hossman, Mike McCandless)
        15. +
      15. LUCENE-6131: Optimize SortingMergePolicy. +
        (Robert Muir)
        16. +
      16. LUCENE-6133: Improve default StoredFieldsWriter.merge() to be more efficient. +
        (Robert Muir)
        17. +
      17. LUCENE-6145: Make EarlyTerminatingSortingCollector able to early-terminate +when the sort order is a prefix of the index-time order. +
        (Adrien Grand)
        18. +
      18. LUCENE-6178: Score boolean queries containing MUST_NOT clauses with BooleanScorer2, +to use skip list data and avoid unnecessary scoring. +
        (Adrien Grand, Robert Muir)
        19. +
      +
      • +
    • API Changes   (40) +
        +
      1. LUCENE-5900: Deprecated more constructors taking Version in *InfixSuggester and +ICUCollationKeyAnalyzer, and removed TEST_VERSION_CURRENT from the test framework. +
        (Ryan Ernst)
        2. +
      2. LUCENE-4535: oal.util.FilterIterator is now an internal API. +
        (Adrien Grand)
        3. +
      3. LUCENE-4924: DocIdSetIterator.docID() must now return -1 when the iterator is +not positioned. This change affects all classes that inherit from +DocIdSetIterator, including DocsEnum and DocsAndPositionsEnum. +
        (Adrien Grand)
        4. +
      4. LUCENE-5127: Reduce RAM usage of FixedGapTermsIndex. Remove +IndexWriterConfig.setTermIndexInterval, IndexWriterConfig.setReaderTermsIndexDivisor, +and termsIndexDivisor from StandardDirectoryReader. These options have been no-ops +with the default codec since Lucene 4.0. If you want to configure the interval for +this term index, pass it directly in your codec, where it can also be configured +per-field. +
        (Robert Muir)
        5. +
      5. LUCENE-5388: Remove Reader from Tokenizer's constructor and from +Analyzer's createComponents. TokenStreams now always get their input +via setReader. +
        (Benson Margulies via Robert Muir - pull request #16)
        6. +
      6. LUCENE-5527: The Collector API has been refactored to use a dedicated Collector +per leaf. +
        (Shikhar Bhushan, Adrien Grand)
        7. +
      7. LUCENE-5702: The FieldComparator API has been refactor to a per-leaf API, just +like Collectors. +
        (Adrien Grand)
        8. +
      8. LUCENE-4246: IndexWriter.close now always closes, even if it throws +an exception. The new IndexWriterConfig.setCommitOnClose (default +true) determines whether close() should commit before closing. +

        +

        9. +
      9. LUCENE-5608, LUCENE-5565: Refactor SpatialPrefixTree/Cell API. Doesn't use Strings +as tokens anymore, and now iterates cells on-demand during indexing instead of +building a collection. RPT now has more setters. +
        (David Smiley)
        10. +
      10. LUCENE-5666: Change uninverted access (sorting, faceting, grouping, etc) +to use the DocValues API instead of FieldCache. For FieldCache functionality, +use UninvertingReader in lucene/misc (or implement your own FilterReader). +UninvertingReader is more efficient: supports multi-valued numeric fields, +detects when a multi-valued field is single-valued, reuses caches +of compatible types (e.g. SORTED also supports BINARY and SORTED_SET access +without insanity). "Insanity" is no longer possible unless you explicitly want it. +Rename FieldCache* and DocTermOrds* classes in the search package to DocValues*. +Move SortedSetSortField to core and add SortedSetFieldSource to queries/, which +takes the same selectors. Add helper methods to DocValues.java that are better +suited for search code (never return null, etc). +
        (Mike McCandless, Robert Muir)
        11. +
      11. LUCENE-5871: Remove Version from IndexWriterConfig. Use +IndexWriterConfig.setCommitOnClose to change the behavior of IndexWriter.close(). +The default has been changed to match that of 4.x. +
        (Ryan Ernst, Mike McCandless)
        12. +
      12. LUCENE-5965: CorruptIndexException requires a String or DataInput resource. +
        (Robert Muir)
        13. +
      13. LUCENE-5972: IndexFormatTooOldException and IndexFormatTooNewException now + extend from IOException. +
        (Ryan Ernst, Robert Muir)
        14. +
      14. LUCENE-5569: *AtomicReader/AtomicReaderContext have been renamed to *LeafReader/LeafReaderContext. +
        (Ryan Ernst)
        15. +
      15. LUCENE-5938: Removed MultiTermQuery.ConstantScoreAutoRewrite as +MultiTermQuery.CONSTANT_SCORE_FILTER_REWRITE is usually better. +
        (Adrien Grand)
        16. +
      16. LUCENE-5924: Rename CheckIndex -fix option to -exorcise. This option does not +actually fix the index, it just drops data. +
        (Robert Muir)
        17. +
      17. LUCENE-5969: Add Codec.compoundFormat, which handles the encoding of compound +files. Add getMergeInstance() to codec producer APIs, which can be overridden +to return an instance optimized for merging instead of searching. Add +Terms.getStats() which can return additional codec-specific statistics about a field. +Change instance method SegmentInfos.read() to two static methods: SegmentInfos.readCommit() +and SegmentInfos.readLatestCommit(). +
        (Mike McCandless, Robert Muir)
        18. +
      18. LUCENE-5992: Remove FieldInfos from SegmentInfosWriter.write API. +
        (Robert Muir, Mike McCandless)
        19. +
      19. LUCENE-5998: Simplify Field/SegmentInfoFormat to read+write methods. +
        (Robert Muir)
        20. +
      20. LUCENE-6000: Removed StandardTokenizerInterface. Tokenizers now use +their jflex impl directly. +
        (Ryan Ernst)
        21. +
      21. LUCENE-6006: Removed FieldInfo.normType since it's redundant: it +will be DocValuesType.NUMERIC if the field indexed and does not omit +norms, else null. +
        (Robert Muir, Mike McCandless)
        22. +
      22. LUCENE-6013: Removed indexed boolean from IndexableFieldType and +FieldInfo, since it's redundant with IndexOptions != null. +
        (Robert +Muir, Mike McCandless)
        23. +
      23. LUCENE-6021: FixedBitSet.nextSetBit now returns DocIdSetIterator.NO_MORE_DOCS +instead of -1 when there are no more bits which are set. +
        (Adrien Grand)
        24. +
      24. LUCENE-5953: Directory and LockFactory APIs were restructured: Locking is +now under the responsibility of the Directory implementation. LockFactory is +only used by subclasses of BaseDirectory to delegate locking to an impl +class. LockFactories are now singletons and are responsible to create a Lock +instance based on a Directory implementation passed to the factory method. +See MIGRATE.txt for more details. +
        (Uwe Schindler, Robert Muir)
        25. +
      25. LUCENE-6062: Throw exception instead of silently doing nothing if you try to +sort/group/etc on a misconfigured field (e.g. no docvalues, no UninvertingReader, etc). +
        (Robert Muir)
        26. +
      26. LUCENE-6068: LeafReader.fields() never returns null. +
        (Robert Muir)
        27. +
      27. LUCENE-6082: Remove abort() from codec apis. +
        (Robert Muir)
        28. +
      28. LUCENE-6084: IndexOutput's constructor now requires a String +resourceDescription so its toString is sane +
        (Robert Muir, Mike +McCandless)
        29. +
      29. LUCENE-6087: Allow passing custom DirectoryReader to SearcherManager +
        (Mike McCandless)
        30. +
      30. LUCENE-6085: Undeprecate SegmentInfo attributes, but add safety so they +won't be trappy if codec tries to use them during docvalues updates. +
        (Robert Muir)
        31. +
      31. LUCENE-6097: Remove dangerous / overly expert +IndexWriter.abortMerges and waitForMerges methods. +
        (Robert Muir, +Mike McCandless)
        32. +
      32. LUCENE-6099: Add FilterDirectory.unwrap and +FilterDirectoryReader.unwrap +
        (Simon Willnauer, Mike McCandless)
        33. +
      33. LUCENE-6121: CachingTokenFilter.reset() now propagates to its input if called before +incrementToken(). You must call reset() now on this filter instead of doing it a-priori on the +input(), which previously didn't work. +
        (David Smiley, Robert Muir)
        34. +
      34. LUCENE-6147: Make the core Accountables.namedAccountable function public +
        (Ryan Ernst)
        35. +
      35. LUCENE-6150: Remove staleFiles set and onIndexOutputClosed() from FSDirectory. +
        (Uwe Schindler, Robert Muir, Mike McCandless)
        36. +
      36. LUCENE-6146: Replaced Directory.copy() with Directory.copyFrom(). +
        (Robert Muir)
        37. +
      37. LUCENE-6149: Infix suggesters' highlighting and allTermsRequired can +be set at the constructor for non-contextual lookup. +
        (Boon Low, Tomás Fernández Löbbe)
        38. +
      38. LUCENE-6158, LUCENE-6165: IndexWriter.addIndexes(IndexReader...) changed to +addIndexes(CodecReader...) +
        (Robert Muir)
        39. +
      39. LUCENE-6179: Out-of-order scoring is not allowed anymore, so +Weight.scoresDocsOutOfOrder and LeafCollector.acceptsDocsOutOfOrder have been +removed and boolean queries now always score in order. +

        +

        40. +
      40. LUCENE-6212: IndexWriter no longer accepts per-document Analyzer to +add/updateDocument. These methods were trappy as they made it +easy to accidentally index tokens that were not easily +searchable. +
        (Mike McCandless)
        41. +
      +
      • +
    • Bug Fixes   (30) +
        +
      1. LUCENE-5650: Enforce read-only access to any path outside the temporary +folder via security manager, and make test temp dirs absolute. +
        (Ryan Ernst, Dawid Weiss)
        2. +
      2. LUCENE-5948: RateLimiter now fully inits itself on init. +
        (Varun +Thacker via Mike McCandless)
        3. +
      3. LUCENE-5981: CheckIndex obtains write.lock, since with some parameters it +may modify the index, and to prevent false corruption reports, as it does +not have the regular "spinlock" of DirectoryReader.open. It now implements +Closeable and you must close it to release the lock. +
        (Mike McCandless, Robert Muir)
        4. +
      4. LUCENE-6004: Don't highlight the LookupResult.key returned from +AnalyzingInfixSuggester +
        (Christian Reuschling, jane chang via Mike McCandless)
        5. +
      5. LUCENE-5980: Don't let document length overflow. +
        (Robert Muir)
        6. +
      6. LUCENE-5999: Fix backcompat support for StandardTokenizer +
        (Ryan Ernst)
        7. +
      7. LUCENE-5961: Fix the exists() method for FunctionValues returned by many ValueSources to +behave properly when wrapping other ValueSources which do not exist for the specified document +
        (hossman)
        8. +
      8. LUCENE-6039: Add IndexOptions.NONE and DocValuesType.NONE instead of +using null to mean not index and no doc values, renamed +IndexOptions.DOCS_ONLY to DOCS, and pulled IndexOptions and +DocValues out of FieldInfo into their own classes in +org.apache.lucene.index +
        (Simon Willnauer, Robert Muir, Mike +McCandless)
        9. +
      9. LUCENE-6043: Fix backcompat support for UAX29URLEmailTokenizer +
        (Ryan Ernst)
        10. +
      10. LUCENE-6041: Remove sugar methods FieldInfo.isIndexed and +FieldInfo.hasDocValues. +
        (Robert Muir, Mike McCandless)
        11. +
      11. LUCENE-6044: Fix backcompat support for token filters with enablePositionIncrements=false. +Also fixed backcompat for TrimFilter with updateOffsets=true. These options +are supported with a match version before 4.4, and no longer valid at all with 5.0. +
        (Ryan Ernst)
        12. +
      12. LUCENE-6042: CustomScoreQuery explain was incorrect in some cases, +such as when nested inside a boolean query. +
        (Denis Lantsman via Robert Muir)
        13. +
      13. LUCENE-6046: Add maxDeterminizedStates safety to determinize (which has +an exponential worst case) so that if it would create too many states, it +now throws an exception instead of exhausting CPU/RAM. +
        (Nik +Everett via Mike McCandless)
        14. +
      14. LUCENE-6054: Allow repeating the empty automaton +
        (Nik Everett via +Mike McCandless)
        15. +
      15. LUCENE-6049: Don't throw cryptic exception writing a segment when +the only docs in it had fields that hit non-aborting exceptions +during indexing but also had doc values. +
        (Mike McCandless)
        16. +
      16. LUCENE-6055: PayloadAttribute.clone() now does a deep clone of the underlying +bytes. +
        (Shai Erera)
        17. +
      17. LUCENE-6060: Remove dangerous IndexWriter.unlock method +
        (Simon +Willnauer, Mike McCandless)
        18. +
      18. LUCENE-6062: Pass correct fieldinfos to docvalues producer when the +segment has updates. +
        (Mike McCandless, Shai Erera, Robert Muir)
        19. +
      19. LUCENE-6075: Don't overflow int in SimpleRateLimiter +
        (Boaz Leskes +via Mike McCandless)
        20. +
      20. LUCENE-5987: IndexWriter will now forcefully close itself on +aborting exception (an exception that would otherwise cause silent +data loss). +
        (Robert Muir, Mike McCandless)
        21. +
      21. LUCENE-6094: Allow IW.rollback to stop ConcurrentMergeScheduler even +when it's stalling because there are too many merges. +
        (Mike McCandless)
        22. +
      22. LUCENE-6105: Don't cache FST root arcs if the number of root arcs is +small, or if the cache would be > 20% of the size of the FST. +
        (Robert Muir, Mike McCandless)
        23. +
      23. LUCENE-6124: Fix double-close() problems in codec and store APIs. +
        (Robert Muir)
        24. +
      24. LUCENE-6152: Fix double close problems in OutputStreamIndexOutput. +
        (Uwe Schindler)
        25. +
      25. LUCENE-6139: Highlighter: TokenGroup start & end offset getters should have +been returning the offsets of just the matching tokens in the group when +there's a distinction. +
        (David Smiley)
        26. +
      26. LUCENE-6173: NumericTermAttribute and spatial/CellTokenStream do not clone +their BytesRef(Builder)s. Also equals/hashCode was missing. +
        (Uwe Schindler)
        27. +
      27. LUCENE-6205: Fixed intermittent concurrency issue that could cause +FileNotFoundException when writing doc values updates at the same +time that a merge kicks off. +
        (Mike McCandless)
        28. +
      28. LUCENE-6192: Fix int overflow corruption case in skip data for +high frequency terms in extremely large indices +
        (Robert Muir, Mike +McCandless)
        29. +
      29. LUCENE-6093: Don't throw NullPointerException from +BlendedInfixSuggester for lookups that do not end in a prefix +token. +
        (jane chang via Mike McCandless)
        30. +
      30. LUCENE-6214: Fixed IndexWriter deadlock when one thread is +committing while another opens a near-real-time reader and an +unrecoverable (tragic) exception is hit. +
        (Simon Willnauer, Mike +McCandless)
        31. +
      +
      • +
    • Documentation   (3) +
        +
      1. LUCENE-5392: Add/improve analysis package documentation to reflect +analysis API changes. +
        (Benson Margulies via Robert Muir - pull request #17)
        2. +
      2. LUCENE-6057: Improve Sort(SortField) docs +
        (Martin Braun via Mike McCandless)
        3. +
      3. LUCENE-6112: Fix compile error in FST package example code +
        (Tomoko Uchida via Koji Sekiguchi)
        4. +
      +
      • +
    • Tests   (6) +
        +
      1. LUCENE-5957: Add option for tests to not randomize codec +
        (Ryan Ernst)
        2. +
      2. LUCENE-5974: Add check that backcompat indexes use default codecs +
        (Ryan Ernst)
        3. +
      3. LUCENE-5971: Create addBackcompatIndexes.py script to build and add +backcompat test indexes for a given lucene version. Also renamed backcompat +index files to use Version.toString() in filename. +
        (Ryan Ernst)
        4. +
      4. LUCENE-6002: Monster tests no longer fail. Most of them now have an 80 hour +timeout, effectively removing the timeout. The tests that operate near the 2 +billion limit now use IndexWriter.MAX_DOCS instead of Integer.MAX_VALUE. +Some of the slow Monster tests now explicitly choose the default codec. +
        (Mike McCandless, Shawn Heisey)
        5. +
      5. LUCENE-5968: Improve error message when 'ant beast' is run on top-level +modules. +
        (Ramkumar Aiyengar, Uwe Schindler)
        6. +
      6. LUCENE-6120: Fix MockDirectoryWrapper's close() handling. +
        (Mike McCandless, Robert Muir)
        7. +
      +
      • +
    • Build   (5) +
        +
      1. LUCENE-5909: Smoke tester now has better command line parsing and +optionally also runs on Java 8. +
        (Ryan Ernst, Uwe Schindler)
        2. +
      2. LUCENE-5902: Add bumpVersion.py script to manage version increase after release branch is cut. +

        +

        3. +
      3. LUCENE-5962: Rename diffSources.py to createPatch.py and make it work with all text file types. +
        (Ryan Ernst)
        4. +
      4. LUCENE-5995: Upgrade ICU to 54.1 +
        (Robert Muir)
        5. +
      5. LUCENE-6070: Upgrade forbidden-apis to 1.7 +
        (Uwe Schindler)
        6. +
      +
      • +
    • Other   (5) +
        +
      1. LUCENE-5563: Removed sep layout: which has fallen behind on features and doesn't +perform as well as other options. +
        (Robert Muir)
        2. +
      2. LUCENE-4086: Removed support for Lucene 3.x indexes. See migration guide for +more information. +
        (Robert Muir)
        3. +
      3. LUCENE-5858: Moved Lucene 4 compatibility codecs to 'lucene-backward-codecs.jar'. +
        (Adrien Grand, Robert Muir)
        4. +
      4. LUCENE-5915: Remove Pulsing postings format. +
        (Robert Muir)
        5. +
      5. LUCENE-6213: Add useful exception message when commit contains segments from legacy codecs. +
        (Ryan Ernst)
        6. +
      +
      • +
    +

    Release 4.10.4 [2015-03-03]

    +
      +
    • Bug fixes   (12) +
        +
      1. LUCENE-6019, LUCENE-6117: Remove -Dtests.assert to make IndexWriter +infoStream sane. +
        (Robert Muir, Mike McCandless)
        2. +
      2. LUCENE-6161: Resolving deletes was failing to reuse DocsEnum likely +causing substantial performance cost for use cases that frequently +delete old documents +
        (Mike McCandless)
        3. +
      3. LUCENE-6192: Fix int overflow corruption case in skip data for +high frequency terms in extremely large indices +
        (Robert Muir, Mike +McCandless)
        4. +
      4. LUCENE-6207: Fixed consumption of several terms enums on the same +sorted (set) doc values instance at the same time. +
        (Tom Shally, Robert Muir, Adrien Grand)
        5. +
      5. LUCENE-6093: Don't throw NullPointerException from +BlendedInfixSuggester for lookups that do not end in a prefix +token. +
        (jane chang via Mike McCandless)
        6. +
      6. LUCENE-6279: Don't let an abusive leftover _N_upgraded.si in the +index directory cause index corruption on upgrade +
        (Robert Muir, Mike +McCandless)
        7. +
      7. LUCENE-6287: Fix concurrency bug in IndexWriter that could cause +index corruption (missing _N.si files) the first time 4.x kisses a +3.x index if merges are also running. +
        (Simon Willnauer, Mike +McCandless)
        8. +
      8. LUCENE-6205: Fixed intermittent concurrency issue that could cause +FileNotFoundException when writing doc values updates at the same +time that a merge kicks off. +
        (Mike McCandless)
        9. +
      9. LUCENE-6214: Fixed IndexWriter deadlock when one thread is +committing while another opens a near-real-time reader and an +unrecoverable (tragic) exception is hit. +
        (Simon Willnauer, Mike +McCandless)
        10. +
      10. LUCENE-6105: Don't cache FST root arcs if the number of root arcs is +small, or if the cache would be > 20% of the size of the FST. +
        (Robert Muir, Mike McCandless)
        11. +
      11. LUCENE-6001: DrillSideways hits NullPointerException for certain +BooleanQuery searches. +
        (Dragan Jotannovic, jane chang via Mike +McCandless)
        12. +
      12. LUCENE-6306: Merging of doc values and norms now checks whether the +merge was aborted so IndexWriter.rollback can more promptly abort a +running merge. +
        (Robert Muir, Mike McCandless)
        13. +
      +
      • +
    • API Changes   (1) +
        +
      1. LUCENE-6212: Deprecate IndexWriter APIs that accept per-document Analyzer. +These methods were trappy as they made it easy to accidentally index +tokens that were not easily searchable and will be removed in 5.0.0. +
        (Mike McCandless)
        2. +
      +
      • +
    +

    Release 4.10.3 [2014-12-29]

    +
      +
    • Bug fixes   (12) +
        +
      1. LUCENE-6046: Add maxDeterminizedStates safety to determinize (which has +an exponential worst case) so that if it would create too many states, it +now throws an exception instead of exhausting CPU/RAM. +
        (Nik +Everett via Mike McCandless)
        2. +
      2. LUCENE-6054: Allow repeating the empty automaton +
        (Nik Everett via +Mike McCandless)
        3. +
      3. LUCENE-6049: Don't throw cryptic exception writing a segment when +the only docs in it had fields that hit non-aborting exceptions +during indexing but also had doc values. +
        (Mike McCandless)
        4. +
      4. LUCENE-6060: Deprecate IndexWriter.unlock +
        (Simon Willnauer, Mike +McCandless)
        5. +
      5. LUCENE-3229: Overlapping ordered SpanNearQuery spans should not match. +
        (Ludovic Boutros, Paul Elschot, Greg Dearing, ehatcher)
        6. +
      6. LUCENE-6004: Don't highlight the LookupResult.key returned from +AnalyzingInfixSuggester +
        (Christian Reuschling, jane chang via Mike McCandless)
        7. +
      7. LUCENE-6075: Don't overflow int in SimpleRateLimiter +
        (Boaz Leskes +via Mike McCandless)
        8. +
      8. LUCENE-5980: Don't let document length overflow. +
        (Robert Muir)
        9. +
      9. LUCENE-6042: CustomScoreQuery explain was incorrect in some cases, +such as when nested inside a boolean query. +
        (Denis Lantsman via Robert Muir)
        10. +
      10. LUCENE-5948: RateLimiter now fully inits itself on init. +
        (Varun +Thacker via Mike McCandless)
        11. +
      11. LUCENE-6055: PayloadAttribute.clone() now does a deep clone of the underlying +bytes. +
        (Shai Erera)
        12. +
      12. LUCENE-6094: Allow IW.rollback to stop ConcurrentMergeScheduler even +when it's stalling because there are too many merges. +
        (Mike McCandless)
        13. +
      +
      • +
    • Documentation   (1) +
        +
      1. LUCENE-6057: Improve Sort(SortField) docs +
        (Martin Braun via Mike McCandless)
        2. +
      +
      • +
    +

    Release 4.10.2 [2014-10-29]

    +
      +
    • Bug fixes   (2) +
        +
      1. LUCENE-5977: Fix tokenstream safety checks in IndexWriter to properly +work across multi-valued fields. Previously some cases across multi-valued +fields would happily create a corrupt index. +
        (Dawid Weiss, Robert Muir)
        2. +
      2. LUCENE-6019: Detect when DocValuesType illegally changes for the +same field name. Also added -Dtests.asserts=true|false so we can +run tests with and without assertions. +
        (Simon Willnauer, Robert +Muir, Mike McCandless). +

        +

        3. +
      +
      • +
    [... 11160 lines stripped ...]