Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 4D6ED200BFB for ; Wed, 7 Dec 2016 00:13:01 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 4AA22160B30; Tue, 6 Dec 2016 23:13:01 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id D38CB160B2F for ; Wed, 7 Dec 2016 00:12:58 +0100 (CET) Received: (qmail 46971 invoked by uid 500); 6 Dec 2016 23:12:57 -0000 Mailing-List: contact commits-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list commits@lucene.apache.org Received: (qmail 46534 invoked by uid 99); 6 Dec 2016 23:12:57 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Dec 2016 23:12:57 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 21101F16B3; Tue, 6 Dec 2016 23:12:57 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: sarowe@apache.org To: commits@lucene.apache.org Date: Tue, 06 Dec 2016 23:13:01 -0000 Message-Id: <5ebf071ac8064dc8808cb21564630252@git.apache.org> In-Reply-To: <3b2b6f35ed1c4dd3ac885912d9b2da60@git.apache.org> References: <3b2b6f35ed1c4dd3ac885912d9b2da60@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [06/50] [abbrv] lucene-solr:apiv2: reverting the large commit archived-at: Tue, 06 Dec 2016 23:13:01 -0000 http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/e9e4715d/solr/CHANGES.txt ---------------------------------------------------------------------- diff --git a/solr/CHANGES.txt b/solr/CHANGES.txt index 9db4b33..fe67490 100644 --- a/solr/CHANGES.txt +++ b/solr/CHANGES.txt @@ -8,14 +8,1750 @@ caching, replication, and a web administration interface. See http://lucene.apache.org/solr for more information. + +Getting Started +--------------- +You need a Java 1.8 VM or later installed. +In this release, there is an example Solr server including a bundled +servlet container in the directory named "example". +See the Quick Start guide at http://lucene.apache.org/solr/quickstart.html + +================== 7.0.0 ================== + +Upgrading from Solr 6.x +---------------------- + +* HttpClientInterceptorPlugin is now HttpClientBuilderPlugin and must work with a + SolrHttpClientBuilder rather than an HttpClientConfigurer. + +* HttpClientUtil now allows configuring HttpClient instances via SolrHttpClientBuilder + rather than an HttpClientConfigurer. Use of env variable SOLR_AUTHENTICATION_CLIENT_CONFIGURER + no longer works, please use SOLR_AUTHENTICATION_CLIENT_BUILDER + +* SolrClient implementations now use their own internal configuration for socket timeouts, + connect timeouts, and allowing redirects rather than what is set as the default when + building the HttpClient instance. Use the appropriate setters on the SolrClient instance. + +* HttpSolrClient#setAllowCompression has been removed and compression must be enabled as + a constructor param. + +* HttpSolrClient#setDefaultMaxConnectionsPerHost and + HttpSolrClient#setMaxTotalConnections have been removed. These now default very + high and can only be changed via param when creating an HttpClient instance. + +* Query time join with scoring {!join score=none} (even none) doesn't handle single value numeric fields. + Users are advised to convert these fields into string and reindex. + +* Deprecated collection and configset methods on MiniSolrCloudCluster have been + removed + +Bug Fixes +---------------------- +* SOLR-9262: Connection and read timeouts are being ignored by UpdateShardHandler after SOLR-4509. + (Mark Miller, shalin) + +Optimizations +---------------------- + +* SOLR-4509: Move to non deprecated HttpClient impl classes to remove stale connection + check on every request and move connection lifecycle management towards the client. + (Ryan Zezeski, Mark Miller, Shawn Heisey, Steve Davids) + +* SOLR-9255: Rename SOLR_AUTHENTICATION_CLIENT_CONFIGURER -> SOLR_AUTHENTICATION_CLIENT_BUILDER (janhoy) + +* SOLR-9579: Make Solr's SchemaField implement Lucene's IndexableFieldType, removing the + creation of a Lucene FieldType every time a field is indexed. (John Call, yonik) + + +================== 6.4.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.3.8.v20160314 + +Detailed Change List +---------------------- + +Upgrade Notes +---------------------- + +* SOLR-9166: Export handler returns zero for numeric fields that are not in the original doc. One + consequence of this change is that you must be aware that some tuples will not have values if + there were none in the original document. + +* SOLR-8785: Metrics related classes in org.apache.solr.util.stats have been removed in favor of + the dropwizard metrics library. Any custom plugins using these classes should be changed to use + the equivalent classes from the metrics library. + As part of this, the following changes were made to the output of Overseer Status API: + * The "totalTime" metric has been removed because it is no longer supported + * The metrics "75thPctlRequestTime", "95thPctlRequestTime", "99thPctlRequestTime" + and "999thPctlRequestTime" in Overseer Status API have been renamed to "75thPcRequestTime", "95thPcRequestTime" + and so on for consistency with stats output in other parts of Solr. + * The metrics "avgRequestsPerMinute", "5minRateRequestsPerMinute" and "15minRateRequestsPerMinute" have been + replaced by corresponding per-second rates viz. "avgRequestsPerSecond", "5minRateRequestsPerSecond" + and "15minRateRequestsPerSecond" for consistency with stats output in other parts of Solr. + +New Features +---------------------- +* SOLR-9293: Solrj client support for hierarchical clusters and other topics + marker. (Dawid Weiss) + +* SOLR-9681: FacetModule / JSON Facet API added the ability to add filters directly to + any facet command. The filters are applied after any domain change operations. + Example: { type:terms, field:category, filter:"user:yonik" } + (yonik) + +* SOLR-9442: Adds Array of NamedValuePair (json.nl=arrnvp) style to JSONResponseWriter. + (Jonny Marks, Christine Poerschke) + +* SOLR-9481: Authentication and Authorization plugins now work in standalone mode if security.json is placed in + SOLR_HOME on every node. Editing config through API is supported but affects only that one node. + (janhoy) + +* SOLR-8542: Adds Solr Learning to Rank (LTR) plugin for reranking results with machine learning models. + (Michael Nilsson, Diego Ceccarelli, Joshua Pantony, Jon Dorando, Naveen Santhapuri, Alessandro Benedetti, David Grohmann, Christine Poerschke) + +* SOLR-9055: Make collection backup/restore extensible. (Hrishikesh Gadre, Varun Thacker, Mark Miller) + +* SOLR-9682: JSON Facet API: added "param" query type to facet domain filter specification to obtain + filters via query parameters. (yonik) + +* SOLR-9038: Add a command-line tool to manage the snapshots functionality (Hrishikesh Gadre via yonik) + +* SOLR-9633: Limit memory consumed by FastLRUCache with a new 'maxRamMB' config parameter. + (yonik, Michael Sun, shalin) + +* SOLR-9666: SolrJ LukeResponse support dynamic fields (Fengtan via Kevin Risden) + +* SOLR-9077: Streaming expressions should support collection alias (Kevin Risden) + +* SOLR-9324: Support Secure Impersonation / Proxy User for solr authentication + (Gregory Chanan, Hrishikesh Gadre via yonik) + +* SOLR-9721: javabin Tuple parser for streaming and other end points (noble) + +Optimizations +---------------------- +* SOLR-9704: Facet Module / JSON Facet API: Optimize blockChildren facets that have + filters specified by using those filters as acceptDocs. (yonik) + +* SOLR-9726: Reduce number of lookupOrd calls made by the DocValuesFacets.getCounts method. + (Jonny Marks via Christine Poerschke) + +* SOLR-9772: Deriving distributed sort values (fieldSortValues) should reuse + comparator and only invalidate leafComparator. (John Call via yonik) + +* SOLR-9786: FieldType has a new getSetQuery() method that can take a set of terms + and create a more efficient query (such as TermsQuery). The solr query parser has been + changed to use this method when appropriate. The parser also knows when it is being + used to parse a filter and will create TermsQueries from large lists of normal terms + or numbers, resulting in a query that will execute faster. This also acts to avoid + BooleanQuery maximum clause limit. Query parsing itself has also been optimized, + resulting in less produced garbage and 5-7% better performance. + (yonik) + + +Bug Fixes +---------------------- +* SOLR-9701: NPE in export handler when "fl" parameter is omitted. + (Erick Erickson) + +* SOLR-9433: SolrCore clean-up logic uses incorrect path to delete dataDir on failure to create a core. + (Evan Sayer, shalin) + +* SOLR-9360: Solr script not properly checking SOLR_PID + (Alessandro Benedetti via Erick Erickson) + +* SOLR-9716: RecoveryStrategy sends prep recovery command without setting read time out which can cause + replica recovery to hang indefinitely on network partitions. (Cao Manh Dat, shalin) + +* SOLR-9624: In Admin UI, do not attempt to highlight CSV output (Alexandre Rafalovitch) + +* SOLR-9005: In files example, add a guard condition to javascript URP script (Alexandre Rafalovitch) + +* SOLR-9519: JSON Facet API: don't stop at an empty facet bucket if any sub-facets still have a chance + of matching something due to filter exclusions (which can widen the domain again). + (Michael Sun, yonik) + +* SOLR-9740: A bug in macro expansion of multi-valued parameters caused non-expanded values + after the first expanded value in the same multi-valued parameter to be dropped. + (Erik Hatcher, yonik) + +* SOLR-9751: PreAnalyzedField can cause managed schema corruption. (Steve Rowe) + +* SOLR-9736: Solr resolves the collection name against the first available leader or first replica + of the first slice. This puts undue pressure on leader cores and likely on the wrong ones. This is + fixed to randomly pick a leader on updates or a replica core otherwise. (Cao Manh Dat via shalin) + +* SOLR-9284: The HDFS BlockDirectoryCache should not let it's keysToRelease or names maps grow indefinitely. + (Mark Miller, Michael Sun) + +* SOLR-9729: JDBCStream improvements (Kevin Risden) + +* SOLR-9626: new Admin UI now also highlights matched terms in the Analysis screen. (Alexandre Rafalovitch) + +* SOLR-9512: CloudSolrClient's cluster state cache can break direct updates to leaders (noble) + +Other Changes +---------------------- + +* SOLR-7539: Upgrade the clustering plugin to Carrot2 3.15.0. (Dawid Weiss) + +* SOLR-9621: Remove several Guava & Apache Commons calls in favor of java 8 alternatives. + (Michael Braun via David Smiley) + +* SOLR-9720: Refactor Responsewriters to remove dependencies on TupleStream, + Tuple, Explanation (noble) + +* SOLR-9717: Refactor '/export' to not hardcode the JSON output and to use an API (noble) + +* SOLR-9739: JavabinCodec implements PushWriter interface (noble) + +* SOLR-8332: Factor HttpShardHandler[Factory]'s url shuffling out into a ReplicaListTransformer class. + (Christine Poerschke, Noble Paul) + +* SOLR-9597: Add setReadOnly(String ...) to ConnectionImpl (Kevin Risden) + +* SOLR-9609: Change hard-coded keysize from 512 to 1024 (Jeremy Martini via Erick Erickson) + +* SOLR-8785: Use Dropwizard Metrics library for core metrics. The copied over code in + org.apache.solr.util.stats has been removed. (Jeff Wartes, Kelvin Wong, Christine Poerschke, shalin) + +* SOLR-9784: Refactor CloudSolrClient to eliminate direct dependency on ZK (noble) + +================== 6.3.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.12.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.3.8.v20160314 + +Detailed Change List +---------------------- + +Upgrade Notes +---------------------- + +* If you use the JSON Facet API (json.facet) with method=stream, you must now set sort='index asc' to get the streaming + behavior; otherwise it won't stream. Reminder: "method" is a hint that doesn't change defaults of other parameters. + +* If you use the JSON Facet API (json.facet) to facet on a numeric field and if you use mincount=0 or if you set the + prefix, then you will now get an error as these options are incompatible with numeric faceting. + +* Solr's logging verbosity at the INFO level has been greatly reduced, and + you may need to update the log configs to use the DEBUG level to see all the + logging messages you used to see at INFO level before. + +* We are no longer backing up solr.log and solr_gc.log files in date-stamped copies forever. If you relied on + the solr_log_ or solr_gc_log_ being in the logs folder that will no longer be the case. + See SOLR-9570 for details. + +* The create/deleteCollection methods on MiniSolrCloudCluster have been + deprecated. Clients should instead use the CollectionAdminRequest API. In + addition, MiniSolrCloudCluster#uploadConfigDir(File, String) has been + deprecated in favour of #uploadConfigSet(Path, String) + +* The bin/solr.in.sh (bin/solr.in.cmd on Windows) is now completely commented by default. Previously, this wasn't so, + which had the effect of masking existing environment variables. + +New Features +---------------------- +* SOLR-5725: facet.method=enum can bypass exact counts calculation with facet.exists=true, it just returns 1 for + terms which exists in result docset. (Alexey Kozhemiakin, Sebastian Koziel, Radoslaw Zielinski via Mikhail Khludnev) + +* SOLR-9127: Excel workbook (.xlsx) response writer. use 'wt=xlsx' (Tony Moriarty, noble) + +* SOLR-9469: JettySolrRunner now has the option of restarting using a different + port (Alan Woodward) + +* SOLR-9319: DELETEREPLICA can accept a 'count' and remove appropriate replicas (Nitin Sharma, noble) + +* SOLR-8186: Reduce logging to logs/solr--console.log when not running in foreground mode + Show timestamp also in foreground log. Also removes some logging noise. (janhoy) + +* SOLR-8487: Adds CommitStream to support sending commits to a collection being updated. (Dennis Gove) + +* SOLR-9534: You can now set Solr's log level through environment variable SOLR_LOG_LEVEL + Also adds conveience arguments -q (quiet: WARN) and -v (verbose: DEBUG) to bin/solr (janhoy) + +* SOLR-9537: Support facet scoring with the scoreNodes expression (Joel Bernstein) + +* SOLR-9558: DIH TemplateTransformerto to support multivalued fields (Ted Sullivan via noble) + +* SOLR-9557: Every implicit requesthandler now has a default 'useParams' attribute (noble) + +* SOLR-9572: config API to show expanded useParams for request handlers inline (noble) + +* SOLR-9258: Optimizing, storing and deploying AI models with Streaming Expressions (Cao Manh Dat, Joel Bernstein) + +* SOLR-9205: Added method LukeResponse.getSchemaFlags() which returns field + information as an EnumSet (Fengtan, Alan Woodward) + +* SOLR-9520: Kerberos delegation support in SolrJ (Ishan Chattopadhyaya, noble) + +* SOLR-9146: Parallel SQL engine should support >, >=, <, <=, <>, != syntax (Timothy Potter, Joel Bernstein, Kevin Risden) + +* SOLR-9337: Add fetch Streaming Expression (Joel Bernstein) + +* SOLR-9103: Restore ability for users to add custom Streaming Expressions (Cao Manh Dat) + +* SOLR-9657: New TemplateUpdateProcessorFactory added (noble) + +* SOLR-9417: Allow daemons to terminate when they finish iterating a topic (Joel Bernstein) + +* SOLR-8370: Display configured Similarity in Schema-Browser, both global/default and per-field/field-type + (janhoy, Alexandre Rafalovitch) + +* SOLR-9326: Ability to create/delete/list snapshots at collection level. + (Hrishikesh Gadre via yonik) + +* SOLR-9662: New parameter -u in bin/post to pass basicauth credentials (janhoy) + +* SOLR-9654: Add "overrequest" parameter to JSON Facet API to control amount of overrequest + on a distributed terms facet. (yonik) + +* SOLR-2212: Add a factory class corresponding to Lucene's NoMergePolicy. (Lance Norskog, Cao Manh Dat via shalin) + +* SOLR-9670: Support SOLR_AUTHENTICATION_OPTS in solr.cmd (janhoy) + +* SOLR-9559: Add ExecutorStream to execute stored Streaming Expressions (Joel Bernstein) + +* SOLR-1085: Add support for MoreLikeThis queries and responses in SolrJ client. + (Maurice Jumelet, Bill Mitchell, Cao Manh Dat via shalin) + +Bug Fixes +---------------------- + +* SOLR-9310: PeerSync fails on a node restart due to IndexFingerPrint mismatch (Pushkar Raste, noble) + +* SOLR-9484: The modify collection API should wait for the modified properties to show up in the + cluster state. (Cao Manh Dat, shalin) + +* SOLR-9507: CoreContainer threads now correctly set their MDC logging values + (Alan Woodward) + +* SOLR-9522: Improve error handling in ZKPropertiesWriter (Varun Thacker) + +* SOLR-8080: bin/solr start script now exits with informative message if using wrong Java version (janhoy) + +* SOLR-9475: bin/install_solr_service.sh script got improved detection of Linux distro, especially within + virtualized/Docker environment through parsing of /etc/*-release files. Now also supports CentOS. (janhoy) + +* SOLR-9524: SolrIndexSearcher.getIndexFingerprint uses dubious synchronization (Mike Drob, noble) + +* SOLR-9542: Kerberos delegation tokens requires Jackson library (Ishan Chattopadhyaya via noble) + +* SOLR-9330: Fix AlreadyClosedException on admin/mbeans?stats=true (Mikhail Khludnev) + +* SOLR-9411: Better validation for Schema API add-field and add-dynamic-field (janhoy, Steve Rowe) + +* SOLR-9504: A replica with an empty index becomes the leader even when other more qualified replicas + are in line. (shalin) + +* SOLR-9554: Fix NullPointerException when cores are loaded in parallel and switch schema.xml to managed-scheme. + (Alan Woodward, Mikhail Khludnev) + +* SOLR-9556: OverseerAutoFailoverReplicaThread was not exiting on interrupt + (Alan Woodward) + +* SOLR-9563: Collection creation could fail if an ADDREPLICA subrequest arrived + at a node before its local state had updated with the new collection data + (Alan Woodward) + +* SOLR-9278: Index replication interactions with IndexWriter can cause deadlock. (Xunlong via Mark Miller) + +* SOLR-9604: Pooled SSL connections were not being re-used (Alan Woodward, + Mikhail Khludnev, hossman) + +* SOLR-9325: solr.log is now written to $SOLR_LOGS_DIR without changing log4j.properties (janhoy) + +* SOLR-9518: Kerberos Delegation Tokens don't work without a chrooted ZK (Ishan Chattopadhyaya,via noble) + +* SOLR-9687: Fixed Interval Facet count issue in cases of open/close intervals on the same values + (Andy Chillrud, Tomás Fernández Löbbe) + +* SOLR-9441: Solr collection backup on HDFS can only be manipulated by the Solr process owner. + (Hrishikesh Gadre via Mark Miller) + +* SOLR-9536: OldBackupDirectory timestamp field needs to be initialized to avoid NPE. + (Hrishikesh Gadre, hossman via Mark Miller) + +* SOLR-2039: Multivalued fields with dynamic names does not work properly with DIH. + (K A, ruslan.shv, Cao Manh Dat via shalin) + +* SOLR-4164: group.limit=-1 was not supported for grouping in distributed mode. + (Cao Manh Dat, Lance Norskog, Webster Homer, hossman, yonik) + +* SOLR-9692: blockUnknown property makes inter-node communication impossible (noble) + +* SOLR-2094: XPathEntityProcessor should reinitialize the XPathRecordReader instance if + the 'forEach' or 'xpath' attributes are templates & it is not a root entity (Cao Manh Dat, noble) + +* SOLR-9697: zk upconfig broken on windows (Xavier Jmlucjav via janhoy) + +Optimizations +---------------------- + +* SOLR-9374: Speed up Jmx MBean retrieval for FieldCache. (Tim Owen via shalin) + +* SOLR-9449: Example schemas do not index _version_ field anymore because the field + has DocValues enabled already. (shalin) + +* SOLR-9447: Do not clone SolrInputDocument if update processor chain does not contain custom processors. + (shalin) + +* SOLR-9452: JsonRecordReader should not deep copy document before handler.handle(). (noble, shalin) + +* SOLR-9142: JSON Facet API: new method=dvhash can be chosen for fields with high cardinality. (David Smiley) + +* SOLR-9446: Leader failure after creating a freshly replicated index can send nodes into recovery even if + index was not changed (Pushkar Raste, noble) + +* SOLR-9592: retrieving docValues as stored values was sped up by using the proper leaf + reader rather than ask for a global view. In extreme cases, this leads to a 100x speedup. + (Takahiro Ishikawa, yonik) + +* SOLR-9566: Don't put replicas into recovery when first creating a Collection + (Alan Woodward) + +* SOLR-9546: Eliminate unnecessary boxing/unboxing going on in SolrParams (Pushkar Raste, noble) + +* SOLR-9506: cache IndexFingerprint for each segment (Pushkar Raste, yonik, noble) + +* SOLR-7506: Roll over GC logs by default via bin/solr scripts (shalin, janhoy) + +Other Changes +---------------------- + +* SOLR-9412: Add failOnMissingParams option to MacroExpander, add TestMacroExpander class. + (Jon Dorando, Christine Poerschke) + +* SOLR-9406: SolrSuggester should selectively register close hook (Gethin James, Joel Bernstein) + +* SOLR-8961: Add a test module for solr-test-framework (Alan Woodward) + +* SOLR-9474: MiniSolrCloudCluster will not reuse ports by default when + restarting its JettySolrRunners (Alan Woodward) + +* SOLR-9498: Remove HDFS properties from DIH solrconfig.xml, as started in SOLR-6943 (Alexandre Rafalovitch) + +* SOLR-9365: Reduce noise in solr logs during graceful shutdown. (Cao Manh Dat via shalin) + +* SOLR-9451: Make clusterstatus command logging less verbose. (Varun Thacker) + +* SOLR-9502: ResponseWriters should natively support MapSerializable (noble) + +* SOLR-9538: Relocate (BinaryResponse|JSON|Smile)Writer tests to org.apache.solr.response + which is the package of the classes they test. (Jonny Marks via Christine Poerschke) + +* SOLR-9508: Install script install_solr_service.sh now checks existence of tools. + New option -n to avoid starting service after installation (janhoy) + +* SOLR-7826: Refuse "bin/solr create" if run as root, unless -force is specified (janhoy, Binoy Dalal) + +* SOLR-6871: Updated the quickstart tutorial to cover the 6.2.0 release, and added ant target + "generate-website-quickstart" to convert the bundled version of the tutorial into one suitable + for the website. + +* SOLR-5563: Move lots of SolrCloud logging from 'info' to 'debug' (janhoy, Alan + Woodward) + +* SOLR-9544: Allow ObjectReleaseTracker more time to check for asynchronously + closing resources (Alan Woodward) + +* SOLR-6677: Reduced logging during startup and shutdown, moved more logs to DEBUG level + (janhoy, Shawn Heisey, Alan Woodward) + +* SOLR-6090: Remove unreachable printLayout usage in cloud tests. (Cao Manh Dat via shalin) + +* SOLR-9551: Add JSONWriter constructor variant, JSONWriterTest.testConstantsUnchanged test. + (Jonny Marks, Christine Poerschke) + +* SOLR-9500: Add a LogLevel annotation to set log levels on specific tests (Alan + Woodward) + +* SOLR-9548: The beginning of solr.log now starts with a more informative welcome message (janhoy) + +* SOLR-9547: Do not allow bin/solr start as root user, unless -force param specified (janhoy) + +* SOLR-9567: Make ReRankQParserPlugin's private ReRankCollector a public class of its own. (Christine Poerschke) + +* SOLR-7436: Solr stops printing stacktraces in log and output (janhoy, hossman, Markus Jelsma) + +* SOLR-9576: Make FieldAnalysisRequestHandler, DocumentAnalysisRequestHandler & DumpRequestHandler + implicit (noble) + +* SOLR-9574: Factor out AbstractReRankQuery from ReRankQParserPlugin's private ReRankQuery. (Christine Poerschke) + +* SOLR-5041: Add a test to make sure that a leader always recovers from log on startup. (Cao Manh Dat, shalin) + +* SOLR-9588: Remove Guava dependency from SolrJ (Ishan Chattopadhyaya, noble) + +* SOLR-8140: Remove mentions of unimplemented admin-extra from the new Admin UI (Alexandre Rafalovitch) + +* SOLR-9589: Remove jackson dependency from SolrJ (Ishan Chattopadhyaya, noble) + +* SOLR-8385: Narrow StreamFactory.withFunctionName clazz parameter to prevent misconfiguration (Jason Gerlowski, Kevin Risden) + +* SOLR-8969: SQLHandler causes NPE in non-cloud mode (Markus Jelsma, Kevin Risden) + +* SOLR-9610: New AssertTool in SolrCLI for easier cross platform assertions from command line (janhoy) + +* SOLR-9680: Better error messages in SolrCLI when authentication required (janhoy) + +* SOLR-9639: Test only fix. Prevent CDCR tests from removing collection during recovery that used to blow up jvm (Mikhail Khludnev) + +* SOLR-9625: Add HelloWorldSolrCloudTestCase class (Christine Poerschke, Alan Woodward, Alexandre Rafalovitch) + +* SOLR-9642: Refactor the core level snapshot cleanup mechanism to rely on Lucene (Hrishikesh Gadre via yonik) + +* SOLR-9627: Add QParser.getSortSpec, deprecate misleadingly named QParser.getSort (Judith Silverman, Christine Poerschke) + +* SOLR-9632: Add MiniSolrCloudCluster#deleteAllCollections() method (Alan Woodward) + +* SOLR-9634: Deprecate collection methods on MiniSolrCloudCluster (Alan Woodward) + +* SOLR-7850: Moved defaults within bin/solr.in.sh (and bin/solr.in.cmd on Windows) to bin/solr (and bin/solr.cmd) + such that the default state of these files is to set nothing. This makes Solr work better with Docker. (David Smiley) + +* SOLR-9570: Various log tidying now happens at Solr startup: + Old solr_log_ and solr_gc_log_ files are removed, avoiding disks to fill up, + solr.log.X files are rotated, preserving solr.log from last run in solr.log.1, solr.log.1 => solr.log.2 etc + solr-*-console.log files are moved into $SOLR_LOGS_DIR/archived/ instead of being overwritten + Last JVM garbage collection log solr_gc.log is moved into $SOLR_LOGS_DIR/archived/ + (janhoy) + +* SOLR-4531: Add tests to ensure that recovery does not fail on corrupted tlogs. + (Simon Scofield, Cao Manh Dat via shalin) + +* SOLR-5245: Add a test to ensure that election contexts are keyed off both collection name and coreNodeName + so that killing a shard in one collection does not result in leader election in a different collection. + See SOLR-5243 for the related bug. (Cao Manh Dat via shalin) + +* SOLR-9533: Reload core config when a core is reloaded (Gethin James, Joel Bernstein) + +* SOLR-9371: Fix bin/solr calculations for start/stop wait time and RMI_PORT. + (Shawn Heisey via Erick Erickson) + + +================== 6.2.1 ================== + +Bug Fixes +---------------------- + +* SOLR-9494: Use of {!collapse} sometimes doesn't correctly return true for Collector.needsScores(), especially when the + query was cached. This can cause an exception when 'q' is a SpanQuery or potentially others. (David Smiley) + +* SOLR-6744: fl renaming / alias of uniqueKey field generates null pointer exception in SolrCloud configuration + (Mike Drob via Tomás Fernández Löbbe) + +* SOLR-9445: Admin requests are retried by CloudSolrClient and LBHttpSolrClient on failure. (shalin) + +* SOLR-9439: Shard split clean up logic for older failed splits is faulty. The delete shard API + has also been made more resilient against failures resulting from non-existent cores. (shalin) + +* SOLR-9430: Fix locale lookup in DIH to use BCP47 language tags + to be consistent with other places in Solr. Language names still work for backwards + compatibility. (Uwe Schindler, Boris Steiner) + +* SOLR-9389: HDFS Transaction logs stay open for writes which leaks Xceivers. (Tim Owen via Mark Miller) + +* SOLR-9188: blockUnknown property makes inter-node communication impossible (noble) + +* SOLR-9455: Deleting a sub-shard in recovery state can mark parent shard as inactive. (shalin) + +* SOLR-9461: DELETENODE, REPLACENODE should pass down the 'async' param to subcommands (shalin, noble) + +* SOLR-9444: Fix path usage for cloud backup/restore. (Hrishikesh Gadre, Uwe Schindler, Varun Thacker) + +* SOLR-9381: Snitch for freedisk uses '/' instead of 'coreRootDirectory' (Tim Owen, noble) + +* SOLR-9488: Shard split can fail to write commit data on shutdown/restart causing replicas to recover + without replicating the index. This can cause data loss. (shalin) + +* SOLR-9490: Fixed bugs in BoolField that caused it to erroneously return "false" for all docs depending + on usage (Colvin Cowie, Dan Fox, hossman) + +* SOLR-9438: Shard split can be marked successful and sub-shard states switched to 'active' even when + one or more sub-shards replicas do not recover due to the leader crashing or restarting between the time + the replicas are created and before they can recover. This can cause data loss. (shalin) + +* SOLR-9408: Fix TreeMergeOutputFormat to add timestamp metadata to a commit. SolrCloud replication relies on this. + (Jessica Cheng Mallet via Varun Thacker) + +Other Changes +---------------------- + +* SOLR-7362: Fix TestReqParamsAPI test failures (noble, Varun Thacker) + +================== 6.2.0 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.12.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.3.8.v20160314 + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-9187: Support dates and booleans in /export handler, support boolean DocValues fields (Erick Erickson) + +* SOLR-8048: bin/solr script should support basic auth credentials provided in solr.in.sh (noble) + +* SOLR-7374: Core level Backup/Restore now supports specifying the directory implementation to use + via the "repository" parameter. (Hrishikesh Gadre, Varun Thacker, Mark Miller) + +* SOLR-9216: Support collection.configName in MODIFYCOLLECTION request (Keith Laban, noble) + +* SOLR-9251: Support for a new tag 'role' in replica placement rules (noble) + +* SOLR-9194: Enhance the bin/solr script to perform file operations to/from Zookeeper (Erick Erickson, janhoy) + +* SOLR-9242: Collection Backup/Restore now supports specifying the directory implementation to use + via the "repository" parameter. (Hrishikesh Gadre, Varun Thacker) + +* SOLR-9193: Add scoreNodes Streaming Expression (Joel Bernstein) + +* SOLR-9243: Add terms.list parameter to the TermsComponent to fetch the docFreq for a list of terms + (Joel Bernstein) + +* SOLR-9090: Add directUpdatesToLeadersOnly flag to solrj CloudSolrClient. + (Marvin Justice, Christine Poerschke) + +* SOLR-9270: Allow spatialContextFactory to be simply "JTS". And if any spatial params include the old + Spatial4j package "com.spatial4j.core" it is rewritten to "org.locationtech.spatial4j" with a warning. + (David Smiley) + +* SOLR-9240: Support parallel ETL with the topic expression (Joel Bernstein) + +* SOLR-9275: XML QueryParser support (defType=xmlparser) now extensible via configuration. + (Christine Poerschke) + +* SOLR-9200: Add Delegation Token Support to Solr. + (Gregory Chanan) + +* SOLR-9038: Solr core snapshots: The current commit can be snapshotted which retains the commit and associates it with + a name. The core admin API can create snapshots, list them, and delete them. Snapshot names can be referenced in + doing a core backup, and in replication. Snapshot metadata is stored in a new snapshot_metadata/ dir. + (Hrishikesh Gadre via David Smiley) + +* SOLR-9279: New boolean comparison function queries comparing numeric arguments: gt, gte, lt, lte, eq + (Doug Turnbull, David Smiley) + +* SOLR-9252: Feature selection and logistic regression on text (Cao Manh Dat, Joel Bernstein) + +* SOLR-6465: CDCR: fall back to whole-index replication when tlogs are insufficient. + (Noble Paul, Renaud Delbru, shalin) + +* SOLR-9320: A REPLACENODE command to decommission an existing node with another new node + (noble, Nitin Sharma, Varun Thacker) + +* SOLR-9318: A DELETENODE command to delete all replicas in that node (noble, Nitin Sharma, Varun Thacker) + +Bug Fixes +---------------------- + +* SOLR-9191: OverseerTaskQueue.peekTopN() fatally flawed (Scott Blum, Noble Paul) + +* SOLR-9199: ZkController#publishAndWaitForDownStates logic is inefficient (Hrishikesh Gadre) + +* SOLR-9161: Change SolrPluginUtils.invokeSetters implementation to accommodate setter variants. + (Christine Poerschke, Steve Rowe, Uwe Schindler) + +* SOLR-9234: srcField parameter works only when all fields are captured in the /update/json/docs + endpoint (noble) + +* SOLR-8546: SOLR-8546: TestLazyCores is failing a lot on the Jenkins cluster. (Erick Erickson) + +* SOLR-9237: DefaultSolrHighlighter.doHighlightingByFastVectorHighlighter can't be overidden (janhoy) + +* SOLR-8626: 404 error when clicking nodes in cloud graph view in angular UI. (janhoy, Trey Grainger via shalin) + +* SOLR-9254: GraphTermsQueryQParserPlugin throws NPE when field being searched is not present in segment + (Joel Bernstein) + +* SOLR-8657: Fix SolrRequestInfo error logs if QuerySenderListener is being used (Pascal Chollet, + Tomás Fernández Löbbe) + +* SOLR-8777: Duplicate Solr process can cripple a running process. (Jessica Cheng Mallet, Scott Blum, shalin) + +* SOLR-9246: If the JDBCStream sees an unknown column type it will now throw a detailed exception. (Dennis Gove) + +* SOLR-9181: Fix some races in CollectionStateWatcher API (Alan Woodward, Scott + Blum) + +* SOLR-9235: Fixed NPE when using non-numeric range query in deleteByQuery (hossman) + +* SOLR-9088: Fixed TestManagedSchemaAPI failures which exposed race conditions in the schema API ( Varun Thacker, noble) + +* SOLR-9207: PeerSync recovery failes if number of updates requested is high. A new useRangeVersions config option + is introduced (defaults to true) to send version ranges instead of individual versions for peer sync. + (Pushkar Raste, shalin) + +* SOLR-8858: SolrIndexSearcher#doc() completely ignores field filters unless lazy field loading is enabled. + (Caleb Rackliffe, David Smiley, shalin) + +* SOLR-9236: AutoAddReplicas will append an extra /tlog to the update log location on replica failover. + (Eungsop Yoo, Mark Miller) + +* SOLR-9291: ZkSolrResourceLoader should not retry fetching resources if the server has been shutdown. + (shalin) + +* SOLR-9287: Including 'score' in the 'fl' param when doing an RTG no longer causes an NPE + (hossman, Ishan Chattopadhyaya) + +* SOLR-7280: In cloud-mode sort the cores smartly before loading & limit threads to improve cluster stability + (noble, Erick Erickson, shalin) + +* SOLR-9285: Fixed AIOOBE when using ValueSourceAugmenter in single node RTG (hossman) + +* SOLR-9288: Fix [docid] transformer to return -1 when used in RTG with uncommitted doc (hossman) + +* SOLR-9309: Fix SolrCloud RTG response structure when multi ids requested but only 1 found (hossman) + +* SOLR-9334: CloudSolrClient.collectionStateCache is unbounded (noble) + +* SOLR-9339: NPE in CloudSolrClient when the response is null (noble) + +* SOLR-8596: Web UI doesn't correctly generate queries which include local parameters (Alexandre Rafalovitch, janhoy) + +* SOLR-8645: managed-schema is now syntax highlighted in cloud->Tree view (Alexandre Rafalovitch via janhoy) + +* SOLR-8379: UI Cloud->Tree view now shows .txt files correctly (Alexandre Rafalovitch via janhoy) + +* SOLR-9003: New Admin UI's Dataimport screen now correctly displays DIH Debug output (Alexandre Rafalovitch) + +* SOLR-9308: Fix distributed RTG to forward request params, fixes fq and non-default fl params (hossman) + +* SOLR-9179: NPE in IndexSchema using IBM JDK (noble, Colvin Cowie) + +* SOLR-9397: Config API does not support adding caches (noble) + +* SOLR-9405: ConcurrentModificationException in ZkStateReader.getStateWatchers. + (Alan Woodward, Edward Ribeiro, shalin) + +* SOLR-9232: Admin UI now fully implements Swap Cores interface (Alexandre Rafalovitch) + +* SOLR-8715: Admin UI's Schema screen now works for fields with stored=false and some content indexed (Alexandre Rafalovitch) + +* SOLR-8911: In Admin UI, enable scrolling for overflowing Versions and JVM property values (Alexandre Rafalovitch) + +* SOLR-9002: Admin UI now correctly displays json and text files in the collection/Files screen (Upayavira, Alexandre Rafalovitch) + +* SOLR-8993: Admin UI now correctly supports multiple DIH handler end-points (Upayavira, Alexandre Rafalovitch) + +* SOLR-9032: Admin UI now correctly implements Create Alias command (Upayavira, Alexandre Rafalovitch) + +* SOLR-9391: LBHttpSolrClient.request now correctly returns Rsp.server when + previously skipped servers were successfully tried. (Christine Poerschke) + +Optimizations +---------------------- + +* SOLR-9219: Make hdfs blockcache read buffer sizes configurable and improve cache concurrency. (Mark Miller) + +* SOLR-9264: Optimize ZkController.publishAndWaitForDownStates to not read all collection states and + watch relevant collections instead. (Hrishikesh Gadre, shalin) + +* SOLR-9335: Solr cache/search/update stats counters now use LongAdder which are supposed to have higher throughput + under high contention. (Varun Thacker) + +* SOLR-9350: JSON Facets: method="stream" will no longer always uses & populates the filter cache, likely + flushing it. 'cacheDf' can be configured to set a doc frequency threshold, now defaulting to 1/16th doc count. + Using -1 Disables use of the cache. (David Smiley, yonik) + +Other Changes +---------------------- + +* SOLR-9195: Remove unnecessary allocation and null check in UpdateRequestProcessorChain's + getReqProcessors method. (Christine Poerschke) + +* SOLR-8981: Upgraded Extraction module to Apache Tika 1.13. + (Tim Allison, Lewis John McGibbney via Uwe Schindler) + +* SOLR-9076: Update to Hadoop 2.7.2 + (Mark Miller, Gregory Chanan) + +* SOLR-8787: TestAuthenticationFramework should not extend TestMiniSolrCloudCluster. (Trey Cahill via shalin) + +* SOLR-9180: More comprehensive tests of psuedo-fields for RTG and SolrCloud requests (hossman) + +* SOLR-7930: Comment out trappy references to example docs in elevate.xml files (Erick Erickson) + +* SOLR-9277: Clean up some more remnants of supporting old and new style solr.xml in tests (Erick Erickson) + +* SOLR-9163: Sync up basic_configs and data_driven_schema_configs, removing almost all differences + except what is required for schemaless. (yonik) + +* SOLR-9340: Change ZooKeeper disconnect and session expiry related logging from INFO to WARN to + make debugging easier (Varun Thacker) + +* SOLR-9358: [AngularUI] In Cloud->Tree file view area, collapse metadata by default (janhoy) + +* SOLR-9256: asserting hasNext() contract in JdbcDataSource in DataImportHandler (Kristine Jetzke via Mikhai Khludnev) + +* SOLR-9209: extracting JdbcDataSource.createResultSetIterator() for extension (Kristine Jetzke via Mikhai Khludnev) + +* SOLR-9353: Factor out ReRankQParserPlugin.ReRankQueryRescorer private class. (Christine Poerschke) + +* SOLR-9392: Fixed CDCR Test failures which were due to leaked resources. (shalin) + +* SOLR-9385: Add QParser.getParser(String,SolrQueryRequest) variant. (Christine Poerschke) + +* SOLR-9367: Improved TestInjection's randomization logic to use LuceneTestCase.random() (hossman) + +* SOLR-9331: Remove ReRankQuery's length constructor argument and member. (Christine Poerschke) + +* SOLR-9092: For the delete replica command we attempt to send the core admin delete request only + if that node is actually up. (Jessica Cheng Mallet, Varun Thacker) + +* SOLR-9410: Make ReRankQParserPlugin's private ReRankWeight a public class of its own. (Christine Poerschke) + +* SOLR-9404: Refactor move/renames in JSON FacetProcessor and FacetFieldProcessor. (David Smiley) + +* SOLR-9421: Refactored out OverseerCollectionMessageHandler to smaller classes (noble) + +* SOLR-8643: BlockJoinFacetComponent is substituted by BlockJoinFacetDocSetComponent. It doesn't need to change solrconfig.xml (Mikhail Khludnev) + +* SOLR-8644: Test asserts that block join facets work with parent level fq exclusions. (Dr. Oleg Savrasov via Mikhail Khludnev) + +================== 6.1.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.7 +Carrot2 3.12.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.3.8.v20160314 + +Upgrading from Solr any prior release +---------------------- + +* If you use historical dates, specifically on or before the year 1582, you should re-index. + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-8782: Add asynchronous sugar methods to the SolrJ Collections API. You + can now call .processAsync() to run a method asynchronously, or + .processAndWait() to wait for a call to finish without holding HTTP + collections open. (Alan Woodward) + +* SOLR-8765: Enforce required parameters at query construction time in the SolrJ + Collections API, add static factory methods, and deprecate old setter methods. + (Alan Woodward, Jason Gerlowski) + +* SOLR-8842: authorization APIs do not use name as an identifier for a permission + for update, delete commands and 'before' attribute (noble) + +* SOLR-8814: Support GeoJSON response writer and general spatial formatting. Adding + &wt=geojson&geojson.field= + Will return a FeatureCollection for each SolrDocumentList and a Feature with the + requested geometry for each SolrDocument. The requested geometry field needs + to either extend AbstractSpatialFieldType or store a GeoJSON string. This also adds + a [geo] DocumentTransformer that can return the Shape in a variety of formats: + &fl=[geo f= w=(GeoJSON|WKT|POLY)] + The default format is GeoJSON. For information on the supported formats, see: + https://github.com/locationtech/spatial4j/blob/master/FORMATS.md + To return the FeatureCollection as the root element, add '&omitHeader=true" (ryan) + +* SOLR-8859: Spatial fields like RPT can now be configured to use Spatial4j registered shape formats + e.g. via format="GeoJSON". (ryan, David Smiley) + +* SOLR-445: new TolerantUpdateProcessorFactory to support skipping update commands that cause + failures when sending multiple updates in a single request. + (Erick Erickson, Tomás Fernández Löbbe, Anshum Gupta, hossman) + +* SOLR-8890: New static method in DistributedUpdateProcessorFactory to allow UpdateProcessorFactories + to indicate request params that should be forwarded when DUP distributes updates. (hossman) + +* SOLR-8888: Add shortestPath Streaming Expression. This expression performs a breadth first + graph traversal to find the shortest path(s) in an unweighted, directed graph stored in a + SolrCloud collection. (Joel Bernstein) + +* SOLR-8938: Add optional --excluderegex argument to ZkCLI. (Christine Poerschke) + +* SOLR-8976: Add SolrJ support for REBALANCELEADERS Collections API (Anshum Gupta) + +* SOLR-8962: Add sort Streaming Expression. The expression takes a single input stream and a + comparator and outputs tuples in stable order of the comparator. (Dennis Gove) + +* SOLR-8349: Allow sharing of large in memory data structures across cores (Gus Heck, noble) + +* SOLR-9009: Adds ability to get an Explanation of a Streaming Expression (Dennis Gove) + +* SOLR-8918: Adds Streaming to the admin page under the collections section. Includes + ability to see graphically the expression explanation (Dennis Gove) + +* SOLR-8913: When using a shared filesystem we should store data dir and tlog dir locations in + the cluster state. (Mark Miller) + +* SOLR-8809: Implement Connection.prepareStatement (Kevin Risden) + +* SOLR-9020: Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper errors for traversal methods (Kevin Risden) + +* SOLR-9041: 'core-admin-read' and 'core-admin-edit' are well known permissions (noble) + +* SOLR-8986: Add Random Streaming Expression (Joel Bernstein) + +* SOLR-8925: Add gatherNodes Streaming Expression to support breadth first traversals (Joel Bernstein) + +* SOLR-9027: Add GraphTermsQuery to limit traversal on high frequency nodes (Joel Bernstein, David Smiley) + +* SOLR-5750: Add /admin/collections?action=BACKUP and RESTORE assuming access to a shared file system. + (Varun Thacker, David Smiley) + +* SOLR-9049: RuleBasedAuthorizationPlugin supports regex in param values eg: "command" : "REGEX:(i?)create" (noble) + +* SOLR-8972: Add GraphHandler and GraphMLResponseWriter to support graph visualizations (Joel Bernstein) + +* SOLR-9026: Extend facet telemetry support to legacy (non-json) facets under "debug/facet-debug" in + the response. (Michael Sun, yonik) + +* SOLR-7117: Provide an option to limit the maximum number of cores that can be created on a node by the + Auto Add Replica feature. For this you can set a "maxCoresPerNode" property via the Cluster Property API + (Varun Thacker, Mark Miller) + +* SOLR-8208: [subquery] document transformer executes separate requests per result document. (Cao Manh Dat via Mikhail Khludnev) + +* SOLR-8323, SOLR-9113: Add CollectionStateWatcher API (Alan Woodward, Scott Blum) + +* SOLR-8988: Adds query option facet.distrib.mco which when set to true allows the use of facet.mincount=1 in cloud mode. + (Keith Laban, Dennis Gove) + +* SOLR-8583: Apply highlighting to hl.alternateField by default for Default and FastVectorHighlighter. + Turn off with hl.highlightAlternate=false (janhoy, David Smiley) + +* SOLR-7123: '/update/json/docs' path supports nested documents (noble) + +* SOLR-8610: Resolve variables in encryptKeyFile of DIH's JdbcDataSource (Kristine Jetzke via Mikhail Khludnev) + +* SOLR-7739: Add a new ClassificationUpdateProcessorFactory. (Alessandro Benedetti via Tommaso Teofili) + +Bug Fixes +---------------------- + +* SOLR-8855: The HDFS BlockDirectory should not clean up it's cache on shutdown. (Mark Miller) + +* SOLR-8948: OverseerTaskQueue.containsTaskWithRequestId encounters json parse error if a + SolrResponse node is in the overseer queue. (Jessica Cheng Mallet via shalin) + +* SOLR-7729: ConcurrentUpdateSolrClient ignores the collection parameter in some methods. + (Nicolas Gavalda, Jorge Luis Betancourt Gonzalez via Mark Miller) + +* SOLR-8662: SchemaManager waits correctly for replicas to be notified of a new change. + (sarowe, Noble Paul, Varun Thacker) + +* SOLR-8983: Cleanup clusterstate and replicas for a failed create collection request + (Varun Thacker, Anshum Gupta) + +* SOLR-9029: fix rare ZkStateReader visibility race during collection state format update (Scott Blum, hossman) + +* SOLR-9046: Fix solr.cmd that wrongly assumes Jetty will always listen on 0.0.0.0. + (Bram Van Dam, Uwe Schindler) + +* SOLR-9064: Adds an explanation of the incoming stream to an UpdateStream's explanation (Dennis Gove) + +* SOLR-9128: Fix error handling issues in Streaming classes (Joel Bernstein) + +* SOLR-9151: Fix SolrCLI so that bin/solr -e cloud example can be run from any CWD (janhoy) + +* SOLR-9141: Fix ClassCastException when using the /sql handler count() function with + single-shard collections (Minoru Osuka via James Dyer) + +* SOLR-9165: Spellcheck does not return collations if "maxCollationTries" is used with "cursorMark". + (James Dyer) + +* SOLR-8940: Fix group.sort option (hossman) + +* SOLR-8612: closing JDBC Statement on failures in DataImportHandler (DIH) (Kristine Jetzke via Mikhail Khludnev) + +* SOLR-8676: keep LOG4J_CONFIG in solr.cmd (Kristine Jetzke via Mikhail Khludnev) + +* SOLR-9198: config APIs unable to add multiple values with same name (noble) + +* SOLR-8812: edismax: turn off mm processing if no explicit mm spec is provided + and there are explicit operators (except for AND) - addresses problems caused by SOLR-2649. + (Greg Pendlebury, Jan Høydahl, Erick Erickson, Steve Rowe) + +* SOLR-9176: facet method ENUM was sometimes unnecessarily being rewritten to + FCS, causing slowdowns (Alessandro Benedetti, Jesse McLaughlin, Alan Woodward) + +Optimizations +---------------------- +* SOLR-8722: Don't force a full ZkStateReader refresh on every Overseer operation. + (Scott Blum via shalin) + +* SOLR-8745: Deprecate costly ZkStateReader.updateClusterState(), replace with a narrow + forceUpdateCollection(collection) (Scott Blum via shalin) + +* SOLR-8856: Do not cache merge or 'read once' contexts in the hdfs block cache. (Mark Miller, Mike Drob) + +* SOLR-8922: Optimize filter creation (DocSetCollector) to minimize the amount of garbage + produced. This resulted in up to 3x throughput when small filter creation was the bottleneck, + as well as orders of magnitude less garbage. (Jeff Wartes, yonik) + +* SOLR-8937: bin/post (SimplePostTool) now streams the standard input instead of buffering fully. + (David Smiley) + +* SOLR-8973: Zookeeper frenzy when a core is first created. (Janmejay Singh, Scott Blum, shalin) + +* SOLR-9014: Deprecate and reduce usage of ClusterState methods which may make calls to ZK via + the lazy collection reference. (Scott Blum, shalin) + +* SOLR-9106: Cluster properties are now cached on ZkStateReader. (Alan Woodward) + +* SOLR-9147: Upgrade commons-io to 2.5, avoid expensive array resizing in EmbeddedSolrServer (Mikhail Khludnev) + +* SOLR-8744: Overseer operations performed with fine grained mutual exclusion (noble, Scott Blum) + +* SOLR-9204: Improve performance of getting directory size with hdfs. (Mark Miller) + +Other Changes +---------------------- +* SOLR-8860: Remove back-compat handling of router format made in SOLR-4221 in 4.5.0. (shalin) + +* SOLR-8866: UpdateLog will now throw an exception if it doesn't know how to serialize a value. + (David Smiley) + +* SOLR-8842: security rules made more foolproof by asking the requesthandler about the well known + permission name.
 The APIs are also modified to ue 'index' as the unique identifier instead of name. + Name is an optional attribute
 now and only to be used when specifying well-known permissions (noble) + +* SOLR-5616: Simplifies grouping code to use ResponseBuilder.needDocList() to determine if it needs to + generate a doc list for grouped results. (Steven Bower, Keith Laban, Dennis Gove) + +* SOLR-8869: Optionally disable printing field cache entries in SolrFieldCacheMBean (Gregory Chanan) + +* SOLR-8892: Allow SolrInfoMBeans to return different statistics for /jmx vs web ui calls. + (Gregory Chanan, Mark Miller) + +* SOLR-8097: Implement builder pattern design for constructing SolrJ clients and also deprecate direct construction + of client objects. (Jason Gerlowski, Shawn Heisey, Anshum Gupta) + +* SOLR-9015: Adds SelectStream as a default function in the StreamHandler (Dennis Gove) + +* SOLR-8929: Add an idea module for solr/server to enable launching start.jar (Scott Blum, Steve Rowe) + +* SOLR-8933: Solr should not close container streams. (Mike Drob, Uwe Schindler, Mark Miller) + +* SOLR-9037: Replace multiple "/replication" strings with one static constant. (Christine Poerschke) + +* SOLR-9047: zkcli should allow alternative locations for log4j configuration (Gregory Chanan) + +* SOLR-9066: Make CountMetric return long instead of double (Kevin Risden) + +* SOLR-9065, SOLR-9072, SOLR-9132: Migrate some distributed tests to SolrCloudTestCase. (Alan Woodward) + +* SOLR-8184: Negative tests for JDBC Connection String (Susheel Kumar, Jason Gerlowski, Kevin Risden) + +* SOLR-8458: Add Streaming Expressions tests for parameter substitution (Joel Bernstein, Cao Manh Dat, Dennis Gove, Kevin Risden) + +* SOLR-8467: CloudSolrStream and FacetStream should take a SolrParams object rather than a + Map to allow more complex Solr queries to be specified. (Erick Erickson) + +* SOLR-9083: Remove all and from schemas. NOTE: as in the JIRA I left a few in to insure the (no cost) + back compat. (Erick Erickson) + +* SOLR-9105: Fix a bunch of typos across 103 files (Bartosz Krasiński via janhoy) + +* SOLR-9159: New cloud based concurrent atomic update test (hossman) + +* SOLR-9119: several static methods in ValueSourceParser have been made private (hossman) + +* SOLR-9110: Move JoinFromCollection- SubQueryTransformer- BlockJoinFacet- Distrib Tests to SolrCloudTestCase (Mikhail Khludnev) + +* SOLR-9136: Separate out the error statistics into server-side error vs client-side error + (Jessica Cheng Mallet via Erick Erickson) + +* SOLR-9107: new @RandomizeSSL annotation for more fine grained control of SSL testing (hossman, sarowe) + +* SOLR-9081: Make SolrTestCaseJ4.beforeClass() / .afterClass() public so it + works with Mockito (Georg Sorst, Alan Woodward) + +* SOLR-8445: fix line separator in log4j.properties files (Ahmet Arslan via Mikhail Khludnev) + +* SOLR-2199: DataImportHandler (DIH) JdbcDataSource supports multiple resultsets per query (Kristine Jetzke, Mark Waddle via Mikhail Khludnev) + +================== 6.0.1 ================== + +Upgrade Notes +---------------------- + +* If you use historical dates, specifically on or before the year 1582, you should re-index. + +Bug Fixes +---------------------- + +* SOLR-8914: ZkStateReader's refreshLiveNodes(Watcher) is not thread safe. (Scott Blum, hoss, + sarowe, Erick Erickson, Mark Miller, shalin) + +* SOLR-9016: Fix SolrIdentifierValidator to not allow empty identifiers. (Shai Erera) + +* SOLR-8992: Restore Schema API GET method functionality removed in 6.0 (noble, Steve Rowe) + +* SOLR-9080, SOLR-9085: (6.0 bug) For years <= 1582, date math (round,add,sub) introduced error. Range faceting + on such dates was also affected. With this fixed, this is the first release range faceting works on BC years. + (David Smiley) + +* SOLR-8857: HdfsUpdateLog does not use configured or new default number of version buckets and is + hard coded to 256. (Mark Miller, yonik, Gregory Chanan) + +* SOLR-8902: Make sure ReturnFields only returns the requested fields from (fl=) evn when + DocumentTransformers ask for getExtraRequestFields() (ryan) + +* SOLR-8875: SolrCloud Overseer clusterState could unexpectedly be null resulting in NPE. + (Scott Blum via David Smiley) + +* SOLR-8946: bin/post failed to detect stdin usage on Ubuntu; maybe other unixes. (David Smiley) + +* SOLR-9004: Fix "name" field type definition in films example. (Alexandre Rafalovitch via Varun Thacker) + +* SOLR-8990: Fix top term links from schema browser page to use {!term} parser (hossman) + +* SOLR-8971: Preserve root cause when wrapping exceptions (hossman) + +* SOLR-9034: Atomic updates failed to work when there were copyField targets that had docValues + enabled. (Karthik Ramachandran, Ishan Chattopadhyaya, yonik) + +* SOLR-9028: Fixed some test related bugs preventing SSL + ClientAuth from ever being tested (hossman) + +* SOLR-9059: NPE in SolrClientCache following collection reload (Joel Bernstein, Ryan Yacyshyn) + +* SOLR-8792: ZooKeeper ACL support fixed. (Esther Quansah, Ishan Chattopadhyaya, Steve Rowe) + +* SOLR-9030: The 'downnode' overseer command can trip asserts in ZkStateWriter. + (Scott Blum, Mark Miller, shalin) + +* SOLR-9036: Solr slave is doing full replication (entire index) of index after master restart. + (Lior Sapir, Mark Miller, shalin) + +* SOLR-9058: Makes HashJoinStream and OuterHashJoinStream support different field names in the + incoming streams, eg. fieldA=fieldB. (Dennis Gove, Stephan Osthold) + +* SOLR-9093: Fix NullPointerException in TopGroupsShardResponseProcessor. (Christine Poerschke) + +* SOLR-9118: HashQParserPlugin should trim partition keys (Joel Bernstein) + +* SOLR-9117: The first SolrCore is leaked after reload. (Jessica Cheng Mallet via shalin) + +* SOLR-9116: Race condition causing occasional SolrIndexSearcher leak when SolrCore is reloaded. + (Jessica Cheng Mallet via shalin) + +* SOLR-8801: /bin/solr create script always returns exit code 0 when a collection/core already exists. + (Khalid Alharbi, Marius Grama via Steve Rowe) + +* SOLR-9134: Fix RestManager.addManagedResource return value. (Christine Poerschke) + +Other Changes +---------------------- +* SOLR-7516: Improve javadocs for JavaBinCodec, ObjectResolver and enforce the single-usage policy. + (Jason Gerlowski, Benoit Vanalderweireldt, shalin) + +* SOLR-8967: In SolrCloud mode, under the 'Core Selector' dropdown in the UI the Replication tab won't be displayed + anymore. The Replication tab is only beneficial to users running Solr in master-slave mode. (Varun Thacker) + +* SOLR-8985: Added back support for 'includeDynamic' flag to /schema/fields endpoint (noble) + +* SOLR-9131: Fix "start solr" text in cluster.vm Velocity template (janhoy) + +* SOLR-9053: Upgrade commons-fileupload to 1.3.1, fixing a potential vulnerability (Jeff Field, Mike Drob via janhoy) + +* SOLR-9115: Get rid of javax.xml.bind.DatatypeConverter in SimplePostTool + for Java 9 compatibility. (Uwe Schindler) + +* SOLR-5776,SOLR-9068,SOLR-8970: + - Refactor SSLConfig so that SSLTestConfig can provide SSLContexts using a NullSecureRandom + to prevent SSL tests from blocking on entropy starved machines. + - SSLTestConfig: Alternate (psuedo random) NullSecureRandom for Constants.SUN_OS. + - SSLTestConfig: Replace NullSecureRandom w/ NotSecurePsuedoRandom. + - Change SSLTestConfig to use a keystore file that is included as a resource in the + test-framework jar so users subclassing SolrTestCaseJ4 don't need to preserve magic paths. + (hossman) + +================== 6.0.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release + +Versions of Major Components +--------------------- +Apache Tika 1.7 +Carrot2 3.12.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.3.8.v20160314 + +System Requirements +---------------------- +* LUCENE-5950: Move to Java 8 as minimum Java version. + (Ryan Ernst, Uwe Schindler) + +Upgrading from Solr 5.x +---------------------- + +* The deprecated SolrServer and subclasses have been removed, use SolrClient + instead. + +* The deprecated configuration in solrconfig.xml has been removed. + Please remove it from solrconfig.xml. + +* SolrClient.shutdown() has been removed, use SolrClient.close() instead. + +* The deprecated zkCredientialsProvider element in solrcloud section of solr.xml + is now removed. Use the correct spelling (zkCredentialsProvider) instead. + +* SOLR-7957: internal/expert - ResultContext was significantly changed and expanded + to allow for multiple full query results (DocLists) per Solr request. + TransformContext was rendered redundant and was removed. (yonik) + +* Several changes have been made regarding the "Similarity" used in Solr, in order to provide + better default behavior for new users. There are 3 key impacts of these changes on existing + users who upgrade: + * DefaultSimilarityFactory has been removed. If you currently have DefaultSimilarityFactory explicitly + referenced in your schema.xml, edit your config to use the functionally identical + ClassicSimilarityFactory. See SOLR-8239 for more details. + * The implicit default Similarity used when no is configured in schema.xml has + been changed to SchemaSimilarityFactory. Users who wish to preserve back-compatible behavior should + either explicitly configure ClassicSimilarityFactory, or ensure that the luceneMatchVersion + for the collection is less then 6.0. See SOLR-8270 + SOLR-8271 for details. + * SchemaSimilarityFactory has been modified to use BM25Similarity as the default for fieldTypes that + do not explicitly declare a Similarity. The legacy behavior of using ClassicSimilarity as the + default will occur if the luceneMatchVersion for the collection is less then 6.0, or the + 'defaultSimFromFieldType' configuration option may be used to specify any default of your choosing. + See SOLR-8261 + SOLR-8329 for more details. + +* If your solrconfig.xml file doesn't explicitly mention the schemaFactory to use then Solr will choose + the ManagedIndexSchemaFactory by default. Previously it would have chosen ClassicIndexSchemaFactory. + This means that the Schema APIs ( //schema ) are enabled and the schema is mutable. + When Solr starts your schema.xml file will be renamed to managed-schema. If you want to retain the old behaviour + then please ensure that the solrconfig.xml explicitly uses the ClassicIndexSchemaFactory : + or your luceneMatchVersion in the solrconfig.xml is less than 6.0 + +* SolrIndexSearcher.QueryCommand and QueryResult were moved to their own classes. If you reference them + in your code, you should import them under o.a.s.search (or use your IDE's "Organize Imports"). + +* SOLR-8698: 'useParams' attribute specified in request handler cannot be overridden from request params + +* When requesting stats in date fields, "sum" is now a double value instead of a date. See SOLR-8671 + +* SOLR-8736: The deprecated GET methods for schema are now accessible through the bulk API. The output + has less details and is not backward compatible. + +* In the past, Solr guaranteed that retrieval of multi-valued fields would preserve the order of values. + Because values may now be retrieved from column-stored fields (docValues="true"), in conjunction with the + fact that docValues do not currently preserve order, means that users should set useDocValuesAsStored="false" + to prevent future optizations from using the column-stored values over the row-stored values when + fields have both stored="true" and docValues="true". + +* Formatted date-times from Solr have some differences. If the year is more than 4 digits, there is a leading '+'. + When there is a non-zero number of milliseconds, it is padded with zeros to 3 digits. Negative year (BC) dates are + now possible. Parsing: It is now an error to supply a portion of the date out of its, range, like 67 seconds. + +* SolrJ no longer includes DateUtil. If for some reason you need to format or parse dates, simply use Instant.format() + and Instant.parse(). + +* If you are using an RPT or other spatial field referencing Spatial4j in its configuration, then replace the string + "com.spatial4j.core" with "org.locationtech.spatial4j". Consider updating direct to Solr 6.2 which accepts the + old value, albeit with a warning. + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-3085: New edismax param mm.autoRelax which helps in certain cases of the stopwords/zero-hits issue (janhoy) + +* SOLR-7560: Parallel SQL Support (Joel Bernstein) + +* SOLR-7707: Add StreamExpression Support to RollupStream (Dennis Gove, Joel Bernstein) + +* SOLR-7903: Add the FacetStream to the Streaming API and wire it into the SQLHandler (Joel Bernstein) + +* SOLR-7986: JDBC Driver for SQL Interface (Uwe Schindler, Joel Bernstein) + +* SOLR-8038: Add the StatsStream to the Streaming API and wire it into the SQLHandler (Joel Bernstein) + +* SOLR-8086: Add support for SELECT DISTINCT queries to the SQL interface (Joel Bernstein) + +* SOLR-7543: Basic graph traversal query + Example: {!graph from="node_id" to="edge_id"}id:doc_1 + (Kevin Watters, yonik) + +* SOLR-6273: Cross Data Center Replication. Active/passive replication for separate + SolrClouds hosted on separate data centers. (Renaud Delbru, Yonik Seeley via Erick Erickson) + +* SOLR-7938: MergeStream now supports merging more than 2 streams together (Dennis Gove) + +* SOLR-8198: Change ReducerStream to use StreamEqualitor instead of StreamComparator (Dennis Gove) + +* SOLR-8268: StatsStream now implements the Expressible interface (Dennis Gove) + +* SOLR-7584: Adds Inner and LeftOuter Joins to the Streaming API and Streaming Expressions (Dennis Gove, Corey Wu) + +* SOLR-8188: Adds Hash and OuterHash Joins to the Streaming API and Streaming Expressions (Dennis Gove) + +* SOLR-7669: Add SelectStream and Tuple Operations to the Streaming API and Streaming Expressions (Dennis Gove) + +* SOLR-8337: Add ReduceOperation and wire it into the ReducerStream (Joel Bernstein) + +* SOLR-7904: Add StreamExpression Support to FacetStream (Dennis Gove) + +* SOLR-6398: Add IterativeMergeStrategy to support running Parallel Iterative Algorithms inside of Solr + (Joel Bernstein) + +* SOLR-8436: Real-time get now supports filters. (yonik) + +* SOLR-7535: Add UpdateStream to Streaming API and Streaming Expression (Jason Gerlowski, Joel Bernstein) + +* SOLR-8479: Add JDBCStream to Streaming API and Streaming Expressions for integration with external data sources + (Dennis Gove) + +* SOLR-8002: Add column alias support to the Parallel SQL Interface (Joel Bernstein) + +* SOLR-7525: Add ComplementStream and IntersectStream to the Streaming API and Streaming Expressions + (Dennis Gove, Jason Gerlowski, Joel Bernstein) + +* SOLR-8415: Provide command to switch between non/secure mode in ZK + (Mike Drob, Gregory Chanan) + +* SOLR-8556: Add ConcatOperation to be used with the SelectStream (Joel Bernstein, Dennis Gove) + +* SOLR-8550: Add asynchronous DaemonStreams to the Streaming API (Joel Bernstein) + +* SOLR-8285: Ensure the /export handler works with NULL field values (Joel Bernstein) + +* SOLR-8502: Improve Solr JDBC Driver to support SQL Clients like DBVisualizer (Kevin Risden, Joel Bernstein) + +* SOLR-8588: Add TopicStream to the streaming API to support publish/subscribe messaging (Joel Bernstein, Kevin Risden) + +* SOLR-8666: Adds header 'zkConnected' to response of SearchHandler and PingRequestHandler to notify the client when + a connection to zookeeper has been lost and there is a possibility of stale data on the node the request is coming + from. (Keith Laban, Dennis Gove) + +* SOLR-8522: Make it possible to use ip fragments in replica placement rules , such as ip_1, ip_2 etc (Arcadius Ahouansou, noble) + +* SOLR-8698: params.json can now specify 'appends' and 'invariants' (noble) + +* SOLR-8831: allow _version_ field to be retrievable via docValues (yonik) + + +Bug Fixes +---------------------- +* SOLR-8386: Add field option in the new admin UI schema page loads up even when no schemaFactory has been + explicitly specified since the default is ManagedIndexSchemaFactory. (Erick Erickson, Upayavira, Varun Thacker) + +* SOLR-8191: Guard against CloudSolrStream close method NullPointerException + (Kevin Risden, Joel Bernstein) + +* SOLR-8485: SelectStream now properly handles non-lowercase and/or quoted select field names (Dennis Gove) + +* SOLR-8525: Fix a few places that were failing to pass dimensional + values settings when copying a FieldInfo (Ishan Chattopadhyaya via + Mike McCandless) + +* SOLR-8409: Ensures that quotes in solr params (eg. q param) are properly handled (Dennis Gove) + +* SOLR-8640: CloudSolrClient does not send credentials for update request (noble, hoss) + +* SOLR-8461: CloudSolrStream and ParallelStream can choose replicas that are not active + (Cao Manh Dat, Varun Thacker, Joel Bernstein) + +* SOLR-8527: Improve JdbcTest to cleanup properly on failures (Kevin Risden, Joel Bernstein) + +* SOLR-8578: Successful or not, requests are not always fully consumed by Solrj clients and we + count on HttpClient or the JVM. (Mark Miller) + +* SOLR-8683: Always consume the full request on the server, not just in the case of an error. + (Mark Miller) + +* SOLR-8416: The collections create API should return after all replicas are active. + (Michael Sun, Mark Miller, Alexey Serba) + +* SOLR-8701: CloudSolrClient decides that there are no healthy nodes to handle a request too early. + (Mark Miller) + +* SOLR-8694: DistributedMap/Queue can create too many Watchers and some code simplification. + (Scott Blum via Mark Miller) + +* SOLR-8695: Ensure ZK watchers are not triggering our watch logic on connection events and + make this handling more consistent. (Scott Blum via Mark Miller) + +* SOLR-8633: DistributedUpdateProcess processCommit/deleteByQuery call finish on DUP and + SolrCmdDistributor, which violates the lifecycle and can cause bugs. (hossman via Mark Miller) + +* SOLR-8656: PeerSync should use same nUpdates everywhere. (Ramsey Haddad via Mark Miller) + +* SOLR-8697, SOLR-8837: Scope ZK election nodes by session to prevent elections from interfering with each other + and other small LeaderElector improvements. (Scott Blum via Mark Miller, Alan + Woodward) + +* SOLR-8599: After a failed connection during construction of SolrZkClient attempt to retry until a connection + can be made. (Keith Laban, Dennis Gove) + +* SOLR-8497: Merge index does not mark the Directory objects it creates as 'done' and they are retained in the + Directory cache. (Sivlio Sanchez, Mark Miller) + +* SOLR-8696: Start the Overseer before actions that need the overseer on init and when reconnecting after + zk expiration and improve init logic. (Scott Blum, Mark Miller) + +* SOLR-8420: Fix long overflow in sumOfSquares for Date statistics. (Tom Hill, Christine Poerschke, + Tomás Fernández Löbbe) + +* SOLR-8748: OverseerTaskProcessor limits number of concurrent tasks to just 10 even though the thread pool + size is 100. The limit has now been increased to 100. (Scott Blum, shalin) + +* SOLR-8375: ReplicaAssigner rejects valid nodes (Kelvin Tan, noble) + +* SOLR-8738: Fixed false success response when invalid deleteByQuery requests intially hit non-leader + cloud nodes (hossman) + +* SOLR-8771: Multi-threaded core shutdown creates executor per core. (Mike Drob via Mark Miller) + +* SOLR-8145: Fix position of OOM killer script when starting Solr in the background (Jurian Broertjes via + Timothy Potter) + +* SOLR-8769: Fix document exclusion in mlt query parser in Cloud mode for schemas that have non-"id" + unique field (Erik Hatcher, Anshum Gupta) + +* SOLR-8728: ReplicaAssigner throws NPE when a partial list of nodes are only participating in replica + placement. splitshard should preassign nodes using rules, if rules are present (noble, Shai Erera) + +* SOLR-8779: Fix missing InterruptedException handling in ZkStateReader.java (Varun Thacker) + +* SOLR-8449: Fix the core restore functionality to allow restoring multiple times on the same core + (Johannes Brucher, Varun Thacker) + +* SOLR-8155: JSON Facet API - field faceting on a multi-valued string field without + docValues (i.e. UnInvertedField implementation), but with a prefix or with a sort + other than count, resulted in incorrect results. This has been fixed, and facet.prefix + support for facet.method=uif has been enabled. (Mikhail Khludnev, yonik) + +* SOLR-8790: Collections API responses contain node name in the core-level responses that are + returned. (Anshum Gupta) + +* SOLR-8804: Fix a race condition in the ClusterStatus API call whereby the call would fail when a concurrent delete + collection api command was executed (Alexey Serba, Varun Thacker) + +* SOLR-8835: JSON Facet API: fix faceting exception on multi-valued numeric fields that + have docValues. (yonik) + +* SOLR-8838: Returning non-stored docValues is incorrect for negative floats and doubles. + (Ishan Chattopadhyaya, Steve Rowe) + +* SOLR-8867: {!frange} queries will now avoid matching documents without a value in the + numeric field. For more complex functions, FunctionValues.exists() must also return true + for the document to match. (yonik) + +* SOLR-8886: Fix TrieField.toObject(IndexableField) to work for field with docValues + enabled. (yonik) + +* SOLR-8891: Fix StrField.toObject and toExternal to work with docValue IndexableField + instances. (yonik) + +* SOLR-8865: Real-time get sometimes fails to retrieve stored fields from docValues. + (Ishan Chattopadhyaya, yonik) + +Optimizations +---------------------- +* SOLR-7876: Speed up queries and operations that use many terms when timeAllowed has not been + specified. Speedups of up to 8% were observed. (yonik) + +* SOLR-8037: Speed up creation of filters from term range queries (i.e. non-numeric range queries) + and use the filter cache for term range queries that are part of larger queries. Some observed + speedups were up to 2.5x for production of filters, and up to 10x for query evaluation with + embedded term range queres that resulted in filter cache hits. (yonik) + +* SOLR-8559: FCS facet performance optimization which significantly speeds up processing when terms + are high cardinality and the matching docset is small. When facet minCount > 0 and the number of + matching documents is small (or 0) this enhancement prevents considering terms which have a 0 + count. Also includes change to move to the next non-zero term value when selecting a segment + position. (Keith Laban, Steve Bower, Dennis Gove) + +* SOLR-8532: Optimize GraphQuery when maxDepth is set by not collecting edges at the maxDepth level. + (Kevin Watters via yonik) + +* SOLR-8669: Non binary responses use chunked encoding because we flush the outputstream early. + (Mark Miller) + +* SOLR-8720: ZkController#publishAndWaitForDownStates should use #publishNodeAsDown. (Mark Miller) + +* SOLR-8082: Can't query against negative float or double values when indexed="false" + docValues="true" multiValued="false". (hossman, Ishan Chattopadhyaya, yonik, Steve Rowe) + +Other Changes +---------------------- + +* SOLR-6127: Improve example docs, using films data (Varun Thacker via ehatcher) + +* SOLR-6895: Deprecated SolrServer classes have been removed (Alan Woodward, + Erik Hatcher) + +* SOLR-6954: Deprecated SolrClient.shutdown() method removed (Alan Woodward) + +* SOLR-7355: Switch from Google's ConcurrentLinkedHashMap to Caffeine. Only + affects HDFS support. (Ben Manes via Shawn Heisey) + +* SOLR-7624: Remove deprecated zkCredientialsProvider element in solrcloud section of solr.xml. + (Xu Zhang, Per Steffensen, Ramkumar Aiyengar, Mark Miller) + +* SOLR-7513: Add Equalitors to Streaming Expressions (Dennis Gove, Joel Bernstein) + +* SOLR-7528: Simplify Interfaces used in Streaming Expressions (Dennis Gove, Joel Bernstein) + +* SOLR-7554: Add checks in Streams for incoming stream order (Dennis Gove, Joel Bernstein) + +* SOLR-7441: Improve overall robustness of the Streaming stack: Streaming API, + Streaming Expressions, Parallel SQL (Joel Bernstein) + +* SOLR-8153: Support upper case and mixed case column identifiers in the SQL interface + (Joel Bernstein) + +* SOLR-8132: HDFSDirectoryFactory now defaults to using the global block cache. (Mark Miller) + +* SOLR-8261: Change SchemaSimilarityFactory default to BM25Similarity (hossman) + +* SOLR-8259: Remove deprecated JettySolrRunner.getDispatchFilter() + +* SOLR-8258: Change default hdfs tlog replication factor from 1 to 3. (Mark Miller) + +* SOLR-8270: Change implicit default Similarity to use BM25 when luceneMatchVersion >= 6 (hossman) + +* SOLR-8271: Change implicit default Similarity to use SchemaSimilarityFactory when luceneMatchVersion >= 6 (hossman) + +* SOLR-8179: SQL JDBC - DriverImpl loadParams doesn't support keys with no values in the connection string + (Kevin Risden, Joel Bernstein) + +* SOLR-8131: Make ManagedIndexSchemaFactory the default schemaFactory when luceneMatchVersion >= 6 + (Uwe Schindler, shalin, Varun Thacker) + +* SOLR-8266: Remove Java Serialization from the Streaming API. The /stream handler now only accepts + Streaming Expressions. (Jason Gerlowski, Joel Bernstein) + +* SOLR-8426: Enable /export, /stream and /sql handlers by default and remove them from example configs. (shalin) + +* SOLR-8443: Change /stream handler http param from "stream" to "expr" (Joel Bernstein, Dennis Gove) + +* SOLR-5209: Unloading or deleting the last replica of a shard now no longer + cascades to remove the shard from the clusterstate. (Christine Poerschke) + +* SOLR-8190: Implement Closeable on TupleStream (Kevin Risden, Joel Bernstein) + +* SOLR-8529: Improve JdbcTest to not use plain assert statements (Kevin Risden, Joel Bernstein) + +* SOLR-7339: Upgrade Jetty to v9.3.8.v20160314. (Gregg Donovan, shalin, Mark Miller, Steve Rowe) + +* SOLR-5730: Make Lucene's SortingMergePolicy and EarlyTerminatingSortingCollector configurable in Solr. + (Christine Poerschke, hossmann, Tomás Fernández Löbbe, Shai Erera) + +* SOLR-8677: Prevent shards containing invalid characters from being created. Checks added server-side + and in SolrJ. (Shai Erera, Jason Gerlowski, Anshum Gupta) + +* SOLR-8693: Improve ZkStateReader logging. (Scott Blum via Mark Miller) + +* SOLR-8710: Upgrade morfologik-stemming to version 2.1.0. (Dawid Weiss) + +* SOLR-8711: Upgrade Carrot2 clustering dependency to 3.12.0. (Dawid Weiss) + +* SOLR-8690: Make peersync fingerprinting optional with solr.disableFingerprint system + property. (yonik) + +* SOLR-8691: Cache index fingerprints per searcher. (yonik) + +* SOLR-8746: Renamed Overseer.getInQueue to getStateUpdateQueue, getInternalQueue to getInternalWorkQueue + and added javadocs. (Scott Blum, shalin) + +* SOLR-8752: Add a test for SizeLimitedDistributedMap and improve javadocs. (shalin) + +* SOLR-8671: Date statistics: make "sum" a double instead of a long/date (Tom Hill, Christine Poerschke, + Tomás Fernández Löbbe) + +* SOLR-8713: new UI and example solrconfig files point to Reference Guide for Solr Query Syntax instead + of the wiki. (Marius Grama via Tomás Fernández Löbbe) + +* SOLR-8758: Add a new SolrCloudTestCase class, using MiniSolrCloudCluster (Alan + Woodward) + +* SOLR-8736: schema GET operations on fields, dynamicFields, fieldTypes, copyField are + reimplemented as a part of the bulk API with less details (noble) + +* SOLR-8766: Remove deprecated tag in solrconfig.xml and support for admin/gettableFiles + (noble, Jason Gerlowski, Varun Thacker) + +* SOLR-8799: Improve error message when tuple can't be read by SolrJ JDBC (Kevin Risden, Joel Bernstein) + +* SOLR-8836: Return 400, and a SolrException when an invalid json is provided to the update handler + instead of 500. (Jason Gerlowski via Anshum Gupta) + +* SOLR-8740: docValues are now enabled by default for most non-text (string, date, and numeric) fields + in the schema templates. (yonik) + +* SOLR-8819: Implement DatabaseMetaDataImpl getTables() and fix getSchemas(). (Trey Cahill, + Joel Bernstein, Kevin Risden) + +* SOLR-8810: Implement Connection.setReadOnly, Statement.set/getFetchSize, + ResultSet.getType (Kevin Risden) + +* SOLR-8904: All dates are formatted via Instant.toString() (ISO-8601); see Solr upgrade notes for differences. Will + now parse (and format) dates with a leading '+' or '-' (BC dates or dates > 4 digit year. + [value] and ms() and contrib/analytics now parse with date math. (David Smiley) + +* SOLR-8904: DateUtil in SolrJ moved to the extraction contrib as ExtractionDateUtil. Obsolete methods were removed. + (David Smiley) + +======================= 5.5.3 ======================= + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.12.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.3.8.v20160314 + + +(No Changes) + + ======================= 5.5.2 ======================= +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.7 +Carrot2 3.10.4 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.2.13.v20150730 + Bug Fixes +--------------------- + +* SOLR-8695: Ensure ZK watchers are not triggering our watch logic on connection events and + make this handling more consistent. (Scott Blum via Mark Miller) * SOLR-9198: config APIs unable to add multiple values with same name (noble) -* SOLR-9234: srcField works only when all fields are captured in the /update/json/docs - endpoint (noble) +* SOLR-9191: OverseerTaskQueue.peekTopN() fatally flawed (Scott Blum, Noble Paul) + +* SOLR-8812: edismax: turn off mm processing if no explicit mm spec is provided + and there are explicit operators (except for AND) - addresses problems caused by SOLR-2649. + (Greg Pendlebury, Jan Høydahl, Erick Erickson, Steve Rowe) + +* SOLR-9034: Atomic updates failed to work when there were copyField targets that had docValues + enabled. (Karthik Ramachandran, Ishan Chattopadhyaya, yonik) + +* SOLR-8940: Fix group.sort option (hossman) + +* SOLR-8857: HdfsUpdateLog does not use configured or new default number of version buckets and is + hard coded to 256. (Mark Miller, yonik, Gregory Chanan) + +* SOLR-8875: SolrCloud Overseer clusterState could unexpectedly be null resulting in NPE. + (Scott Blum via David Smiley) + +* SOLR-8946: bin/post failed to detect stdin usage on Ubuntu; maybe other unixes. (David Smiley) + +* SOLR-9004: Fix "name" field type definition in films example. (Alexandre Rafalovitch via Varun Thacker) + +* SOLR-8990: Fix top term links from schema browser page to use {!term} parser (hossman) + +* SOLR-8971: Preserve root cause when wrapping exceptions (hossman) + +* SOLR-8792: ZooKeeper ACL support fixed. (Esther Quansah, Ishan Chattopadhyaya, Steve Rowe) + +* SOLR-9030: The 'downnode' overseer command can trip asserts in ZkStateWriter. + (Scott Blum, Mark Miller, shalin) + +* SOLR-9036: Solr slave is doing full replication (entire index) of index after master restart. + (Lior Sapir, Mark Miller, shalin) + +* SOLR-9093: Fix NullPointerException in TopGroupsShardResponseProcessor. (Christine Poerschke) + +* SOLR-9118: HashQParserPlugin should trim partition keys (Joel Bernstein) + +* SOLR-9117: The first SolrCore is leaked after reload. (Jessica Cheng Mallet via shalin) + +* SOLR-9116: Race condition causing occasional SolrIndexSearcher leak when SolrCore is reloaded. + (Jessica Cheng Mallet via shalin) + +* SOLR-8801: /bin/solr create script always returns exit code 0 when a collection/core already exists. + (Khalid Alharbi, Marius Grama via Steve Rowe) + +* SOLR-9134: Fix RestManager.addManagedResource return value. (Christine Poerschke) + +* SOLR-9151: Fix SolrCLI so that bin/solr -e cloud example can be run from any CWD (janhoy) + +* SOLR-9165: Spellcheck does not return collations if "maxCollationTries" is used with "cursorMark". + (James Dyer) + +* SOLR-8612: closing JDBC Statement on failures in DataImportHandler (DIH) (Kristine Jetzke via Mikhail Khludnev) + +* SOLR-8676: keep LOG4J_CONFIG in solr.cmd (Kristine Jetzke via Mikhail Khludnev) + +* SOLR-9176: facet method ENUM was sometimes unnecessarily being rewritten to + FCS, causing slowdowns (Alessandro Benedetti, Jesse McLaughlin, Alan Woodward) + +Other Changes +---------------------- + +* SOLR-7516: Improve javadocs for JavaBinCodec, ObjectResolver and enforce the single-usage policy. + (Jason Gerlowski, Benoit Vanalderweireldt, shalin) + +* SOLR-8967: In SolrCloud mode, under the 'Core Selector' dropdown in the UI the Replication tab won't be displayed + anymore. The Replication tab is only beneficial to users running Solr in master-slave mode. (Varun Thacker) + +* SOLR-9131: Fix "start solr" text in cluster.vm Velocity template (janhoy) + +* SOLR-9053: Upgrade commons-fileupload to 1.3.1, fixing a potential vulnerability (Jeff Field, Mike Drob via janhoy) + +* SOLR-8866: UpdateLog will now throw an exception if it doesn't know how to serialize a value. + (David Smiley) + +* SOLR-8933: Solr should not close container streams. (Mike Drob, Uwe Schindler, Mark Miller) + +* SOLR-9037: Replace multiple "/replication" strings with one static constant. (Christine Poerschke) + +* SOLR-9047: zkcli should allow alternative locations for log4j configuration (Gregory Chanan) + +* SOLR-9105: Fix a bunch of typos across 103 files (Bartosz Krasiński via janhoy) + +* SOLR-8445: fix line separator in log4j.properties files (Ahmet Arslan via Mikhail Khludnev) + +* SOLR-8674: Stop ignoring sysprop solr.tests.mergePolicy, and make tests randomly choose between + setting and , which was added in SOLR-8621. (Christine Poerschke) ======================= 5.5.1 ======================= @@ -302,9 +2038,6 @@ New Features * SOLR-8560: Added RequestStatusState enum which can be used when comparing states of asynchronous requests. (Shai Erera) -* SOLR-8415: Provide command to switch between non/secure mode in ZK - (Mike Drob, Gregory Chanan) - * SOLR-8586: added index fingerprint, a hash over all versions currently in the index. PeerSync now uses this to check if replicas are in sync. (yonik) @@ -423,31 +2156,27 @@ Bug Fixes * SOLR-8607: The Schema API refuses to add new fields that match existing dynamic fields. (Jan Høydahl, Steve Rowe) -* SOLR-8640: CloudSolrClient does not send credentials for update request (noble, hoss) - +* SOLR-8575: Fix HDFSLogReader replay status numbers, a performance bug where we can reopen + FSDataInputStream much too often, and an hdfs tlog data integrity bug. + (Mark Miller, Patrick Dvorack, yonik) + * SOLR-8651: The commitWithin parameter is not passed on for deleteById in UpdateRequest in distributed queries (Jessica Cheng Mallet via Erick Erickson) * SOLR-8551: Make collection deletion more robust. (Mark Miller) - + Optimizations ---------------------- * SOLR-8501: Specify the entity request size when known in HttpSolrClient. (Mark Miller) -* SOLR-8559: FCS facet performance optimization which significantly speeds up processing when terms - are high cardinality and the matching docset is small. When facet minCount > 0 and the number of - matching documents is small (or 0) this enhancement prevents considering terms which have a 0 - count. Also includes change to move to the next non-zero term value when selecting a segment - position. (Keith Laban, Steve Bower, Dennis Gove) - * SOLR-8615: Just like creating cores, we should use multiple threads when closing cores. (Mark Miller) * SOLR-7281: Add an overseer action to publish an entire node as 'down'. (Mark Miller, shalin) -* SOLR-8082: Can't query against negative float or double values when indexed="false" - docValues="true" multiValued="false". (hossman, Ishan Chattopadhyaya, yonik, Steve Rowe) +* SOLR-8669: Non binary responses use chunked encoding because we flush the outputstream early. + (Mark Miller) Other Changes ---------------------- @@ -465,7 +2194,7 @@ Other Changes (Christine Poerschke) * SOLR-8338: in OverseerTest replace strings such as "collection1" and "state" with variable - or enum equivalent. (Christine Poerschke) + or enum equivalent (Christine Poerschke) * SOLR-8333: Several API tweaks so that public APIs were no longer refering to private classes (ehatcher, Shawn Heisey, hossman) @@ -538,8 +2267,6 @@ Other Changes * SOLR-8455: RecovertStrategy logging improvements and sleep-between-recovery-attempts bug fix. (Shai Erera) -* SOLR-8476: Refactor and cleanup CoreAdminHandler (noble, Varun THacker) - * SOLR-8481: TestSearchPerf no longer needs to duplicate SolrIndexSearcher.(NO_CHECK_QCACHE|NO_CHECK_FILTERCACHE) (Christine Poerschke) @@ -567,6 +2294,8 @@ Other Changes * SOLR-7042: bin/post now uses /update/json/docs for application/json content types, including support for .jsonl (JSON Lines) files. (Erik Hatcher and shalin) +* SOLR-8476: Refactor and cleanup CoreAdminHandler (noble, Varun Thacker) + * SOLR-8535: Support forcing define-lucene-javadoc-url to be local (Gregory Chanan) * SOLR-8549: Solr start script checks for cores which have failed to load as well before attempting to @@ -629,15 +2358,11 @@ Bug Fixes * SOLR-8418: Adapt to changes in LUCENE-6590 for use of boosts with MLTHandler and Simple/CloudMLTQParser (Jens Wille, Ramkumar Aiyengar) -* SOLR-8561: Doing a rolling upgrade to 5.4.0 fails because the new nodes cannot find the - shard leader properties where they expect. This bug was first introduced in 5.4.0 and a - fallback was added so that the leader properties are searched in both the old location - and the new one. (Shai Erera) - New Features ---------------------- * SOLR-8470: Make TTL of PKIAuthenticationPlugin's tokens configurable through a system property + (pkiauth.ttl) (noble) ================== 5.4.0 ================== @@ -917,7 +2642,6 @@ Bug Fixes * SOLR-8355: update permissions were failing node recovery (noble , Anshum Gupta) - Optimizations ---------------------- @@ -1244,11 +2968,11 @@ New Features * SOLR-7742: Support for Immutable ConfigSets (Gregory Chanan) -* SOLR-6234: Scoring for query time join (Mikhail Khludnev) - * SOLR-2522: new two argument option for the existing field() function; picks the min/max value of a docValues field to use as a ValueSource: "field(field_name,min)" and "field(field_name,max)" (hossman) +* SOLR-6234: Scoring for query time join (Mikhail Khludnev) + * SOLR-5882: score local parameter for block join query parser {!parent} (Andrey Kudryavtsev, Mikhail Khludnev) * SOLR-7799: Added includeIndexFieldFlags (backwards compatible default is true) to /admin/luke. @@ -1256,6 +2980,8 @@ New Features * SOLR-7769: Add bin/post -p alias for -port parameter. (ehatcher) +* SOLR-7766: support creation of a coreless collection via createNodeSet=EMPTY (Christine Poerschke) + * SOLR-7849: Solr-managed inter-node authentication when authentication enabled (Noble Paul) * SOLR-7220: Nested C-style comments in queries. (yonik) @@ -1270,7 +2996,6 @@ New Features * SOLR-7837: An AuthenticationPlugin which implements the HTTP BasicAuth protocol and stores credentials securely in ZooKeeper (Noble Paul, Anshum Gupta,Ishan Chattopadhyaya) -* SOLR-7766: support creation of a coreless collection via createNodeSet=EMPTY (Christine Poerschke) Bug Fixes ---------------------- @@ -1455,8 +3180,6 @@ Other Changes * SOLR-7680: Use POST instead of GET when finding versions for mismatches with CloudInspectUtil for tests (Ramkumar Aiyengar) -* SOLR-7662: Refactored response writing to consolidate the logic in one place (Noble Paul) - * SOLR-7665: deprecate the class TransformerWithContext (noble