lucene-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ctarg...@apache.org
Subject [lucene-solr] branch branch_8x updated: Ref Guide: Upgrade notes for Solr 8.1
Date Tue, 21 May 2019 18:08:26 GMT
This is an automated email from the ASF dual-hosted git repository.

ctargett pushed a commit to branch branch_8x
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git


The following commit(s) were added to refs/heads/branch_8x by this push:
     new 1d3d635  Ref Guide: Upgrade notes for Solr 8.1
1d3d635 is described below

commit 1d3d635439e5c820898287ab9dff8a6492d32056
Author: Cassandra Targett <ctargett@apache.org>
AuthorDate: Tue May 21 12:51:20 2019 -0500

    Ref Guide: Upgrade notes for Solr 8.1
---
 solr/solr-ref-guide/src/solr-upgrade-notes.adoc | 256 ++++--------------------
 1 file changed, 36 insertions(+), 220 deletions(-)

diff --git a/solr/solr-ref-guide/src/solr-upgrade-notes.adoc b/solr/solr-ref-guide/src/solr-upgrade-notes.adoc
index eb0aa3d..fedc6dc 100644
--- a/solr/solr-ref-guide/src/solr-upgrade-notes.adoc
+++ b/solr/solr-ref-guide/src/solr-upgrade-notes.adoc
@@ -1,6 +1,7 @@
 = Solr Upgrade Notes
 :page-children: major-changes-in-solr-8, major-changes-in-solr-7, major-changes-from-solr-5-to-solr-6
 :page-toclevels: 3
+:page-tocclass: right
 // Licensed to the Apache Software Foundation (ASF) under one
 // or more contributor license agreements.  See the NOTICE file
 // distributed with this work for additional information
@@ -28,260 +29,75 @@ Detailed steps for upgrading a Solr cluster are in the section <<upgrading-a-sol
 
 == Upgrading to 8.x Releases
 
-=== Solr 8.1
-
-*Logging*
-
-Solr 8.1 changed the default log4j2 logging mode from synchronous to asynchronous. This will
improve logging throughput and reduce system contention at the cost of a _slight_ chance that
some logging messages may be missed in the event of abnormal Solr termination.
-
-If even this slight risk is unacceptable, the log4j configuration file ../server/resources/log4j2.xml
has the synchronous logging configuration in a commented section and can be edited to re-enable
synchronous logging.
-
-== Upgrading to 7.x Releases
-
-=== Solr 7.7
-
-See the https://wiki.apache.org/solr/ReleaseNote77[7.7 Release Notes] for an overview of
the main new features in Solr 7.7.
-
-When upgrading to Solr 7.7.x, users should be aware of the following major changes from v7.6:
-
-*Admin UI*
-
-* The Admin UI now presents a login screen for any users with authentication enabled on their
cluster.  Clusters with <<basic-authentication-plugin.adoc#basic-authentication-plugin,Basic
Authentication>> will prompt users to enter a username and password.  On clusters configured
to use <<kerberos-authentication-plugin.adoc#kerberos-authentication-plugin,Kerberos
Authentication>>, users will be directed to configure their browser to provide an appropriate
Kerberos ticket.
-+
-The login screen's purpose is cosmetic only - Admin UI-triggered Solr requests were subject
to authentication prior to 7.7 and still are today.  The login screen changes only the user
experience of providing this authentication.
+If you are upgrading from 7.x, see the section <<Upgrading from 7.x Releases>>
below.
 
-*Distributed Requests*
-
-* The `shards` parameter, used to manually select the shards and replicas that receive distributed
requests, now checks nodes against a whitelist of acceptable values for security reasons.
-+
-In SolrCloud mode this whitelist is automatically configured to contain all live nodes. 
In standalone mode the whitelist is empty by default.  Upgrading users who use the `shards`
parameter in standalone mode can correct this value by setting the `shardsWhitelist` property
in any `shardHandler` configurations in their `solrconfig.xml` file.
-+
-For more information, see the <<distributed-requests.adoc#configuring-the-shardhandlerfactory,Distributed
Request>> documentation.
-
-=== Solr 7.6
-
-See the https://wiki.apache.org/solr/ReleaseNote76[7.6 Release Notes] for an overview of
the main new features in Solr 7.6.
-
-When upgrading to Solr 7.6, users should be aware of the following major changes from v7.5:
-
-*Collections*
-
-* The JSON parameter to set cluster-wide default cluster properties with the <<collections-api.adoc#clusterprop,CLUSTERPROP>>
command has changed.
-+
-The old syntax nested the defaults into a property named `clusterDefaults`. The new syntax
uses only `defaults`. The command to use is still `set-obj-property`.
-+
-An example of the new syntax is:
-+
-[source,json]
-----
-{
-  "set-obj-property": {
-    "defaults" : {
-      "collection": {
-        "numShards": 2,
-        "nrtReplicas": 1,
-        "tlogReplicas": 1,
-        "pullReplicas": 1
-      }
-    }
-  }
-}
-----
-+
-The old syntax will be supported until at least Solr 9, but users are advised to begin using
the new syntax as soon as possible.
-
-* The parameter `min_rf` has been deprecated and no longer needs to be provided in order
to see the achieved replication factor. This information will now always be returned to the
client with the response.
+=== Solr 8.1
 
-*Autoscaling*
+See the https://wiki.apache.org/solr/ReleaseNote810[8.1 Release Notes] for an overview of
the main new features of Solr 8.1.
 
-* An autoscaling policy is now used as the default strategy for selecting nodes on which
new replicas or replicas of new collections are created.
-+
-A default policy is now in place for all users, which will sort nodes by the number of cores
and available freedisk, which means by default a node with the fewest number of cores already
on it and the highest available freedisk will be selected for new core creation.
+When upgrading to 8.1.x, users should be aware of the following major changes from v8.0.
 
-* The change described above has two additional impacts on the `maxShardsPerNode` parameter:
+*Global maxBooleanClauses Parameter*
 
-. It removes the restriction against using `maxShardsPerNode` when an autoscaling policy
is in place. This parameter can now always be set when creating a collection.
-. It removes the default setting of `maxShardsPerNode=1` when an autoscaling policy is in
place. It will be set correctly (if required) regardless of whether an autoscaling policy
is in place or not.
+* The behavior of the `maxBooleanClauses` parameter has changed to reduce the risk of exponential
query expansion when dealing with pathological query strings.
 +
-The default value of `maxShardsPerNode` is still `1`. It can be set to `-1` if the old behavior
of unlimited `maxSharedsPerNode` is desired.
-
-*DirectoryFactory*
-
-* Lucene has introduced the `ByteBuffersDirectoryFactory` as a replacement for the `RAMDirectoryFactory`,
which will be removed in Solr 9.
+A default upper limit of 1024 clauses is now enforced at the node level. This was the default
prior to 7.0, and it can be overridden with a new global parameter in `solr.xml`. This limit
will be enforced for all queries whether explicitly defined by the user (or client), or created
by Solr and Lucene internals.
 +
-While most users are still encouraged to use the `NRTCachingDirectoryFactory`, which allows
Lucene to select the best directory factory to use, if you have explicitly configured Solr
to use the `RAMDirectoryFactory`, you are encouraged to switch to the new implementation as
soon as possible before Solr 9 is released.
-+
-For more information about the new directory factory, see the Jira issue https://issues.apache.org/jira/browse/LUCENE-8438[LUCENE-8438].
-+
-For more information about the directory factory configuration in Solr, see the section <<datadir-and-directoryfactory-in-solrconfig.adoc#datadir-and-directoryfactory-in-solrconfig,DataDir
and DirectoryFactory in SolrConfig>>.
-
-=== Solr 7.5
-
-See the https://wiki.apache.org/solr/ReleaseNote75[7.5 Release Notes] for an overview of
the main new features in Solr 7.5.
-
-When upgrading to Solr 7.5, users should be aware of the following major changes from v7.4:
-
-*Schema Changes*
-
-* Since Solr 7.0, Solr's schema field-guessing has created `_str` fields for all `_txt` fields,
and returned those by default with queries. As of 7.5, `_str` fields will no longer be returned
by default. They will still be available and can be requested with the `fl` parameter on queries.
See also the section on <<schemaless-mode.adoc#enable-field-class-guessing,field guessing>>
for more information about how schema field guessing works.
-* The Standard Filter, which has been non-operational since at least Solr v4, has been removed.
-
-*Index Merge Policy*
-
-* When using the <<indexconfig-in-solrconfig.adoc#mergepolicyfactory,`TieredMergePolicy`>>,
the default merge policy for Solr, `optimize` and `expungeDeletes` now respect the `maxMergedSegmentMB`
configuration parameter, which defaults to `5000` (5GB).
+An identical parameter is available in `solrconfig.xml` for limiting the size of queries
explicitly defined by the user (or client), but this per-collection limit will still be restricted
by the global limit set in `solr.xml`.
 +
-If it is absolutely necessary to control the number of segments present after optimize, specify
`maxSegments` as a positive integer. Setting `maxSegments` higher than `1` are honored on
a "best effort" basis.
+If your use case demands that you a lot of OR or AND clauses in your queries, upon upgrade
to 8.1 you may need to adjust the global `maxBooleanClauses` parameter since between 7.0 and
8.1 the limit was effectively unbounded.
 +
-The `TieredMergePolicy` will also reclaim resources from segments that exceed `maxMergedSegmentMB`
more aggressively than earlier.
+For more information about the new parameter, see the section <<format-of-solr-xml.adoc#global-maxbooleanclauses,Format
of solr.xml: maxBooleanClauses>>.
 
-*UIMA Removed*
+*Security*
 
-* The UIMA contrib has been removed from Solr and is no longer available.
+* JSON Web Tokens (JWT) are now supported for authentication. These allow Solr to assert
a user is already authenticated via an external identity provider, such as an OpenID Connect-enabled
IdP. For more information, see the section <<jwt-authentication-plugin.adoc#jwt-authentication-plugin,JWT
Authentication Plugin>>.
 
-*Logging*
+* A new security plugin for audit logging has been added. A default class `SolrLogAuditLoggerPlugin`
is available and configurable in `security.json`. The base class is also extendable for adding
custom audit plugins if needed. See the section <<audit-logging.adoc#audit-logging,Audit
Logging>> for more information.
 
-* Solr's logging configuration file is now located in `server/resources/log4j2.xml` by default.
+*Collections API*
 
-* A bug for Windows users has been corrected. When using Solr's examples (`bin/solr start
-e`) log files will now be put in the correct location (`example/` instead of `server`). See
also <<installing-solr.adoc#solr-examples,Solr Examples>> and <<solr-control-script-reference.adoc#solr-control-script-reference,Solr
Control Script Reference>> for more information.
+* The output of the REQUESTSTATUS command in the Collections API will now include internal
asynchronous requests (if any) in the "success" or "failed" keys.
 
+* The CREATE command will now return the appropriate status code (4xx, 5xx, etc.) when the
command has failed. Previously, it always returned `0`, even in failure.
 
-=== Solr 7.4
+* The MODIFYCOLLECTION command now accepts an attribute to set a collection as read-only.
This can be used to block a collection from receiving any updates while still allowing queries
to be served. See the section <<collections-api.adoc#modifycollection,MODIFYCOLLECTION>>
for details on how to use it.
 
-See the https://wiki.apache.org/solr/ReleaseNote74[7.4 Release Notes] for an overview of
the main new features in Solr 7.4.
+* A new command RENAME allows renaming a collection by setting up a one-to-one alias using
the new name. For more information, see the section <<collections-api.adoc#rename,RENAME>>.
 
-When upgrading to Solr 7.4, users should be aware of the following major changes from v7.3:
+* A new command REINDEXCOLLECTION allows indexing existing stored fields from a source collection
into a new collection. For more information, please see the section <<collections-api.adoc#reindexcollection,REINDEXCOLLECTION>>.
 
 *Logging*
 
-* Solr now uses Log4j v2.11. The Log4j configuration is now in `log4j2.xml` rather than `log4j.properties`
files. This is a server side change only and clients using SolrJ won't need any changes. Clients
can still use any logging implementation which is compatible with SLF4J. We now let Log4j
handle rotation of Solr logs at startup, and `bin/solr` start scripts will no longer attempt
this nor move existing console or garbage collection logs into `logs/archived` either. See
<<configuring- [...]
-
-* Configuring `slowQueryThresholdMillis` now logs slow requests to a separate file named
`solr_slow_requests.log`. Previously they would get logged in the `solr.log` file.
-
-*Legacy Scaling (non-SolrCloud)*
-
-* In the <<index-replication.adoc#index-replication,master-slave model>> of scaling
Solr, a slave no longer commits an empty index when a completely new index is detected on
master during replication. To return to the previous behavior pass `false` to `skipCommitOnMasterVersionZero`
in the slave section of replication handler configuration, or pass it to the `fetchindex`
command.
-
-If you are upgrading from a version earlier than Solr 7.3, please see previous version notes
below.
-
-=== Solr 7.3
-
-See the https://wiki.apache.org/solr/ReleaseNote73[7.3 Release Notes] for an overview of
the main new features in Solr 7.3.
-
-When upgrading to Solr 7.3, users should be aware of the following major changes from v7.2:
-
-*ConfigSets*
-
-* Collections created without specifying a configset name have used a copy of the `_default`
configset since Solr 7.0. Before 7.3, the copied configset was named the same as the collection
name, but from 7.3 onwards it will be named with a new ".AUTOCREATED" suffix. This is to prevent
overwriting custom configset names.
-
-*Learning to Rank*
-
-* The `rq` parameter used with Learning to Rank `rerank` query parsing no longer considers
the `defType` parameter. See <<learning-to-rank.adoc#running-a-rerank-query,Running
a Rerank Query>> for more information about this parameter.
-
-*Autoscaling & AutoAddReplicas*
-
-* The behaviour of the autoscaling system will now pause all triggers from execution between
the start of actions and the end of a cool down period. The triggers will resume after the
cool down period expires. Previously, the cool down period was a fixed period started after
actions for a trigger event completed and during this time all triggers continued to run but
any events were rejected and tried later.
-
-* The throttling mechanism used to limit the rate of autoscaling events processed has been
removed. This deprecates the `actionThrottlePeriodSeconds` setting in the <<solrcloud-autoscaling-api.adoc#change-autoscaling-properties,`set-properties`
Autoscaling API>> which is now non-operational. Use the `triggerCooldownPeriodSeconds`
parameter instead to pause event processing.
-
-* The default value of `autoReplicaFailoverWaitAfterExpiration`, used with the AutoAddReplicas
feature, has increased to 120 seconds from the previous default of 30 seconds. This affects
how soon Solr adds new replicas to replace the replicas on nodes which have either crashed
or shutdown.
-
-*Logging*
-
-* The default Solr log file size and number of backups have been raised to 32MB and 10 respectively.
See the section <<configuring-logging.adoc#configuring-logging,Configuring Logging>>
for more information about how to configure logging.
-
-*SolrCloud*
-
-* The old Leader-In-Recovery implementation (implemented in Solr 4.9) is now deprecated and
replaced. Solr will support rolling upgrades from old 7.x versions of Solr to future 7.x releases
until the last release of the 7.x major version.
-+
-This means to upgrade to Solr 8 in the future, you will need to be on Solr 7.3 or higher.
-
-* Replicas which are not up-to-date are no longer allowed to become leader. Use the <<collections-api.adoc#forceleader,FORCELEADER
command>> of the Collections API to allow these replicas become leader.
-
-*Spatial*
-
-* If you are using the spatial JTS library with Solr, you must upgrade to 1.15.0. This new
version of JTS is now dual-licensed to include a BSD style license. See the section on <<spatial-search.adoc#spatial-search,Spatial
Search>> for more information.
-
-*Highlighting*
-
-* The top-level `<highlighting>` element in `solrconfig.xml` is now officially deprecated
in favour of the equivalent `<searchComponent>` syntax. This element has been out of
use in default Solr installations for several releases already.
-
-If you are upgrading from a version earlier than Solr 7.2, please see previous version notes
below.
-
-=== Solr 7.2
-
-See the https://wiki.apache.org/solr/ReleaseNote72[7.2 Release Notes] for an overview of
the main new features in Solr 7.2.
-
-When upgrading to Solr 7.2, users should be aware of the following major changes from v7.1:
-
-*Local Parameters*
-
-* Starting a query string with <<local-parameters-in-queries.adoc#local-parameters-in-queries,local
parameters>> `{!myparser ...}` is used to switch from one query parser to another, and
is intended for use by Solr system developers, not end users doing searches. To reduce negative
side-effects of unintended hack-ability, Solr now limits the cases when local parameters will
be parsed to only contexts in which the default parser is "<<other-parsers.adoc#lucene-query-parser,lucene>>"
or "< [...]
-+
-So, if `defType=edismax` then `q={!myparser ...}` won't work. In that example, put the desired
query parser into the `defType` parameter.
-+
-Another example is if `deftype=edismax` then `hl.q={!myparser ...}` won't work for the same
reason. In this example, either put the desired query parser into the `hl.qparser` parameter
or set `hl.qparser=lucene`. Most users won't run into these cases but some will need to change.
-+
-If you must have full backwards compatibility, use `luceneMatchVersion=7.1.0` or an earlier
version.
-
-*eDisMax Parser*
-
-* The eDisMax parser by default no longer allows subqueries that specify a Solr parser using
either local parameters, or the older `\_query_` magic field trick.
-+
-For example, `{!prefix f=myfield v=enterp}` or `\_query_:"{!prefix f=myfield v=enterp}"`
are not supported by default any longer. If you want to allow power-users to do this, set
`uf=* _query_` or some other value that includes `\_query_`.
-+
-If you need full backwards compatibility for the time being, use `luceneMatchVersion=7.1.0`
or something earlier.
-
-If you are upgrading from a version earlier than Solr 7.1, please see previous version notes
below.
-
-=== Solr 7.1
-
-See the https://wiki.apache.org/solr/ReleaseNote71[7.1 Release Notes] for an overview of
the main new features of Solr 7.1.
-
-When upgrading to Solr 7.1, users should be aware of the following major changes from v7.0:
-
-*AutoAddReplicas*
-
-* The feature to automatically add replicas if a replica goes down, previously available
only when storing indexes in HDFS, has been ported to the autoscaling framework. Due to this,
`autoAddReplicas` is now available to all users even if their indexes are on local disks.
-+
-Existing users of this feature should not have to change anything. However, they should note
these changes:
-
-** Behavior: Changing the `autoAddReplicas` property from disabled (`false`) to enabled (`true`)
using <<collections-api.adoc#modifycollection,MODIFYCOLLECTION API>> no longer
replaces down replicas for the collection immediately. Instead, replicas are only added if
a node containing them went down while `autoAddReplicas` was enabled. The parameters `autoReplicaFailoverBadNodeExpiration`
and `autoReplicaFailoverWorkLoopDelay` are no longer used.
-** Deprecations: Enabling/disabling autoAddReplicas cluster-wide with the API will be deprecated;
use suspend/resume trigger APIs with `name=".auto_add_replicas"` instead.
-+
-More information about the changes to this feature can be found in the section <<solrcloud-autoscaling-auto-add-replicas.adoc#solrcloud-autoscaling-auto-add-replicas,SolrCloud
Automatically Adding Replicas>>.
-
-*Metrics Reporters*
-
-* Shard and cluster metric reporter configuration now require a `class` attribute.
-** If a reporter configures the `group="shard"` attribute then please also configure the
`class="org.apache.solr.metrics.reporters.solr.SolrShardReporter"` attribute.
-** If a reporter configures the `group="cluster"` attribute then please also configure the
 `class="org.apache.solr.metrics.reporters.solr.SolrClusterReporter"` attribute.
+* The default Log4j2 logging mode has been changed from synchronous to asynchronous. This
will improve logging throughput and reduce system contention at the cost of a _slight_ chance
that some logging messages may be missed in the event of abnormal Solr termination.
 +
-See the section <<metrics-reporting.adoc#shard-and-cluster-reporters,Shard and Cluster
Reporters>> for more information.
+If even this slight risk is unacceptable, the Log4j configuration file found in `server/resources/log4j2.xml`
has the synchronous logging configuration in a commented section and can be edited to re-enable
synchronous logging.
 
-*Streaming Expressions*
+*Metrics*
 
-* All Stream Evaluators in `solrj.io.eval` have been refactored to have a simpler and more
robust structure. This simplifies and condenses the code required to implement a new Evaluator
and makes it much easier for evaluators to handle differing data types (primitives, objects,
arrays, lists, and so forth).
+* The SolrGangliaReporter has been removed from Solr. The metrics library used by Solr, Dropwizard
Metrics, was updated to version 4, and Ganglia support was removed from it due to a dependency
on the LGPL license.
 
-*ReplicationHandler*
+*Browse UI (Velocity)*
 
-* In the ReplicationHandler, the `master.commitReserveDuration` sub-element is deprecated.
Instead please configure a direct `commitReserveDuration` element for use in all modes (master,
slave, cloud).
+* Velocity and Velocity Tools were both upgraded as part of this release. Velocity upgraded
from 1.7 to 2.0. Please see https://velocity.apache.org/engine/2.0/upgrading.html about upgrading.
Velocity Tools upgraded from 2.0 to 3.0. For more details, please see https://velocity.apache.org/tools/3.0/upgrading.html
for details about the upgrade.
 
-*RunExecutableListener*
+*Default Garbage Collector (GC)*
 
-* The `RunExecutableListener` was removed for security reasons. If you want to listen to
events caused by updates, commits, or optimize, write your own listener as native Java class
as part of a Solr plugin.
+* Solr's default GC has been changed from CMS to G1. If you prefer to use CMS or any other
GC method, you can modify the `GC_TUNE` section of `solr.in.sh` (*nix) or `solr.in.cmd` (Windows).
 
-*XML Query Parser*
 
-* In the XML query parser (`defType=xmlparser` or `{!xmlparser ... }`) the resolving of external
entities is now disallowed by default.
+== Upgrading from 7.x Releases
 
-If you are upgrading from a version earlier than Solr 7.0, please see <<major-changes-in-solr-7.adoc#major-changes-in-solr-7,Major
Changes in Solr 7>> before starting your upgrade.
+The upgrade from 7.x to Solr 8.0 introduces several major changes that you should be aware
of before upgrading.
+These changes are described in the section <<major-changes-in-solr-8.adoc#major-changes-in-solr-8,Major
Changes in Solr 8>>. It's strongly recommended that you do a thorough review of that
section before starting your upgrade.
 
-== Upgrading to 7.x from Any 6.x Release
+[NOTE]
+If you run in SolrCloud mode, you must be on Solr version 7.3 or higher in order to upgrade
to 8.x.
 
-The upgrade from Solr 6.x to Solr 7.0 introduces several *major* changes that you should
be aware of before upgrading. Please do a thorough review of the section <<major-changes-in-solr-7.adoc#major-changes-in-solr-7,Major
Changes in Solr 7>> before starting your upgrade.
+== Upgrading from Pre-7.x Versions
 
-== Upgrading to 7.x from pre-6.x Versions of Solr
+Users upgrading from versions of Solr prior to 7.x are strongly encouraged to consult {solr-javadocs}/changes/Changes.html[`CHANGES.txt`]
for the details of _all_ changes since the version they are upgrading from.
 
-Users upgrading from versions of Solr prior to 6.x are strongly encouraged to consult {solr-javadocs}/changes/Changes.html[`CHANGES.txt`]
for the details of _all_ changes since the version they are upgrading from.
+The upgrade from Solr 6.x to Solr 7.0 introduced several *major* changes that you should
be aware of before upgrading. Please do a thorough review of the section <<major-changes-in-solr-7.adoc#major-changes-in-solr-7,Major
Changes in Solr 7>> before starting your upgrade.
 
 A summary of the significant changes between Solr 5.x and Solr 6.0 is in the section <<major-changes-from-solr-5-to-solr-6.adoc#major-changes-from-solr-5-to-solr-6,Major
Changes from Solr 5 to Solr 6>>.


Mime
View raw message