From commits-return-101590-archive-asf-public=cust-asf.ponee.io@lucene.apache.org Tue Jun 12 15:33:11 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 51DF018063A for ; Tue, 12 Jun 2018 15:33:10 +0200 (CEST) Received: (qmail 11714 invoked by uid 500); 12 Jun 2018 13:33:09 -0000 Mailing-List: contact commits-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list commits@lucene.apache.org Received: (qmail 11699 invoked by uid 99); 12 Jun 2018 13:33:09 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 12 Jun 2018 13:33:09 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id D92EEE09F3; Tue, 12 Jun 2018 13:33:08 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: sarowe@apache.org To: commits@lucene.apache.org Date: Tue, 12 Jun 2018 13:33:10 -0000 Message-Id: <5e98258407a04e50b070b250ccbf02d9@git.apache.org> In-Reply-To: <3199ac4e9a72486a8d8d1bec5b668a64@git.apache.org> References: <3199ac4e9a72486a8d8d1bec5b668a64@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [3/3] lucene-solr:branch_7_4: LUCENE-8278: Some end-of-input no-scheme domain-only URL tokens are typed as rather than LUCENE-8278: Some end-of-input no-scheme domain-only URL tokens are typed as rather than Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/3ed77ebd Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/3ed77ebd Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/3ed77ebd Branch: refs/heads/branch_7_4 Commit: 3ed77ebd8dc85ebda2817033ec20df372735c650 Parents: a4c6a83 Author: Steve Rowe Authored: Mon Jun 11 17:09:30 2018 -0400 Committer: Steve Rowe Committed: Tue Jun 12 09:32:28 2018 -0400 ---------------------------------------------------------------------- lucene/CHANGES.txt | 3 + lucene/analysis/common/build.xml | 1 + .../analysis/standard/ASCIITLD.jflex-macro | 263 +- .../standard/UAX29URLEmailTokenizerImpl.java | 47679 +++++++++-------- .../standard/UAX29URLEmailTokenizerImpl.jflex | 26 +- .../standard/TestUAX29URLEmailTokenizer.java | 158 + .../standard/GenerateJflexTLDMacros.java | 148 +- 7 files changed, 24281 insertions(+), 23997 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/3ed77ebd/lucene/CHANGES.txt ---------------------------------------------------------------------- diff --git a/lucene/CHANGES.txt b/lucene/CHANGES.txt index 6a1fd1a..ce68a3b 100644 --- a/lucene/CHANGES.txt +++ b/lucene/CHANGES.txt @@ -158,6 +158,9 @@ Bug Fixes * LUCENE-8186: LowerCaseTokenizerFactory now lowercases text in multi-term queries. (Tim Allison via Adrien Grand) + +* LUCENE-8278: Some end-of-input no-scheme domain-only URL tokens are typed as + rather than . (Junte Zhang, Steve Rowe) Other http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/3ed77ebd/lucene/analysis/common/build.xml ---------------------------------------------------------------------- diff --git a/lucene/analysis/common/build.xml b/lucene/analysis/common/build.xml index 56ca4d0..6e4f118 100644 --- a/lucene/analysis/common/build.xml +++ b/lucene/analysis/common/build.xml @@ -111,6 +111,7 @@ + http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/3ed77ebd/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/ASCIITLD.jflex-macro ---------------------------------------------------------------------- diff --git a/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/ASCIITLD.jflex-macro b/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/ASCIITLD.jflex-macro index 08df6c4..af88bf8 100644 --- a/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/ASCIITLD.jflex-macro +++ b/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/ASCIITLD.jflex-macro @@ -15,10 +15,11 @@ * limitations under the License. */ // Generated from IANA Root Zone Database -// file version from Friday, October 6, 2017 5:41:00 AM UTC -// generated on Friday, October 6, 2017 4:00:48 PM UTC +// file version from Sunday, May 27, 2018 5:16:00 PM UTC +// generated on Sunday, May 27, 2018 7:38:07 PM UTC // by org.apache.lucene.analysis.standard.GenerateJflexTLDMacros +// LUCENE-8278: None of the TLDs in {ASCIITLD} is a 1-character-shorter prefix of another TLD ASCIITLD = "." ( [aA][aA][aA] | [aA][aA][rR][pP] @@ -30,31 +31,24 @@ ASCIITLD = "." ( | [aA][bB][lL][eE] | [aA][bB][oO][gG][aA][dD][oO] | [aA][bB][uU][dD][hH][aA][bB][iI] - | [aA][cC] | [aA][cC][aA][dD][eE][mM][yY] | [aA][cC][cC][eE][nN][tT][uU][rR][eE] - | [aA][cC][cC][oO][uU][nN][tT][aA][nN][tT] | [aA][cC][cC][oO][uU][nN][tT][aA][nN][tT][sS] | [aA][cC][oO] | [aA][cC][tT][iI][vV][eE] | [aA][cC][tT][oO][rR] - | [aA][dD] | [aA][dD][aA][cC] | [aA][dD][sS] | [aA][dD][uU][lL][tT] - | [aA][eE] | [aA][eE][gG] | [aA][eE][rR][oO] | [aA][eE][tT][nN][aA] - | [aA][fF] | [aA][fF][aA][mM][iI][lL][yY][cC][oO][mM][pP][aA][nN][yY] | [aA][fF][lL] | [aA][fF][rR][iI][cC][aA] | [aA][gG] | [aA][gG][aA][kK][hH][aA][nN] | [aA][gG][eE][nN][cC][yY] - | [aA][iI] - | [aA][iI][gG] | [aA][iI][gG][oO] | [aA][iI][rR][bB][uU][sS] | [aA][iI][rR][fF][oO][rR][cC][eE] @@ -80,20 +74,17 @@ ASCIITLD = "." ( | [aA][nN][dD][rR][oO][iI][dD] | [aA][nN][qQ][uU][aA][nN] | [aA][nN][zZ] - | [aA][oO] | [aA][oO][lL] | [aA][pP][aA][rR][tT][mM][eE][nN][tT][sS] | [aA][pP][pP] | [aA][pP][pP][lL][eE] | [aA][qQ] | [aA][qQ][uU][aA][rR][eE][lL][lL][eE] - | [aA][rR] | [aA][rR][aA][bB] | [aA][rR][aA][mM][cC][oO] | [aA][rR][cC][hH][iI] | [aA][rR][mM][yY] | [aA][rR][pP][aA] - | [aA][rR][tT] | [aA][rR][tT][eE] | [aA][sS] | [aA][sS][dD][aA] @@ -104,21 +95,16 @@ ASCIITLD = "." ( | [aA][tT][tT][oO][rR][nN][eE][yY] | [aA][uU] | [aA][uU][cC][tT][iI][oO][nN] - | [aA][uU][dD][iI] | [aA][uU][dD][iI][bB][lL][eE] | [aA][uU][dD][iI][oO] | [aA][uU][sS][pP][oO][sS][tT] | [aA][uU][tT][hH][oO][rR] - | [aA][uU][tT][oO] | [aA][uU][tT][oO][sS] | [aA][vV][iI][aA][nN][cC][aA] - | [aA][wW] | [aA][wW][sS] - | [aA][xX] | [aA][xX][aA] | [aA][zZ] | [aA][zZ][uU][rR][eE] - | [bB][aA] | [bB][aA][bB][yY] | [bB][aA][iI][dD][uU] | [bB][aA][nN][aA][mM][eE][xX] @@ -135,14 +121,12 @@ ASCIITLD = "." ( | [bB][aA][sS][kK][eE][tT][bB][aA][lL][lL] | [bB][aA][uU][hH][aA][uU][sS] | [bB][aA][yY][eE][rR][nN] - | [bB][bB] | [bB][bB][cC] | [bB][bB][tT] | [bB][bB][vV][aA] | [bB][cC][gG] | [bB][cC][nN] | [bB][dD] - | [bB][eE] | [bB][eE][aA][tT][sS] | [bB][eE][aA][uU][tT][yY] | [bB][eE][eE][rR] @@ -155,11 +139,9 @@ ASCIITLD = "." ( | [bB][gG] | [bB][hH] | [bB][hH][aA][rR][tT][iI] - | [bB][iI] | [bB][iI][bB][lL][eE] | [bB][iI][dD] | [bB][iI][kK][eE] - | [bB][iI][nN][gG] | [bB][iI][nN][gG][oO] | [bB][iI][oO] | [bB][iI][zZ] @@ -171,22 +153,17 @@ ASCIITLD = "." ( | [bB][lL][oO][gG] | [bB][lL][oO][oO][mM][bB][eE][rR][gG] | [bB][lL][uU][eE] - | [bB][mM] | [bB][mM][sS] | [bB][mM][wW] - | [bB][nN] | [bB][nN][lL] | [bB][nN][pP][pP][aA][rR][iI][bB][aA][sS] - | [bB][oO] | [bB][oO][aA][tT][sS] | [bB][oO][eE][hH][rR][iI][nN][gG][eE][rR] | [bB][oO][fF][aA] | [bB][oO][mM] | [bB][oO][nN][dD] - | [bB][oO][oO] | [bB][oO][oO][kK] | [bB][oO][oO][kK][iI][nN][gG] - | [bB][oO][oO][tT][sS] | [bB][oO][sS][cC][hH] | [bB][oO][sS][tT][iI][kK] | [bB][oO][sS][tT][oO][nN] @@ -212,15 +189,11 @@ ASCIITLD = "." ( | [bB][vV] | [bB][wW] | [bB][yY] - | [bB][zZ] | [bB][zZ][hH] - | [cC][aA] | [cC][aA][bB] | [cC][aA][fF][eE] - | [cC][aA][lL] | [cC][aA][lL][lL] | [cC][aA][lL][vV][iI][nN][kK][lL][eE][iI][nN] - | [cC][aA][mM] | [cC][aA][mM][eE][rR][aA] | [cC][aA][mM][pP] | [cC][aA][nN][cC][eE][rR][rR][eE][sS][eE][aA][rR][cC][hH] @@ -228,11 +201,9 @@ ASCIITLD = "." ( | [cC][aA][pP][eE][tT][oO][wW][nN] | [cC][aA][pP][iI][tT][aA][lL] | [cC][aA][pP][iI][tT][aA][lL][oO][nN][eE] - | [cC][aA][rR] | [cC][aA][rR][aA][vV][aA][nN] | [cC][aA][rR][dD][sS] | [cC][aA][rR][eE] - | [cC][aA][rR][eE][eE][rR] | [cC][aA][rR][eE][eE][rR][sS] | [cC][aA][rR][sS] | [cC][aA][rR][tT][iI][eE][rR] @@ -254,7 +225,6 @@ ASCIITLD = "." ( | [cC][eE][nN][tT][eE][rR] | [cC][eE][oO] | [cC][eE][rR][nN] - | [cC][fF] | [cC][fF][aA] | [cC][fF][dD] | [cC][gG] @@ -265,7 +235,6 @@ ASCIITLD = "." ( | [cC][hH][aA][tT] | [cC][hH][eE][aA][pP] | [cC][hH][iI][nN][tT][aA][iI] - | [cC][hH][lL][oO][eE] | [cC][hH][rR][iI][sS][tT][mM][aA][sS] | [cC][hH][rR][oO][mM][eE] | [cC][hH][rR][yY][sS][lL][eE][rR] @@ -275,7 +244,6 @@ ASCIITLD = "." ( | [cC][iI][rR][cC][lL][eE] | [cC][iI][sS][cC][oO] | [cC][iI][tT][aA][dD][eE][lL] - | [cC][iI][tT][iI] | [cC][iI][tT][iI][cC] | [cC][iI][tT][yY] | [cC][iI][tT][yY][eE][aA][tT][sS] @@ -292,7 +260,6 @@ ASCIITLD = "." ( | [cC][lL][uU][bB][mM][eE][dD] | [cC][mM] | [cC][nN] - | [cC][oO] | [cC][oO][aA][cC][hH] | [cC][oO][dD][eE][sS] | [cC][oO][fF][fF][eE][eE] @@ -317,17 +284,14 @@ ASCIITLD = "." ( | [cC][oO][oO][pP] | [cC][oO][rR][sS][iI][cC][aA] | [cC][oO][uU][nN][tT][rR][yY] - | [cC][oO][uU][pP][oO][nN] | [cC][oO][uU][pP][oO][nN][sS] | [cC][oO][uU][rR][sS][eE][sS] - | [cC][rR] | [cC][rR][eE][dD][iI][tT] | [cC][rR][eE][dD][iI][tT][cC][aA][rR][dD] | [cC][rR][eE][dD][iI][tT][uU][nN][iI][oO][nN] | [cC][rR][iI][cC][kK][eE][tT] | [cC][rR][oO][wW][nN] | [cC][rR][sS] - | [cC][rR][uU][iI][sS][eE] | [cC][rR][uU][iI][sS][eE][sS] | [cC][sS][cC] | [cC][uU] @@ -349,8 +313,6 @@ ASCIITLD = "." ( | [dD][aA][yY] | [dD][cC][lL][kK] | [dD][dD][sS] - | [dD][eE] - | [dD][eE][aA][lL] | [dD][eE][aA][lL][eE][rR] | [dD][eE][aA][lL][sS] | [dD][eE][gG][rR][eE][eE] @@ -378,7 +340,6 @@ ASCIITLD = "." ( | [dD][kK] | [dD][mM] | [dD][nN][pP] - | [dD][oO] | [dD][oO][cC][sS] | [dD][oO][cC][tT][oO][rR] | [dD][oO][dD][gG][eE] @@ -400,7 +361,6 @@ ASCIITLD = "." ( | [dD][zZ] | [eE][aA][rR][tT][hH] | [eE][aA][tT] - | [eE][cC] | [eE][cC][oO] | [eE][dD][eE][kK][aA] | [eE][dD][uU] @@ -419,13 +379,11 @@ ASCIITLD = "." ( | [eE][rR] | [eE][rR][iI][cC][sS][sS][oO][nN] | [eE][rR][nN][iI] - | [eE][sS] | [eE][sS][qQ] | [eE][sS][tT][aA][tT][eE] | [eE][sS][uU][rR][aA][nN][cC][eE] | [eE][tT] | [eE][tT][iI][sS][aA][lL][aA][tT] - | [eE][uU] | [eE][uU][rR][oO][vV][iI][sS][iI][oO][nN] | [eE][uU][sS] | [eE][vV][eE][nN][tT][sS] @@ -440,7 +398,6 @@ ASCIITLD = "." ( | [fF][aA][iI][rR][wW][iI][nN][dD][sS] | [fF][aA][iI][tT][hH] | [fF][aA][mM][iI][lL][yY] - | [fF][aA][nN] | [fF][aA][nN][sS] | [fF][aA][rR][mM] | [fF][aA][rR][mM][eE][rR][sS] @@ -450,7 +407,6 @@ ASCIITLD = "." ( | [fF][eE][eE][dD][bB][aA][cC][kK] | [fF][eE][rR][rR][aA][rR][iI] | [fF][eE][rR][rR][eE][rR][oO] - | [fF][iI] | [fF][iI][aA][tT] | [fF][iI][dD][eE][lL][iI][tT][yY] | [fF][iI][dD][oO] @@ -474,8 +430,6 @@ ASCIITLD = "." ( | [fF][lL][oO][wW][eE][rR][sS] | [fF][lL][yY] | [fF][mM] - | [fF][oO] - | [fF][oO][oO] | [fF][oO][oO][dD] | [fF][oO][oO][dD][nN][eE][tT][wW][oO][rR][kK] | [fF][oO][oO][tT][bB][aA][lL][lL] @@ -485,7 +439,6 @@ ASCIITLD = "." ( | [fF][oO][rR][uU][mM] | [fF][oO][uU][nN][dD][aA][tT][iI][oO][nN] | [fF][oO][xX] - | [fF][rR] | [fF][rR][eE][eE] | [fF][rR][eE][sS][eE][nN][iI][uU][sS] | [fF][rR][lL] @@ -495,25 +448,20 @@ ASCIITLD = "." ( | [fF][tT][rR] | [fF][uU][jJ][iI][tT][sS][uU] | [fF][uU][jJ][iI][xX][eE][rR][oO][xX] - | [fF][uU][nN] | [fF][uU][nN][dD] | [fF][uU][rR][nN][iI][tT][uU][rR][eE] | [fF][uU][tT][bB][oO][lL] | [fF][yY][iI] - | [gG][aA] | [gG][aA][lL] | [gG][aA][lL][lL][eE][rR][yY] | [gG][aA][lL][lL][oO] | [gG][aA][lL][lL][uU][pP] - | [gG][aA][mM][eE] | [gG][aA][mM][eE][sS] | [gG][aA][pP] | [gG][aA][rR][dD][eE][nN] | [gG][bB] | [gG][bB][iI][zZ] - | [gG][dD] | [gG][dD][nN] - | [gG][eE] | [gG][eE][aA] | [gG][eE][nN][tT] | [gG][eE][nN][tT][iI][nN][gG] @@ -523,17 +471,14 @@ ASCIITLD = "." ( | [gG][gG][eE][eE] | [gG][hH] | [gG][iI] - | [gG][iI][fF][tT] | [gG][iI][fF][tT][sS] | [gG][iI][vV][eE][sS] | [gG][iI][vV][iI][nN][gG] - | [gG][lL] | [gG][lL][aA][dD][eE] | [gG][lL][aA][sS][sS] | [gG][lL][eE] | [gG][lL][oO][bB][aA][lL] | [gG][lL][oO][bB][oO] - | [gG][mM] | [gG][mM][aA][iI][lL] | [gG][mM][bB][hH] | [gG][mM][oO] @@ -543,7 +488,6 @@ ASCIITLD = "." ( | [gG][oO][lL][dD] | [gG][oO][lL][dD][pP][oO][iI][nN][tT] | [gG][oO][lL][fF] - | [gG][oO][oO] | [gG][oO][oO][dD][hH][aA][nN][dD][sS] | [gG][oO][oO][dD][yY][eE][aA][rR] | [gG][oO][oO][gG] @@ -590,7 +534,6 @@ ASCIITLD = "." ( | [hH][iI][sS][aA][mM][iI][tT][sS][uU] | [hH][iI][tT][aA][cC][hH][iI] | [hH][iI][vV] - | [hH][kK] | [hH][kK][tT] | [hH][mM] | [hH][nN] @@ -616,7 +559,6 @@ ASCIITLD = "." ( | [hH][rR] | [hH][sS][bB][cC] | [hH][tT] - | [hH][tT][cC] | [hH][uU] | [hH][uU][gG][hH][eE][sS] | [hH][yY][aA][tT][tT] @@ -636,7 +578,6 @@ ASCIITLD = "." ( | [iI][mM][dD][bB] | [iI][mM][mM][oO] | [iI][mM][mM][oO][bB][iI][lL][iI][eE][nN] - | [iI][nN] | [iI][nN][dD][uU][sS][tT][rR][iI][eE][sS] | [iI][nN][fF][iI][nN][iI][tT][iI] | [iI][nN][fF][oO] @@ -655,12 +596,10 @@ ASCIITLD = "." ( | [iI][qQ] | [iI][rR] | [iI][rR][iI][sS][hH] - | [iI][sS] | [iI][sS][eE][lL][eE][cC][tT] | [iI][sS][mM][aA][iI][lL][iI] | [iI][sS][tT] | [iI][sS][tT][aA][nN][bB][uU][lL] - | [iI][tT] | [iI][tT][aA][uU] | [iI][tT][vV] | [iI][vV][eE][cC][oO] @@ -676,10 +615,8 @@ ASCIITLD = "." ( | [jJ][iI][oO] | [jJ][lL][cC] | [jJ][lL][lL] - | [jJ][mM] | [jJ][mM][pP] | [jJ][nN][jJ] - | [jJ][oO] | [jJ][oO][bB][sS] | [jJ][oO][bB][uU][rR][gG] | [jJ][oO][tT] @@ -698,7 +635,6 @@ ASCIITLD = "." ( | [kK][fF][hH] | [kK][gG] | [kK][hH] - | [kK][iI] | [kK][iI][aA] | [kK][iI][mM] | [kK][iI][nN][dD][eE][rR] @@ -710,10 +646,8 @@ ASCIITLD = "." ( | [kK][oO][eE][lL][nN] | [kK][oO][mM][aA][tT][sS][uU] | [kK][oO][sS][hH][eE][rR] - | [kK][pP] | [kK][pP][mM][gG] | [kK][pP][nN] - | [kK][rR] | [kK][rR][dD] | [kK][rR][eE][dD] | [kK][uU][oO][kK][gG][rR][oO][uU][pP] @@ -721,7 +655,6 @@ ASCIITLD = "." ( | [kK][yY] | [kK][yY][oO][tT][oO] | [kK][zZ] - | [lL][aA] | [lL][aA][cC][aA][iI][xX][aA] | [lL][aA][dD][bB][rR][oO][kK][eE][sS] | [lL][aA][mM][bB][oO][rR][gG][hH][iI][nN][iI] @@ -767,7 +700,7 @@ ASCIITLD = "." ( | [lL][iI][vV][iI][nN][gG] | [lL][iI][xX][iI][lL] | [lL][kK] - | [lL][oO][aA][nN] + | [lL][lL][cC] | [lL][oO][aA][nN][sS] | [lL][oO][cC][kK][eE][rR] | [lL][oO][cC][uU][sS] @@ -781,8 +714,6 @@ ASCIITLD = "." ( | [lL][pP][lL][fF][iI][nN][aA][nN][cC][iI][aA][lL] | [lL][rR] | [lL][sS] - | [lL][tT] - | [lL][tT][dD] | [lL][tT][dD][aA] | [lL][uU] | [lL][uU][nN][dD][bB][eE][cC][kK] @@ -791,7 +722,6 @@ ASCIITLD = "." ( | [lL][uU][xX][uU][rR][yY] | [lL][vV] | [lL][yY] - | [mM][aA] | [mM][aA][cC][yY][sS] | [mM][aA][dD][rR][iI][dD] | [mM][aA][iI][fF] @@ -801,7 +731,6 @@ ASCIITLD = "." ( | [mM][aA][nN][aA][gG][eE][mM][eE][nN][tT] | [mM][aA][nN][gG][oO] | [mM][aA][pP] - | [mM][aA][rR][kK][eE][tT] | [mM][aA][rR][kK][eE][tT][iI][nN][gG] | [mM][aA][rR][kK][eE][tT][sS] | [mM][aA][rR][rR][iI][oO][tT][tT] @@ -812,16 +741,13 @@ ASCIITLD = "." ( | [mM][cC] | [mM][cC][kK][iI][nN][sS][eE][yY] | [mM][dD] - | [mM][eE] | [mM][eE][dD] | [mM][eE][dD][iI][aA] | [mM][eE][eE][tT] | [mM][eE][lL][bB][oO][uU][rR][nN][eE] | [mM][eE][mM][eE] | [mM][eE][mM][oO][rR][iI][aA][lL] - | [mM][eE][nN] | [mM][eE][nN][uU] - | [mM][eE][oO] | [mM][eE][rR][cC][kK][mM][sS][dD] | [mM][eE][tT][lL][iI][fF][eE] | [mM][gG] @@ -834,13 +760,10 @@ ASCIITLD = "." ( | [mM][iI][tT] | [mM][iI][tT][sS][uU][bB][iI][sS][hH][iI] | [mM][kK] - | [mM][lL] | [mM][lL][bB] | [mM][lL][sS] - | [mM][mM] | [mM][mM][aA] | [mM][nN] - | [mM][oO] | [mM][oO][bB][iI] | [mM][oO][bB][iI][lL][eE] | [mM][oO][bB][iI][lL][yY] @@ -863,9 +786,7 @@ ASCIITLD = "." ( | [mM][pP] | [mM][qQ] | [mM][rR] - | [mM][sS] | [mM][sS][dD] - | [mM][tT] | [mM][tT][nN] | [mM][tT][rR] | [mM][uU] @@ -876,7 +797,6 @@ ASCIITLD = "." ( | [mM][xX] | [mM][yY] | [mM][zZ] - | [nN][aA] | [nN][aA][bB] | [nN][aA][dD][eE][xX] | [nN][aA][gG][oO][yY][aA] @@ -886,22 +806,18 @@ ASCIITLD = "." ( | [nN][aA][vV][yY] | [nN][bB][aA] | [nN][cC] - | [nN][eE] | [nN][eE][cC] | [nN][eE][tT] | [nN][eE][tT][bB][aA][nN][kK] | [nN][eE][tT][fF][lL][iI][xX] | [nN][eE][tT][wW][oO][rR][kK] | [nN][eE][uU][sS][tT][aA][rR] - | [nN][eE][wW] | [nN][eE][wW][hH][oO][lL][lL][aA][nN][dD] | [nN][eE][wW][sS] | [nN][eE][xX][tT] | [nN][eE][xX][tT][dD][iI][rR][eE][cC][tT] | [nN][eE][xX][uU][sS] - | [nN][fF] | [nN][fF][lL] - | [nN][gG] | [nN][gG][oO] | [nN][hH][kK] | [nN][iI] @@ -912,7 +828,6 @@ ASCIITLD = "." ( | [nN][iI][sS][sS][aA][nN] | [nN][iI][sS][sS][aA][yY] | [nN][lL] - | [nN][oO] | [nN][oO][kK][iI][aA] | [nN][oO][rR][tT][hH][wW][eE][sS][tT][eE][rR][nN][mM][uU][tT][uU][aA][lL] | [nN][oO][rR][tT][oO][nN] @@ -920,7 +835,6 @@ ASCIITLD = "." ( | [nN][oO][wW][rR][uU][zZ] | [nN][oO][wW][tT][vV] | [nN][pP] - | [nN][rR] | [nN][rR][aA] | [nN][rR][wW] | [nN][tT][tT] @@ -954,7 +868,6 @@ ASCIITLD = "." ( | [oO][tT][sS][uU][kK][aA] | [oO][tT][tT] | [oO][vV][hH] - | [pP][aA] | [pP][aA][gG][eE] | [pP][aA][nN][aA][sS][oO][nN][iI][cC] | [pP][aA][nN][eE][rR][aA][iI] @@ -966,17 +879,14 @@ ASCIITLD = "." ( | [pP][aA][sS][sS][aA][gG][eE][nN][sS] | [pP][aA][yY] | [pP][cC][cC][wW] - | [pP][eE] | [pP][eE][tT] | [pP][fF] | [pP][fF][iI][zZ][eE][rR] | [pP][gG] - | [pP][hH] | [pP][hH][aA][rR][mM][aA][cC][yY] | [pP][hH][dD] | [pP][hH][iI][lL][iI][pP][sS] | [pP][hH][oO][nN][eE] - | [pP][hH][oO][tT][oO] | [pP][hH][oO][tT][oO][gG][rR][aA][pP][hH][yY] | [pP][hH][oO][tT][oO][sS] | [pP][hH][yY][sS][iI][oO] @@ -985,7 +895,6 @@ ASCIITLD = "." ( | [pP][iI][cC][tT][eE][tT] | [pP][iI][cC][tT][uU][rR][eE][sS] | [pP][iI][dD] - | [pP][iI][nN] | [pP][iI][nN][gG] | [pP][iI][nN][kK] | [pP][iI][oO][nN][eE][eE][rR] @@ -998,19 +907,16 @@ ASCIITLD = "." ( | [pP][lL][uU][mM][bB][iI][nN][gG] | [pP][lL][uU][sS] | [pP][mM] - | [pP][nN] | [pP][nN][cC] | [pP][oO][hH][lL] | [pP][oO][kK][eE][rR] | [pP][oO][lL][iI][tT][iI][eE] | [pP][oO][rR][nN] | [pP][oO][sS][tT] - | [pP][rR] | [pP][rR][aA][mM][eE][rR][iI][cC][aA] | [pP][rR][aA][xX][iI] | [pP][rR][eE][sS][sS] | [pP][rR][iI][mM][eE] - | [pP][rR][oO] | [pP][rR][oO][dD] | [pP][rR][oO][dD][uU][cC][tT][iI][oO][nN][sS] | [pP][rR][oO][fF] @@ -1024,7 +930,6 @@ ASCIITLD = "." ( | [pP][sS] | [pP][tT] | [pP][uU][bB] - | [pP][wW] | [pP][wW][cC] | [pP][yY] | [qQ][aA] @@ -1035,7 +940,6 @@ ASCIITLD = "." ( | [rR][aA][cC][iI][nN][gG] | [rR][aA][dD][iI][oO] | [rR][aA][iI][dD] - | [rR][eE] | [rR][eE][aA][dD] | [rR][eE][aA][lL][eE][sS][tT][aA][tT][eE] | [rR][eE][aA][lL][tT][oO][rR] @@ -1045,11 +949,9 @@ ASCIITLD = "." ( | [rR][eE][dD][sS][tT][oO][nN][eE] | [rR][eE][dD][uU][mM][bB][rR][eE][lL][lL][aA] | [rR][eE][hH][aA][bB] - | [rR][eE][iI][sS][eE] | [rR][eE][iI][sS][eE][nN] | [rR][eE][iI][tT] | [rR][eE][lL][iI][aA][nN][cC][eE] - | [rR][eE][nN] | [rR][eE][nN][tT] | [rR][eE][nN][tT][aA][lL][sS] | [rR][eE][pP][aA][iI][rR] @@ -1057,7 +959,6 @@ ASCIITLD = "." ( | [rR][eE][pP][uU][bB][lL][iI][cC][aA][nN] | [rR][eE][sS][tT] | [rR][eE][sS][tT][aA][uU][rR][aA][nN][tT] - | [rR][eE][vV][iI][eE][wW] | [rR][eE][vV][iI][eE][wW][sS] | [rR][eE][xX][rR][oO][tT][hH] | [rR][iI][cC][hH] @@ -1076,14 +977,11 @@ ASCIITLD = "." ( | [rR][oO][oO][mM] | [rR][sS] | [rR][sS][vV][pP] - | [rR][uU] | [rR][uU][gG][bB][yY] | [rR][uU][hH][rR] | [rR][uU][nN] - | [rR][wW] | [rR][wW][eE] | [rR][yY][uU][kK][yY][uU] - | [sS][aA] | [sS][aA][aA][rR][lL][aA][nN][dD] | [sS][aA][fF][eE] | [sS][aA][fF][eE][tT][yY] @@ -1096,15 +994,12 @@ ASCIITLD = "." ( | [sS][aA][nN][dD][vV][iI][kK][cC][oO][rR][oO][mM][aA][nN][tT] | [sS][aA][nN][oO][fF][iI] | [sS][aA][pP] - | [sS][aA][pP][oO] | [sS][aA][rR][lL] | [sS][aA][sS] | [sS][aA][vV][eE] | [sS][aA][xX][oO] - | [sS][bB] | [sS][bB][iI] | [sS][bB][sS] - | [sS][cC] | [sS][cC][aA] | [sS][cC][bB] | [sS][cC][hH][aA][eE][fF][fF][lL][eE][rR] @@ -1118,7 +1013,6 @@ ASCIITLD = "." ( | [sS][cC][oO][rR] | [sS][cC][oO][tT] | [sS][dD] - | [sS][eE] | [sS][eE][aA][rR][cC][hH] | [sS][eE][aA][tT] | [sS][eE][cC][uU][rR][eE] @@ -1130,7 +1024,6 @@ ASCIITLD = "." ( | [sS][eE][sS] | [sS][eE][vV][eE][nN] | [sS][eE][wW] - | [sS][eE][xX] | [sS][eE][xX][yY] | [sS][fF][rR] | [sS][gG] @@ -1154,8 +1047,6 @@ ASCIITLD = "." ( | [sS][iI][nN][gG][lL][eE][sS] | [sS][iI][tT][eE] | [sS][jJ] - | [sS][kK] - | [sS][kK][iI] | [sS][kK][iI][nN] | [sS][kK][yY] | [sS][kK][yY][pP][eE] @@ -1166,7 +1057,6 @@ ASCIITLD = "." ( | [sS][mM][iI][lL][eE] | [sS][nN] | [sS][nN][cC][fF] - | [sS][oO] | [sS][oO][cC][cC][eE][rR] | [sS][oO][cC][iI][aA][lL] | [sS][oO][fF][tT][bB][aA][nN][kK] @@ -1179,12 +1069,11 @@ ASCIITLD = "." ( | [sS][oO][yY] | [sS][pP][aA][cC][eE] | [sS][pP][iI][eE][gG][eE][lL] + | [sS][pP][oO][rR][tT] | [sS][pP][oO][tT] | [sS][pP][rR][eE][aA][dD][bB][eE][tT][tT][iI][nN][gG] - | [sS][rR] | [sS][rR][lL] | [sS][rR][tT] - | [sS][tT] | [sS][tT][aA][dD][aA] | [sS][tT][aA][pP][lL][eE][sS] | [sS][tT][aA][rR] @@ -1227,11 +1116,8 @@ ASCIITLD = "." ( | [tT][aA][tT][aA][mM][oO][tT][oO][rR][sS] | [tT][aA][tT][aA][rR] | [tT][aA][tT][tT][oO][oO] - | [tT][aA][xX] | [tT][aA][xX][iI] - | [tT][cC] | [tT][cC][iI] - | [tT][dD] | [tT][dD][kK] | [tT][eE][aA][mM] | [tT][eE][cC][hH] @@ -1244,7 +1130,6 @@ ASCIITLD = "." ( | [tT][eE][vV][aA] | [tT][fF] | [tT][gG] - | [tT][hH] | [tT][hH][dD] | [tT][hH][eE][aA][tT][eE][rR] | [tT][hH][eE][aA][tT][rR][eE] @@ -1255,7 +1140,6 @@ ASCIITLD = "." ( | [tT][iI][pP][sS] | [tT][iI][rR][eE][sS] | [tT][iI][rR][oO][lL] - | [tT][jJ] | [tT][jJ][mM][aA][xX][xX] | [tT][jJ][xX] | [tT][kK] @@ -1264,7 +1148,6 @@ ASCIITLD = "." ( | [tT][mM] | [tT][mM][aA][lL][lL] | [tT][nN] - | [tT][oO] | [tT][oO][dD][aA][yY] | [tT][oO][kK][yY][oO] | [tT][oO][oO][lL][sS] @@ -1276,7 +1159,6 @@ ASCIITLD = "." ( | [tT][oO][wW][nN] | [tT][oO][yY][oO][tT][aA] | [tT][oO][yY][sS] - | [tT][rR] | [tT][rR][aA][dD][eE] | [tT][rR][aA][dD][iI][nN][gG] | [tT][rR][aA][iI][nN][iI][nN][gG] @@ -1291,7 +1173,6 @@ ASCIITLD = "." ( | [tT][uU][iI] | [tT][uU][nN][eE][sS] | [tT][uU][sS][hH][uU] - | [tT][vV] | [tT][vV][sS] | [tT][wW] | [tT][zZ] @@ -1314,14 +1195,12 @@ ASCIITLD = "." ( | [vV][aA][nN][aA] | [vV][aA][nN][gG][uU][aA][rR][dD] | [vV][cC] - | [vV][eE] | [vV][eE][gG][aA][sS] | [vV][eE][nN][tT][uU][rR][eE][sS] | [vV][eE][rR][iI][sS][iI][gG][nN] | [vV][eE][rR][sS][iI][cC][hH][eE][rR][uU][nN][gG] | [vV][eE][tT] | [vV][gG] - | [vV][iI] | [vV][iI][aA][jJ][eE][sS] | [vV][iI][dD][eE][oO] | [vV][iI][gG] @@ -1369,14 +1248,12 @@ ASCIITLD = "." ( | [wW][iI][eE][nN] | [wW][iI][kK][iI] | [wW][iI][lL][lL][iI][aA][mM][hH][iI][lL][lL] - | [wW][iI][nN] | [wW][iI][nN][dD][oO][wW][sS] | [wW][iI][nN][eE] | [wW][iI][nN][nN][eE][rR][sS] | [wW][mM][eE] | [wW][oO][lL][tT][eE][rR][sS][kK][lL][uU][wW][eE][rR] | [wW][oO][oO][dD][sS][iI][dD][eE] - | [wW][oO][rR][kK] | [wW][oO][rR][kK][sS] | [wW][oO][rR][lL][dD] | [wW][oO][wW] @@ -1506,6 +1383,7 @@ ASCIITLD = "." ( | [xX][nN]--[nN][yY][qQ][yY]26[aA] | [xX][nN]--[oO]3[cC][wW]4[hH] | [xX][nN]--[oO][gG][bB][pP][fF]8[fF][lL] + | [xX][nN]--[oO][tT][uU]796[dD] | [xX][nN]--[pP]1[aA][cC][fF] | [xX][nN]--[pP]1[aA][iI] | [xX][nN]--[pP][bB][tT]977[cC] @@ -1565,3 +1443,132 @@ ASCIITLD = "." ( | [zZ][wW] ) "."? // Accept trailing root (empty) domain +// LUCENE-8278: Each TLD in {ASCIITLDprefix_1CharSuffix} is a prefix of another TLD by 1 character +ASCIITLDprefix_1CharSuffix = "." ( + [aA][cC] + | [aA][cC][cC][oO][uU][nN][tT][aA][nN][tT] + | [aA][dD] + | [aA][eE] + | [aA][fF] + | [aA][iI][gG] + | [aA][oO] + | [aA][rR][tT] + | [aA][uU][dD][iI] + | [aA][uU][tT][oO] + | [aA][wW] + | [aA][xX] + | [bB][aA] + | [bB][bB] + | [bB][eE] + | [bB][iI] + | [bB][iI][nN][gG] + | [bB][mM] + | [bB][nN] + | [bB][oO][oO] + | [bB][zZ] + | [cC][aA][lL] + | [cC][aA][mM] + | [cC][aA][rR] + | [cC][aA][rR][eE][eE][rR] + | [cC][fF] + | [cC][iI][tT][iI] + | [cC][oO] + | [cC][oO][uU][pP][oO][nN] + | [cC][rR] + | [cC][rR][uU][iI][sS][eE] + | [dD][eE] + | [dD][eE][aA][lL] + | [dD][oO] + | [eE][cC] + | [eE][sS] + | [eE][uU] + | [fF][aA][nN] + | [fF][iI] + | [fF][oO][oO] + | [fF][rR] + | [fF][uU][nN] + | [gG][aA] + | [gG][aA][mM][eE] + | [gG][dD] + | [gG][eE] + | [gG][iI][fF][tT] + | [gG][lL] + | [gG][mM] + | [gG][oO][oO] + | [hH][kK] + | [iI][nN] + | [iI][sS] + | [iI][tT] + | [jJ][mM] + | [jJ][oO] + | [kK][iI] + | [kK][pP] + | [kK][rR] + | [lL][aA] + | [lL][oO][aA][nN] + | [lL][tT][dD] + | [mM][aA] + | [mM][aA][rR][kK][eE][tT] + | [mM][eE][nN] + | [mM][lL] + | [mM][mM] + | [mM][oO] + | [mM][sS] + | [mM][tT] + | [nN][aA] + | [nN][eE][wW] + | [nN][fF] + | [nN][gG] + | [nN][oO] + | [nN][rR] + | [pP][aA] + | [pP][eE] + | [pP][hH] + | [pP][hH][oO][tT][oO] + | [pP][iI][nN] + | [pP][nN] + | [pP][rR][oO] + | [pP][wW] + | [rR][eE][iI][sS][eE] + | [rR][eE][nN] + | [rR][eE][vV][iI][eE][wW] + | [rR][uU] + | [rR][wW] + | [sS][aA] + | [sS][bB] + | [sS][cC] + | [sS][eE][xX] + | [sS][kK][iI] + | [sS][oO] + | [sS][rR] + | [sS][tT] + | [tT][aA][xX] + | [tT][cC] + | [tT][dD] + | [tT][hH] + | [tT][jJ] + | [tT][oO] + | [tT][rR] + | [tT][vV] + | [vV][eE] + | [vV][iI] + | [wW][iI][nN] + | [wW][oO][rR][kK] + ) "."? // Accept trailing root (empty) domain + +// LUCENE-8278: Each TLD in {ASCIITLDprefix_2CharSuffix} is a prefix of another TLD by 2 characters +ASCIITLDprefix_2CharSuffix = "." ( + [aA][iI] + | [aA][rR] + | [bB][oO] + | [cC][aA] + | [fF][oO] + | [lL][tT] + | [mM][eE] + | [nN][eE] + | [pP][rR] + | [rR][eE] + | [sS][eE] + | [sS][kK] + ) "."? // Accept trailing root (empty) domain +