Return-Path: X-Original-To: apmail-commons-dev-archive@www.apache.org Delivered-To: apmail-commons-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B8D7210704 for ; Tue, 18 Feb 2014 15:58:48 +0000 (UTC) Received: (qmail 57757 invoked by uid 500); 18 Feb 2014 15:58:43 -0000 Delivered-To: apmail-commons-dev-archive@commons.apache.org Received: (qmail 57661 invoked by uid 500); 18 Feb 2014 15:58:42 -0000 Mailing-List: contact dev-help@commons.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Commons Developers List" Delivered-To: mailing list dev@commons.apache.org Received: (qmail 57653 invoked by uid 99); 18 Feb 2014 15:58:41 -0000 Received: from minotaur.apache.org (HELO minotaur.apache.org) (140.211.11.9) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 18 Feb 2014 15:58:41 +0000 Received: from localhost (HELO mail-we0-f176.google.com) (127.0.0.1) (smtp-auth username britter, mechanism plain) by minotaur.apache.org (qpsmtpd/0.29) with ESMTP; Tue, 18 Feb 2014 15:58:41 +0000 Received: by mail-we0-f176.google.com with SMTP id q58so11603270wes.21 for ; Tue, 18 Feb 2014 07:58:39 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=7tUxEsEdsn6Akgo6SXehSrz9oo8y1u5nIZZz5U/389Y=; b=J1+lwJ7VGEaJpdRf4jncLQ4wQemHAsQ5+C6RWgeZVZzB6T7JHwriVnoF00vefocAKt EfEyPXVcLalysGn/qApNK2K97UbJ1OOvyZOb2uTnrNH6/GvvLp/FTAPbpVRm16CX4nNS aiJ5XyF6/PpggrJVCXgJs/7Hucoz5JJX5JB7CkXT6AtiLgZbB/lspOxZTSyFa4MRfZW8 a+VPrdS0EtFLL0GfAUaa7R7xVgq3q2ihJosuMCkyNBcX8DJbw2Vj/nitr3TCUvzaWhsj YQwNr3/Pl+ftQw4rcuFpY3BvzwfyumpgVVWUfS+yfNt7OmFzD7LLdh00JVTDboKaiWJ1 dBng== MIME-Version: 1.0 X-Received: by 10.194.186.204 with SMTP id fm12mr23618541wjc.27.1392739119547; Tue, 18 Feb 2014 07:58:39 -0800 (PST) Received: by 10.194.6.231 with HTTP; Tue, 18 Feb 2014 07:58:39 -0800 (PST) In-Reply-To: References: <20140215103536.5BED923889D5@eris.apache.org> Date: Tue, 18 Feb 2014 16:58:39 +0100 Message-ID: Subject: Re: svn commit: r1568612 - in /commons/proper/lang/trunk/src: changes/ main/java/org/apache/commons/lang3/ main/java/org/apache/commons/lang3/text/translate/ test/java/org/apache/commons/lang3/ From: Benedikt Ritter To: Commons Developers List Content-Type: multipart/alternative; boundary=047d7bb04ba67c4e3704f2b0586d --047d7bb04ba67c4e3704f2b0586d Content-Type: text/plain; charset=ISO-8859-1 2014-02-18 16:28 GMT+01:00 Duncan Jones : > On 15 February 2014 10:35, wrote: > > Author: britter > > Date: Sat Feb 15 10:35:35 2014 > > New Revision: 1568612 > > > > URL: http://svn.apache.org/r1568612 > > Log: > > LANG-977: NumericEntityEscaper incorrectly encodes supplementary > characters. Thanks to Chris Karcher. > > > Chris isn't listed as a contributor in the POM file. Should we add him > or is the "due-to" field replacing the need for that? > Well there are no absolute rules about this. But in this case we could add him, I guess. I'll take care of this. Thanks! > > - Duncan > > > > > > > Modified: > > commons/proper/lang/trunk/src/changes/changes.xml > > > commons/proper/lang/trunk/src/main/java/org/apache/commons/lang3/StringEscapeUtils.java > > > commons/proper/lang/trunk/src/main/java/org/apache/commons/lang3/text/translate/CharSequenceTranslator.java > > > commons/proper/lang/trunk/src/test/java/org/apache/commons/lang3/StringEscapeUtilsTest.java > > > > Modified: commons/proper/lang/trunk/src/changes/changes.xml > > URL: > http://svn.apache.org/viewvc/commons/proper/lang/trunk/src/changes/changes.xml?rev=1568612&r1=1568611&r2=1568612&view=diff > > > ============================================================================== > > --- commons/proper/lang/trunk/src/changes/changes.xml [utf-8] (original) > > +++ commons/proper/lang/trunk/src/changes/changes.xml [utf-8] Sat Feb 15 > 10:35:35 2014 > > @@ -22,6 +22,7 @@ > > > > > > > > + NumericEntityEscaper incorrectly encodes supplementary > characters > > Make some private > fields final > > dev="sebb">NumberUtils#isNumber(String) fails to reject invalid Octal > numbers > > NumberUtils#isNumber > does not allow for hex 0XABCD > > > > Modified: > commons/proper/lang/trunk/src/main/java/org/apache/commons/lang3/StringEscapeUtils.java > > URL: > http://svn.apache.org/viewvc/commons/proper/lang/trunk/src/main/java/org/apache/commons/lang3/StringEscapeUtils.java?rev=1568612&r1=1568611&r2=1568612&view=diff > > > ============================================================================== > > --- > commons/proper/lang/trunk/src/main/java/org/apache/commons/lang3/StringEscapeUtils.java > (original) > > +++ > commons/proper/lang/trunk/src/main/java/org/apache/commons/lang3/StringEscapeUtils.java > Sat Feb 15 10:35:35 2014 > > @@ -185,7 +185,7 @@ public class StringEscapeUtils { > > out.write(StringUtils.replace(input.toString(), > CSV_QUOTE_STR, CSV_QUOTE_STR + CSV_QUOTE_STR)); > > out.write(CSV_QUOTE); > > } > > - return input.length(); > > + return Character.codePointCount(input, 0, input.length()); > > } > > } > > > > @@ -314,7 +314,7 @@ public class StringEscapeUtils { > > > > if ( input.charAt(0) != CSV_QUOTE || > input.charAt(input.length() - 1) != CSV_QUOTE ) { > > out.write(input.toString()); > > - return input.length(); > > + return Character.codePointCount(input, 0, > input.length()); > > } > > > > // strip quotes > > @@ -326,7 +326,7 @@ public class StringEscapeUtils { > > } else { > > out.write(input.toString()); > > } > > - return input.length(); > > + return Character.codePointCount(input, 0, input.length()); > > } > > } > > > > > > Modified: > commons/proper/lang/trunk/src/main/java/org/apache/commons/lang3/text/translate/CharSequenceTranslator.java > > URL: > http://svn.apache.org/viewvc/commons/proper/lang/trunk/src/main/java/org/apache/commons/lang3/text/translate/CharSequenceTranslator.java?rev=1568612&r1=1568611&r2=1568612&view=diff > > > ============================================================================== > > --- > commons/proper/lang/trunk/src/main/java/org/apache/commons/lang3/text/translate/CharSequenceTranslator.java > (original) > > +++ > commons/proper/lang/trunk/src/main/java/org/apache/commons/lang3/text/translate/CharSequenceTranslator.java > Sat Feb 15 10:35:35 2014 > > @@ -89,10 +89,10 @@ public abstract class CharSequenceTransl > > pos+= c.length; > > continue; > > } > > -// // contract with translators is that they have to > understand codepoints > > -// // and they just took care of a surrogate pair > > + // contract with translators is that they have to > understand codepoints > > + // and they just took care of a surrogate pair > > for (int pt = 0; pt < consumed; pt++) { > > - pos += Character.charCount(Character.codePointAt(input, > pt)); > > + pos += Character.charCount(Character.codePointAt(input, > pos)); > > } > > } > > } > > > > Modified: > commons/proper/lang/trunk/src/test/java/org/apache/commons/lang3/StringEscapeUtilsTest.java > > URL: > http://svn.apache.org/viewvc/commons/proper/lang/trunk/src/test/java/org/apache/commons/lang3/StringEscapeUtilsTest.java?rev=1568612&r1=1568611&r2=1568612&view=diff > > > ============================================================================== > > --- > commons/proper/lang/trunk/src/test/java/org/apache/commons/lang3/StringEscapeUtilsTest.java > (original) > > +++ > commons/proper/lang/trunk/src/test/java/org/apache/commons/lang3/StringEscapeUtilsTest.java > Sat Feb 15 10:35:35 2014 > > @@ -348,6 +348,9 @@ public class StringEscapeUtilsTest { > > > > assertEquals("Supplementary character must be represented using > a single escape", "𣎴", > > escapeXml.translate("\uD84C\uDFB4")); > > + > > + assertEquals("Supplementary characters mixed with basic > characters should be encoded correctly", "a b c 𣎴", > > + escapeXml.translate("a b c \uD84C\uDFB4")); > > } > > > > @Test > > @@ -377,6 +380,9 @@ public class StringEscapeUtilsTest { > > public void testUnescapeXmlSupplementaryCharacters() { > > assertEquals("Supplementary character must be represented using > a single escape", "\uD84C\uDFB4", > > StringEscapeUtils.unescapeXml("𣎴") ); > > + > > + assertEquals("Supplementary characters mixed with basic > characters should be decoded correctly", "a b c \uD84C\uDFB4", > > + StringEscapeUtils.unescapeXml("a b c 𣎴") ); > > } > > > > // Tests issue #38569 > > @@ -396,22 +402,24 @@ public class StringEscapeUtilsTest { > > > > @Test > > public void testEscapeCsvString() throws Exception { > > - assertEquals("foo.bar", > StringEscapeUtils.escapeCsv("foo.bar")); > > - assertEquals("\"foo,bar\"", > StringEscapeUtils.escapeCsv("foo,bar")); > > - assertEquals("\"foo\nbar\"", > StringEscapeUtils.escapeCsv("foo\nbar")); > > - assertEquals("\"foo\rbar\"", > StringEscapeUtils.escapeCsv("foo\rbar")); > > - assertEquals("\"foo\"\"bar\"", > StringEscapeUtils.escapeCsv("foo\"bar")); > > + assertEquals("foo.bar", > StringEscapeUtils.escapeCsv("foo.bar")); > > + assertEquals("\"foo,bar\"", > StringEscapeUtils.escapeCsv("foo,bar")); > > + assertEquals("\"foo\nbar\"", > StringEscapeUtils.escapeCsv("foo\nbar")); > > + assertEquals("\"foo\rbar\"", > StringEscapeUtils.escapeCsv("foo\rbar")); > > + assertEquals("\"foo\"\"bar\"", > StringEscapeUtils.escapeCsv("foo\"bar")); > > + assertEquals("foo\uD84C\uDFB4bar", > StringEscapeUtils.escapeCsv("foo\uD84C\uDFB4bar")); > > assertEquals("", StringEscapeUtils.escapeCsv("")); > > assertEquals(null, StringEscapeUtils.escapeCsv(null)); > > } > > > > @Test > > public void testEscapeCsvWriter() throws Exception { > > - checkCsvEscapeWriter("foo.bar", "foo.bar"); > > - checkCsvEscapeWriter("\"foo,bar\"", "foo,bar"); > > - checkCsvEscapeWriter("\"foo\nbar\"", "foo\nbar"); > > - checkCsvEscapeWriter("\"foo\rbar\"", "foo\rbar"); > > - checkCsvEscapeWriter("\"foo\"\"bar\"", "foo\"bar"); > > + checkCsvEscapeWriter("foo.bar", "foo.bar"); > > + checkCsvEscapeWriter("\"foo,bar\"", "foo,bar"); > > + checkCsvEscapeWriter("\"foo\nbar\"", "foo\nbar"); > > + checkCsvEscapeWriter("\"foo\rbar\"", "foo\rbar"); > > + checkCsvEscapeWriter("\"foo\"\"bar\"", "foo\"bar"); > > + checkCsvEscapeWriter("foo\uD84C\uDFB4bar", > "foo\uD84C\uDFB4bar"); > > checkCsvEscapeWriter("", null); > > checkCsvEscapeWriter("", ""); > > } > > @@ -428,11 +436,12 @@ public class StringEscapeUtilsTest { > > > > @Test > > public void testUnescapeCsvString() throws Exception { > > - assertEquals("foo.bar", > StringEscapeUtils.unescapeCsv("foo.bar")); > > - assertEquals("foo,bar", > StringEscapeUtils.unescapeCsv("\"foo,bar\"")); > > - assertEquals("foo\nbar", > StringEscapeUtils.unescapeCsv("\"foo\nbar\"")); > > - assertEquals("foo\rbar", > StringEscapeUtils.unescapeCsv("\"foo\rbar\"")); > > - assertEquals("foo\"bar", > StringEscapeUtils.unescapeCsv("\"foo\"\"bar\"")); > > + assertEquals("foo.bar", > StringEscapeUtils.unescapeCsv("foo.bar")); > > + assertEquals("foo,bar", > StringEscapeUtils.unescapeCsv("\"foo,bar\"")); > > + assertEquals("foo\nbar", > StringEscapeUtils.unescapeCsv("\"foo\nbar\"")); > > + assertEquals("foo\rbar", > StringEscapeUtils.unescapeCsv("\"foo\rbar\"")); > > + assertEquals("foo\"bar", > StringEscapeUtils.unescapeCsv("\"foo\"\"bar\"")); > > + assertEquals("foo\uD84C\uDFB4bar", > StringEscapeUtils.unescapeCsv("foo\uD84C\uDFB4bar")); > > assertEquals("", StringEscapeUtils.unescapeCsv("")); > > assertEquals(null, StringEscapeUtils.unescapeCsv(null)); > > > > @@ -441,11 +450,12 @@ public class StringEscapeUtilsTest { > > > > @Test > > public void testUnescapeCsvWriter() throws Exception { > > - checkCsvUnescapeWriter("foo.bar", "foo.bar"); > > - checkCsvUnescapeWriter("foo,bar", "\"foo,bar\""); > > - checkCsvUnescapeWriter("foo\nbar", "\"foo\nbar\""); > > - checkCsvUnescapeWriter("foo\rbar", "\"foo\rbar\""); > > - checkCsvUnescapeWriter("foo\"bar", "\"foo\"\"bar\""); > > + checkCsvUnescapeWriter("foo.bar", "foo.bar"); > > + checkCsvUnescapeWriter("foo,bar", "\"foo,bar\""); > > + checkCsvUnescapeWriter("foo\nbar", "\"foo\nbar\""); > > + checkCsvUnescapeWriter("foo\rbar", "\"foo\rbar\""); > > + checkCsvUnescapeWriter("foo\"bar", "\"foo\"\"bar\""); > > + checkCsvUnescapeWriter("foo\uD84C\uDFB4bar", > "foo\uD84C\uDFB4bar"); > > checkCsvUnescapeWriter("", null); > > checkCsvUnescapeWriter("", ""); > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org > For additional commands, e-mail: dev-help@commons.apache.org > > -- http://people.apache.org/~britter/ http://www.systemoutprintln.de/ http://twitter.com/BenediktRitter http://github.com/britter --047d7bb04ba67c4e3704f2b0586d--