commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "anton (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LANG-708) StringEscapeUtils.escapeEcmaScript from lang3 cuts off long unicode string
Date Fri, 24 Jun 2011 08:12:47 GMT

     [ https://issues.apache.org/jira/browse/LANG-708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

anton updated LANG-708:
-----------------------

    Description: 
Hello, I have a really big JSON string (generated from db) with unicode chars and I need to
pass it though StringEscapeUtils.escapeEcmaScript(value). This string is generated and in
most cases it works ok, but I have met a specific string (attached below) which is not correctly
converted - few symbols (about 10) at the end of the string are cut-off (and actually they
are not already unicode chars).

the original string ends with:
 "geonameId":6544329,"valueCode":""}]

and the produced string ends with:
 \"geonameId\":6544329,\"value

So Code":""}] part is missing and this does not allow to parse the result as JSON on the client
side.


I have tried to debug a bit with StringEscapeUtils.escapeEcmaScript source code and is seems
that the problem is somewhere around here:

CharSequenceTranslator.translate(...){
...
        int sz = Character.codePointCount(input, 0, input.length());
        for (int i = 0; i < sz; i++) {
            // consumed is the number of codepoints consumed
            int consumed = translate(input, i, out);

            if(consumed == 0) { 
                out.write( Character.toChars( Character.codePointAt(input, i) ) );
            }
...
}

If I put breakpoint condition to stop in the loop when i==(sz-5), I can see that the last
chars of "valueCode" literal are being added to the end of "out" stream, but the counter condition
ends too early to reach the end of original input String.

So, it seems that somehow with the provided string either the sz value is calculated incorrectly
or the processing loop did wrong counter adjustmes at some point.


  was:
Hello, I have a really big JSON string (generated from db) with unicode chars and I need to
pass it though StringEscapeUtils.escapeEcmaScript(value). This string is generated and in
most cases it works ok, but I have met a specific string (attached below) which is not correctly
converted - few symbols (about 10) at the end of the string are cut-off (and actually they
are not already unicode chars).

the original string ends with:
 "geonameId":6544329,"valueCode":""}]

and the produced string ends with:
 \"geonameId\":6544329,\"value

So Code":""}] part is missing and this does not allow to parse the result as JSON on the client
side.


I have tried to debug a bit with StringEscapeUtils.escapeEcmaScript source code and is seems
that the problem is somewhere around here:

CharSequenceTranslator.translate(...){
...
        int sz = Character.codePointCount(input, 0, input.length());
        for (int i = 0; i < sz; i++) {
            // consumed is the number of codepoints consumed
            int consumed = translate(input, i, out);

            if(consumed == 0) { 
                out.write( Character.toChars( Character.codePointAt(input, i) ) );
            }
...
}

If I put breakpoint condition to stop in the loop when i==(sz-5), I can see that the last
chars of "valueCode" literal are being added to the end of "out" stream, but the counter condition
ends too early to reach the end of original input String.

So, it seems that somehow with the provided string either the sz value is calculated incorrectly
or the processing loop did wrong counter adjustmes at some point.

So this is the original string (sorry for big text in message body - I can't see where files
could be attached to this bugzilla):

"[{"geonameFeatureClass":"L","values":{"eu":"Mundua","ro":"Pamânt","it":"Globo","ca":"el
món","tr":"Yeryüzü","no":"Jorden","hu":"Föld","lv":"Zeme","de":"Welt","el":"Υδρόγειος","fi":"Maa","la":"Terra","fr":"Monde","eo":"Mondo","en":"World","ru":"Земля","es":"el
planeta","nl":"Aarde"},"geonameFeatureCode":"AREA","_id":32,"name":"","auto":true,"type":"GEO","geonameId":6295630,"valueCode":""},{"geonameFeatureClass":"L","values":{"ro":"Europa","zh":"欧洲","ca":"Europa","vi":"Châu
Âu","tr":"Avrupa","no":"Europa","hu":"Európa","lv":"Eiropa","hi":"यूरोप","lt":"Europa","bs":"Evropa","ga":"an
Eoraip","th":"ยุโรป","id":"Eropa","de":"Europa","fi":"Eurooppa","fr":"Europe","sv":"Europa","bg":"Европа","da":"Europa","eu":"Europa","is":"Evrópa","it":"Europa","cy":"Ewrop","ar":"أوروبا","se":"Eurohpá","he":"אירופה","cs":"Evropa","el":"Ευρώπη","nb":"Europa","pl":"Europa","la":"Europa","pt":"Europa","eo":"Eŭropo","en":"Europe","ru":"Европа","es":"Europa","ja":"ヨーロッパ","nl":"Europa"},"geonameFeatureCode":"CONT","_id":33,"name":"","auto":true,"type":"GEO","geonameId":6255148,"valueCode":""},{"geonameFeatureClass":"A","values":{"no":"Spania","nn":"Spania","fy":"Spanje","gd":"An
Spàinn","ga":"An Spáinn","oc":"Espanha","arc":"ܐܣܦܢܝܐ","fi":"Espanja","fr":"Espagne","fo":"Spania","udm":"Испания","os":"Испани","he":"ספרד","gn":"Epaña","gl":"España","gv":"Yn
Spaainey","pl":"Hiszpania","gu":"સ્પેઇન","lo":"ສະເປນ","ln":"Espania","vi":"Tây
Ban Nha","dz":"Spain","pms":"Spagna","lv":"Spānija","lt":"Ispanija","vo":"Spanyän","de":"Spanien","mg":"Espaina","fur":"Spagne","mk":"Шпанија","ml":"സ്പെയ്ന്\u200D","ceb":"Espanya","mi":"Pāniora","uk":"Іспанія","eu":"Espainia","mr":"स्पेन","ug":"ئىسپانىيە","mt":"Spanja","ms":"Sepanyol","ur":"سپین","fa":"اسپانیا","ty":"Paniora","new":"स्पेन","na":"Pain","el":"Ισπανία","nb":"Spania","ne":"स्पेन","vls":"Spanje","eo":"Hispanio","en":"Kingdom
of Spain","et":"Hispaania","es":"la Madre Patria","nl":"Spanje","vec":"Spagna","to":"Sepeni","ca":"Espanya","tl":"Espanya","tr":"İspanya","tg":"Испониё","haw":"Sepania","bs":"Španija","br":"Spagn","th":"ประเทศสเปน","bn":"স্পেন","bo":"སི་པན།","ta":"ஸ்பெயின்","sv":"Spanien","bg":"Испания","ka":"ესპანეთი","st":"Spain","sw":"Hispania","be":"Іспанія","kw":"Spayn","sl":"Španija","sk":"Španielsko","da":"Spanien","ang":"Spēonland","nds":"Spanien","ks":"Spēna","so":"Isbeyn","ku":"Spanya","sr":"Шпанија","sq":"Spanja","ko":"에스파냐","sc":"Ispagna","cy":"Sbaen","se":"Espánjja","sh":"Španija","cv":"Испани","km":"អេស្ប៉ាញ","cs":"Španělsko","li":"Spanje","co":"Spagna","default":"Spain","jbo":"sangu'e","la":"Hesperia","ru":"Испания","lb":"Spuenien","sco":"Spain","tet":"España","scn":"Spagna","hr":"Španjolska","zh":"西班牙","ro":"Spania","rm":"Spagna","ht":"Espay","hu":"Spanyolország","ast":"España","hi":"स्पेन","hsb":"Španiska","nah":"Caxtillān","war":"Espanya","lad":"Espanya","id":"Spanyol","ia":"Espania","nrm":"Espangne","hy":"Իսպանիա","qu":"Ispaña","ilo":"Espania","az":"İspaniya","is":"Spánn","it":"Spagna","tpi":"Spen","ar":"أسبانيا","io":"Hispania","pam":"Espanya","frp":"Èspagne","am":"እስፓንያ","an":"España","csb":"Szpańskô","pt":"Espanha","ja":"スペイン","ps":"اسپانيا","yi":"שפאניע","af":"Spanje"},"geonameFeatureCode":"PCLI","_id":260,"name":"","auto":true,"type":"GEO","geonameId":2510769,"valueCode":""},{"geonameFeatureClass":"A","values":{"ca":"Andalusia","tr":"Endülüs","krc":"Андалусия","no":"Andalucía","fy":"Andalûsje","bs":"Andaluzija","br":"Andalouzia","ext":"Andaluzia","ga":"An
Andalúis","th":"แคว้นอันดาลูซีอา","bn":"আন্দালুসিয়া","oc":"Andalosia","ka":"ანდალუსია","sv":"Andalusien","fr":"Andalousie","bg":"Андалусия","glk":"آندالوسیا","be":"Андалусія","kw":"Andalousi","sk":"Andalúzia","os":"Андалуси","da":"Andalusien","sr":"Андалузија","ku":"Endulus","ko":"안달루시아
지방","he":"אנדלוסיה","sh":"Andaluzija","arz":"اندلوسيا","cs":"Andalusie","default":"Andalusia","stq":"Andalusien","la":"Vandalitia","pl":"Andaluzja","ru":"Андалусия","lb":"Andalusien","tet":"Andaluzia","got":"𐍅𐌰𐌽𐌳𐌰𐌻𐌹𐍄𐌾𐌰","hr":"Andaluzija","zh":"安達魯西亞","ro":"Andaluzia","hu":"Andalúzia","pms":"Andalusìa","lv":"Andalūzija","lt":"Andalūzija","nah":"Andalucia","lad":"Andaluziya","de":"Andalusien","als":"Andalusien","qu":"Andalusiya","hy":"Անդալուզիա","eu":"Andaluzia","is":"Andalúsía","uk":"Андалусія","az":"Andalusiya","mr":"आंदालुसिया","ug":"Andalusiye","fa":"اندلس","ar":"أندلوسيا","rmy":"Andalusiya","io":"Andaluzia","el":"Ανδαλουσία","frp":"Andalosie","pt":"Andaluzia","eo":"Andaluzio","en":"Andalusia","et":"Andaluusia","es":"Andalucía","ja":"アンダルシア州","nl":"Andalusië","af":"Andalusië","vec":"Andalusìa"},"geonameFeatureCode":"ADM1","_id":261,"name":"","auto":true,"type":"GEO","geonameId":2593109,"valueCode":""},{"geonameFeatureClass":"A","values":{"de":"Granada","default":"Province
of Granada","fr":"Grenade","en":"Province of Granada","es":"Provincia de Granada","ja":"グラナダ"},"geonameFeatureCode":"ADM2","_id":262,"name":"","auto":true,"type":"GEO","geonameId":2517115,"valueCode":""},{"geonameFeatureClass":"A","values":{"default":"Monachil"},"geonameFeatureCode":"ADM3","_id":263,"name":"","auto":true,"type":"GEO","geonameId":6357744,"valueCode":""},{"geonameFeatureClass":"P","values":{"default":"Sierra
Nevada"},"geonameFeatureCode":"PPL","_id":264,"name":"","auto":true,"type":"GEO","geonameId":6544329,"valueCode":""}]"

and this is string, produced by StringEscapeUtils.escapeEcmaScript(value):

"[{\"geonameFeatureClass\":\"L\",\"values\":{\"eu\":\"Mundua\",\"ro\":\"Pam\u00E2nt\",\"it\":\"Globo\",\"ca\":\"el
m\u00F3n\",\"tr\":\"Yery\u00FCz\u00FC\",\"no\":\"Jorden\",\"hu\":\"F\u00F6ld\",\"lv\":\"Zeme\",\"de\":\"Welt\",\"el\":\"\u03A5\u03B4\u03C1\u03CC\u03B3\u03B5\u03B9\u03BF\u03C2\",\"fi\":\"Maa\",\"la\":\"Terra\",\"fr\":\"Monde\",\"eo\":\"Mondo\",\"en\":\"World\",\"ru\":\"\u0417\u0435\u043C\u043B\u044F\",\"es\":\"el
planeta\",\"nl\":\"Aarde\"},\"geonameFeatureCode\":\"AREA\",\"_id\":32,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6295630,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"L\",\"values\":{\"ro\":\"Europa\",\"zh\":\"\u6B27\u6D32\",\"ca\":\"Europa\",\"vi\":\"Ch\u00E2u
\u00C2u\",\"tr\":\"Avrupa\",\"no\":\"Europa\",\"hu\":\"Eur\u00F3pa\",\"lv\":\"Eiropa\",\"hi\":\"\u092F\u0942\u0930\u094B\u092A\",\"lt\":\"Europa\",\"bs\":\"Evropa\",\"ga\":\"an
Eoraip\",\"th\":\"\u0E22\u0E38\u0E42\u0E23\u0E1B\",\"id\":\"Eropa\",\"de\":\"Europa\",\"fi\":\"Eurooppa\",\"fr\":\"Europe\",\"sv\":\"Europa\",\"bg\":\"\u0415\u0432\u0440\u043E\u043F\u0430\",\"da\":\"Europa\",\"eu\":\"Europa\",\"is\":\"Evr\u00F3pa\",\"it\":\"Europa\",\"cy\":\"Ewrop\",\"ar\":\"\u0623\u0648\u0631\u0648\u0628\u0627\",\"se\":\"Eurohp\u00E1\",\"he\":\"\u05D0\u05D9\u05E8\u05D5\u05E4\u05D4\",\"cs\":\"Evropa\",\"el\":\"\u0395\u03C5\u03C1\u03CE\u03C0\u03B7\",\"nb\":\"Europa\",\"pl\":\"Europa\",\"la\":\"Europa\",\"pt\":\"Europa\",\"eo\":\"E\u016Dropo\",\"en\":\"Europe\",\"ru\":\"\u0415\u0432\u0440\u043E\u043F\u0430\",\"es\":\"Europa\",\"ja\":\"\u30E8\u30FC\u30ED\u30C3\u30D1\",\"nl\":\"Europa\"},\"geonameFeatureCode\":\"CONT\",\"_id\":33,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6255148,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"no\":\"Spania\",\"nn\":\"Spania\",\"fy\":\"Spanje\",\"gd\":\"An
Sp\u00E0inn\",\"ga\":\"An Sp\u00E1inn\",\"oc\":\"Espanha\",\"arc\":\"\u0710\u0723\u0726\u0722\u071D\u0710\",\"fi\":\"Espanja\",\"fr\":\"Espagne\",\"fo\":\"Spania\",\"udm\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"os\":\"\u0418\u0441\u043F\u0430\u043D\u0438\",\"he\":\"\u05E1\u05E4\u05E8\u05D3\",\"gn\":\"Epa\u00F1a\",\"gl\":\"Espa\u00F1a\",\"gv\":\"Yn
Spaainey\",\"pl\":\"Hiszpania\",\"gu\":\"\u0AB8\u0ACD\u0AAA\u0AC7\u0A87\u0AA8\",\"lo\":\"\u0EAA\u0EB0\u0EC0\u0E9B\u0E99\",\"ln\":\"Espania\",\"vi\":\"T\u00E2y
Ban Nha\",\"dz\":\"Spain\",\"pms\":\"Spagna\",\"lv\":\"Sp\u0101nija\",\"lt\":\"Ispanija\",\"vo\":\"Spany\u00E4n\",\"de\":\"Spanien\",\"mg\":\"Espaina\",\"fur\":\"Spagne\",\"mk\":\"\u0428\u043F\u0430\u043D\u0438\u0458\u0430\",\"ml\":\"\u0D38\u0D4D\u0D2A\u0D46\u0D2F\u0D4D\u0D28\u0D4D\\u200D\",\"ceb\":\"Espanya\",\"mi\":\"P\u0101niora\",\"uk\":\"\u0406\u0441\u043F\u0430\u043D\u0456\u044F\",\"eu\":\"Espainia\",\"mr\":\"\u0938\u094D\u092A\u0947\u0928\",\"ug\":\"\u0626\u0649\u0633\u067E\u0627\u0646\u0649\u064A\u06D5\",\"mt\":\"Spanja\",\"ms\":\"Sepanyol\",\"ur\":\"\u0633\u067E\u06CC\u0646\",\"fa\":\"\u0627\u0633\u067E\u0627\u0646\u06CC\u0627\",\"ty\":\"Paniora\",\"new\":\"\u0938\u094D\u092A\u0947\u0928\",\"na\":\"Pain\",\"el\":\"\u0399\u03C3\u03C0\u03B1\u03BD\u03AF\u03B1\",\"nb\":\"Spania\",\"ne\":\"\u0938\u094D\u092A\u0947\u0928\",\"vls\":\"Spanje\",\"eo\":\"Hispanio\",\"en\":\"Kingdom
of Spain\",\"et\":\"Hispaania\",\"es\":\"la Madre Patria\",\"nl\":\"Spanje\",\"vec\":\"Spagna\",\"to\":\"Sepeni\",\"ca\":\"Espanya\",\"tl\":\"Espanya\",\"tr\":\"\u0130spanya\",\"tg\":\"\u0418\u0441\u043F\u043E\u043D\u0438\u0451\",\"haw\":\"Sepania\",\"bs\":\"\u0160panija\",\"br\":\"Spagn\",\"th\":\"\u0E1B\u0E23\u0E30\u0E40\u0E17\u0E28\u0E2A\u0E40\u0E1B\u0E19\",\"bn\":\"\u09B8\u09CD\u09AA\u09C7\u09A8\",\"bo\":\"\u0F66\u0F72\u0F0B\u0F54\u0F53\u0F0D\",\"ta\":\"\u0BB8\u0BCD\u0BAA\u0BC6\u0BAF\u0BBF\u0BA9\u0BCD\",\"sv\":\"Spanien\",\"bg\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"ka\":\"\u10D4\u10E1\u10DE\u10D0\u10DC\u10D4\u10D7\u10D8\",\"st\":\"Spain\",\"sw\":\"Hispania\",\"be\":\"\u0406\u0441\u043F\u0430\u043D\u0456\u044F\",\"kw\":\"Spayn\",\"sl\":\"\u0160panija\",\"sk\":\"\u0160panielsko\",\"da\":\"Spanien\",\"ang\":\"Sp\u0113onland\",\"nds\":\"Spanien\",\"ks\":\"Sp\u0113na\",\"so\":\"Isbeyn\",\"ku\":\"Spanya\",\"sr\":\"\u0428\u043F\u0430\u043D\u0438\u0458\u0430\",\"sq\":\"Spanja\",\"ko\":\"\uC5D0\uC2A4\uD30C\uB0D0\",\"sc\":\"Ispagna\",\"cy\":\"Sbaen\",\"se\":\"Esp\u00E1njja\",\"sh\":\"\u0160panija\",\"cv\":\"\u0418\u0441\u043F\u0430\u043D\u0438\",\"km\":\"\u17A2\u17C1\u179F\u17D2\u1794\u17C9\u17B6\u1789\",\"cs\":\"\u0160pan\u011Blsko\",\"li\":\"Spanje\",\"co\":\"Spagna\",\"default\":\"Spain\",\"jbo\":\"sangu\'e\",\"la\":\"Hesperia\",\"ru\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"lb\":\"Spuenien\",\"sco\":\"Spain\",\"tet\":\"Espa\u00F1a\",\"scn\":\"Spagna\",\"hr\":\"\u0160panjolska\",\"zh\":\"\u897F\u73ED\u7259\",\"ro\":\"Spania\",\"rm\":\"Spagna\",\"ht\":\"Espay\",\"hu\":\"Spanyolorsz\u00E1g\",\"ast\":\"Espa\u00F1a\",\"hi\":\"\u0938\u094D\u092A\u0947\u0928\",\"hsb\":\"\u0160paniska\",\"nah\":\"Caxtill\u0101n\",\"war\":\"Espanya\",\"lad\":\"Espanya\",\"id\":\"Spanyol\",\"ia\":\"Espania\",\"nrm\":\"Espangne\",\"hy\":\"\u053B\u057D\u057A\u0561\u0576\u056B\u0561\",\"qu\":\"Ispa\u00F1a\",\"ilo\":\"Espania\",\"az\":\"\u0130spaniya\",\"is\":\"Sp\u00E1nn\",\"it\":\"Spagna\",\"tpi\":\"Spen\",\"ar\":\"\u0623\u0633\u0628\u0627\u0646\u064A\u0627\",\"io\":\"Hispania\",\"pam\":\"Espanya\",\"frp\":\"\u00C8spagne\",\"am\":\"\u12A5\u1235\u1353\u1295\u12EB\",\"an\":\"Espa\u00F1a\",\"csb\":\"Szpa\u0144sk\u00F4\",\"pt\":\"Espanha\",\"ja\":\"\u30B9\u30DA\u30A4\u30F3\",\"ps\":\"\u0627\u0633\u067E\u0627\u0646\u064A\u0627\",\"yi\":\"\u05E9\u05E4\u05D0\u05E0\u05D9\u05E2\",\"af\":\"Spanje\"},\"geonameFeatureCode\":\"PCLI\",\"_id\":260,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2510769,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"ca\":\"Andalusia\",\"tr\":\"End\u00FCl\u00FCs\",\"krc\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"no\":\"Andaluc\u00EDa\",\"fy\":\"Andal\u00FBsje\",\"bs\":\"Andaluzija\",\"br\":\"Andalouzia\",\"ext\":\"Andaluzia\",\"ga\":\"An
Andal\u00FAis\",\"th\":\"\u0E41\u0E04\u0E27\u0E49\u0E19\u0E2D\u0E31\u0E19\u0E14\u0E32\u0E25\u0E39\u0E0B\u0E35\u0E2D\u0E32\",\"bn\":\"\u0986\u09A8\u09CD\u09A6\u09BE\u09B2\u09C1\u09B8\u09BF\u09AF\u09BC\u09BE\",\"oc\":\"Andalosia\",\"ka\":\"\u10D0\u10DC\u10D3\u10D0\u10DA\u10E3\u10E1\u10D8\u10D0\",\"sv\":\"Andalusien\",\"fr\":\"Andalousie\",\"bg\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"glk\":\"\u0622\u0646\u062F\u0627\u0644\u0648\u0633\u06CC\u0627\",\"be\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0456\u044F\",\"kw\":\"Andalousi\",\"sk\":\"Andal\u00FAzia\",\"os\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\",\"da\":\"Andalusien\",\"sr\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0437\u0438\u0458\u0430\",\"ku\":\"Endulus\",\"ko\":\"\uC548\uB2EC\uB8E8\uC2DC\uC544
\uC9C0\uBC29\",\"he\":\"\u05D0\u05E0\u05D3\u05DC\u05D5\u05E1\u05D9\u05D4\",\"sh\":\"Andaluzija\",\"arz\":\"\u0627\u0646\u062F\u0644\u0648\u0633\u064A\u0627\",\"cs\":\"Andalusie\",\"default\":\"Andalusia\",\"stq\":\"Andalusien\",\"la\":\"Vandalitia\",\"pl\":\"Andaluzja\",\"ru\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"lb\":\"Andalusien\",\"tet\":\"Andaluzia\",\"got\":\"\u10345\u10330\u1033D\u10333\u10330\u1033B\u10339\u10344\u1033E\u10330\",\"hr\":\"Andaluzija\",\"zh\":\"\u5B89\u9054\u9B6F\u897F\u4E9E\",\"ro\":\"Andaluzia\",\"hu\":\"Andal\u00FAzia\",\"pms\":\"Andalus\u00ECa\",\"lv\":\"Andal\u016Bzija\",\"lt\":\"Andal\u016Bzija\",\"nah\":\"Andalucia\",\"lad\":\"Andaluziya\",\"de\":\"Andalusien\",\"als\":\"Andalusien\",\"qu\":\"Andalusiya\",\"hy\":\"\u0531\u0576\u0564\u0561\u056C\u0578\u0582\u0566\u056B\u0561\",\"eu\":\"Andaluzia\",\"is\":\"Andal\u00FAs\u00EDa\",\"uk\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0456\u044F\",\"az\":\"Andalusiya\",\"mr\":\"\u0906\u0902\u0926\u093E\u0932\u0941\u0938\u093F\u092F\u093E\",\"ug\":\"Andalusiye\",\"fa\":\"\u0627\u0646\u062F\u0644\u0633\",\"ar\":\"\u0623\u0646\u062F\u0644\u0648\u0633\u064A\u0627\",\"rmy\":\"Andalusiya\",\"io\":\"Andaluzia\",\"el\":\"\u0391\u03BD\u03B4\u03B1\u03BB\u03BF\u03C5\u03C3\u03AF\u03B1\",\"frp\":\"Andalosie\",\"pt\":\"Andaluzia\",\"eo\":\"Andaluzio\",\"en\":\"Andalusia\",\"et\":\"Andaluusia\",\"es\":\"Andaluc\u00EDa\",\"ja\":\"\u30A2\u30F3\u30C0\u30EB\u30B7\u30A2\u5DDE\",\"nl\":\"Andalusi\u00EB\",\"af\":\"Andalusi\u00EB\",\"vec\":\"Andalus\u00ECa\"},\"geonameFeatureCode\":\"ADM1\",\"_id\":261,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2593109,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"de\":\"Granada\",\"default\":\"Province
of Granada\",\"fr\":\"Grenade\",\"en\":\"Province of Granada\",\"es\":\"Provincia de Granada\",\"ja\":\"\u30B0\u30E9\u30CA\u30C0\"},\"geonameFeatureCode\":\"ADM2\",\"_id\":262,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2517115,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"default\":\"Monachil\"},\"geonameFeatureCode\":\"ADM3\",\"_id\":263,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6357744,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"P\",\"values\":{\"default\":\"Sierra
Nevada\"},\"geonameFeatureCode\":\"PPL\",\"_id\":264,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6544329,\"value"


> StringEscapeUtils.escapeEcmaScript from lang3 cuts off long unicode string
> --------------------------------------------------------------------------
>
>                 Key: LANG-708
>                 URL: https://issues.apache.org/jira/browse/LANG-708
>             Project: Commons Lang
>          Issue Type: Bug
>            Reporter: anton
>         Attachments: Test.java, input.txt
>
>
> Hello, I have a really big JSON string (generated from db) with unicode chars and I need
to pass it though StringEscapeUtils.escapeEcmaScript(value). This string is generated and
in most cases it works ok, but I have met a specific string (attached below) which is not
correctly converted - few symbols (about 10) at the end of the string are cut-off (and actually
they are not already unicode chars).
> the original string ends with:
>  "geonameId":6544329,"valueCode":""}]
> and the produced string ends with:
>  \"geonameId\":6544329,\"value
> So Code":""}] part is missing and this does not allow to parse the result as JSON on
the client side.
> I have tried to debug a bit with StringEscapeUtils.escapeEcmaScript source code and is
seems that the problem is somewhere around here:
> CharSequenceTranslator.translate(...){
> ...
>         int sz = Character.codePointCount(input, 0, input.length());
>         for (int i = 0; i < sz; i++) {
>             // consumed is the number of codepoints consumed
>             int consumed = translate(input, i, out);
>             if(consumed == 0) { 
>                 out.write( Character.toChars( Character.codePointAt(input, i) ) );
>             }
> ...
> }
> If I put breakpoint condition to stop in the loop when i==(sz-5), I can see that the
last chars of "valueCode" literal are being added to the end of "out" stream, but the counter
condition ends too early to reach the end of original input String.
> So, it seems that somehow with the provided string either the sz value is calculated
incorrectly or the processing loop did wrong counter adjustmes at some point.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message