couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam Kocoloski (JIRA)" <j...@apache.org>
Subject [jira] Updated: (COUCHDB-327) Replication of document results in invalid_json error
Date Fri, 05 Jun 2009 15:24:07 GMT

     [ https://issues.apache.org/jira/browse/COUCHDB-327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Adam Kocoloski updated COUCHDB-327:
-----------------------------------


I also cannot get that particular utf8_replication test to fail, but I agree that there is
a problem here.  I believe the root of the problem is mochijson2's inability to decode unicode
characters outside the Basic Multilingual Plane that are transmitted escaped as UTF-16 surrogate
pairs.  There's a couchdb and a  mochiweb issue for this:

https://issues.apache.org/jira/browse/COUCHDB-333
http://code.google.com/p/mochiweb/issues/detail?id=35

Replication breaks on documents containing raw UTF-8 corresponding to Unicode outside the
BMP, because the mochijson2 encoder transmits the characters as UTF-16 surrogate pairs, but
the decoder on the decoder on the other end can't handle that.

Matthew was able to work around the problem by making the encoder transmit raw unicode instead
of escaped.  I thought we were discussing switching to that mode of operation in another ticket,
but I can't find it at the moment.

I submitted a patch for this to mochiweb (in addition to one that was already available),
but we can also apply it in our repo now if so desired.


> Replication of document results in invalid_json error
> -----------------------------------------------------
>
>                 Key: COUCHDB-327
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-327
>             Project: CouchDB
>          Issue Type: Bug
>    Affects Versions: 0.9
>         Environment: Ubuntu 8.10
>            Reporter: Matthew Gregg
>
> While replicating a document containing UTf8 characters replication fails with invalid_json
errors in the receiving couch instance.
> The test below will cause the replication error.
> couchTests.utf8_replication = function(debug) {
>   if (debug) debugger;
>   var host = CouchDB.host;
>   var texts = [];
>   var dbPairs = [
>     {source:"http://"+host+"/test_suite_db_a",
>       target:"http://"+host+"/test_suite_db_b"}
>   ]
>   var dbA = new CouchDB("test_suite_db_a");
>   var dbB = new CouchDB("test_suite_db_b");
>   var numDocs = 10;
>   var xhr;
>   for (var testPair = 0; testPair < dbPairs.length; testPair++) {
>     var A = dbPairs[testPair].source
>     var B = dbPairs[testPair].target
>     dbA.deleteDb();
>     dbA.createDb();
>     dbB.deleteDb();
>     dbB.createDb();
>     text = "􀂀 Fakain: 􀂀 Faktor produksi yang digunakan hanya tenaga kerja saja.
􀂀 Kualitas ba";
>     dbA.save({_id:"1", text:text})
>     var result = CouchDB.replicate(A, B);
>   }
> };

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message