Return-Path: Delivered-To: apmail-couchdb-commits-archive@www.apache.org Received: (qmail 10352 invoked from network); 26 Jul 2010 17:22:31 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 26 Jul 2010 17:22:31 -0000 Received: (qmail 49396 invoked by uid 500); 26 Jul 2010 17:22:30 -0000 Delivered-To: apmail-couchdb-commits-archive@couchdb.apache.org Received: (qmail 49271 invoked by uid 500); 26 Jul 2010 17:22:29 -0000 Mailing-List: contact commits-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list commits@couchdb.apache.org Received: (qmail 49145 invoked by uid 99); 26 Jul 2010 17:22:29 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 26 Jul 2010 17:22:29 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 26 Jul 2010 17:22:25 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id E492F2388A39; Mon, 26 Jul 2010 17:21:32 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r979368 [1/3] - in /couchdb/trunk: etc/couchdb/ share/www/script/ share/www/script/jspec/ share/www/script/test/ src/couchdb/ src/mochiweb/ Date: Mon, 26 Jul 2010 17:21:32 -0000 To: commits@couchdb.apache.org From: rnewson@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20100726172132.E492F2388A39@eris.apache.org> Author: rnewson Date: Mon Jul 26 17:21:30 2010 New Revision: 979368 URL: http://svn.apache.org/viewvc?rev=979368&view=rev Log: Add SSL support to CouchDB. To enable SSL you need to do three things; 1) enable the httpsd daemon in local.ini (you can just uncomment the line). 2) supply your PEM-encoded cert and key files in the [ssl] section. 3) start CouchDB. CouchDB will now, in addition to handling HTTP on port 5984, accept SSL connections on port 6984. The patch itself adds SSL support by updating the local version of Mochiweb to the latest. The upstream release includes our local tweak to support large numbers and to handle Accept-Encoding headers. Our local Mochiweb fork changed the default idle timeout from 10 seconds to 5 minutes, and it was agreed on #irc to revert this change. The only tweaks to Mochiweb were in mochiweb.app.src (to record the git commit I built from) and the removal of Makefile (replaced by Makefile.am). Futon received many tweaks as we have 'http://' hardcoded all over. All such instances now use window.location.protocol + '//'. CouchDB received a tweak to use the right scheme in couch_httpd:absolute_uri (it now gets it from the Mochireq and not mochiweb_socket_server). Added: couchdb/trunk/src/mochiweb/internal.hrl couchdb/trunk/src/mochiweb/mochiglobal.erl couchdb/trunk/src/mochiweb/mochilists.erl couchdb/trunk/src/mochiweb/mochilogfile2.erl couchdb/trunk/src/mochiweb/mochitemp.erl couchdb/trunk/src/mochiweb/mochiutf8.erl couchdb/trunk/src/mochiweb/mochiweb.app.src couchdb/trunk/src/mochiweb/mochiweb_acceptor.erl couchdb/trunk/src/mochiweb/mochiweb_cover.erl couchdb/trunk/src/mochiweb/mochiweb_io.erl couchdb/trunk/src/mochiweb/mochiweb_mime.erl couchdb/trunk/src/mochiweb/mochiweb_socket.erl Modified: couchdb/trunk/etc/couchdb/default.ini.tpl.in couchdb/trunk/etc/couchdb/local.ini couchdb/trunk/share/www/script/couch.js couchdb/trunk/share/www/script/jspec/jspec.js couchdb/trunk/share/www/script/test/basics.js couchdb/trunk/share/www/script/test/config.js couchdb/trunk/share/www/script/test/http.js couchdb/trunk/share/www/script/test/oauth.js couchdb/trunk/share/www/script/test/replication.js couchdb/trunk/share/www/script/test/security_validation.js couchdb/trunk/src/couchdb/couch_httpd.erl couchdb/trunk/src/mochiweb/Makefile.am couchdb/trunk/src/mochiweb/mochifmt.erl couchdb/trunk/src/mochiweb/mochifmt_records.erl couchdb/trunk/src/mochiweb/mochifmt_std.erl couchdb/trunk/src/mochiweb/mochihex.erl couchdb/trunk/src/mochiweb/mochijson.erl couchdb/trunk/src/mochiweb/mochijson2.erl couchdb/trunk/src/mochiweb/mochinum.erl couchdb/trunk/src/mochiweb/mochiweb.app.in couchdb/trunk/src/mochiweb/mochiweb.erl couchdb/trunk/src/mochiweb/mochiweb_app.erl couchdb/trunk/src/mochiweb/mochiweb_charref.erl couchdb/trunk/src/mochiweb/mochiweb_cookies.erl couchdb/trunk/src/mochiweb/mochiweb_echo.erl couchdb/trunk/src/mochiweb/mochiweb_headers.erl couchdb/trunk/src/mochiweb/mochiweb_html.erl couchdb/trunk/src/mochiweb/mochiweb_http.erl couchdb/trunk/src/mochiweb/mochiweb_multipart.erl couchdb/trunk/src/mochiweb/mochiweb_request.erl couchdb/trunk/src/mochiweb/mochiweb_response.erl couchdb/trunk/src/mochiweb/mochiweb_skel.erl couchdb/trunk/src/mochiweb/mochiweb_socket_server.erl couchdb/trunk/src/mochiweb/mochiweb_sup.erl couchdb/trunk/src/mochiweb/mochiweb_util.erl couchdb/trunk/src/mochiweb/reloader.erl Modified: couchdb/trunk/etc/couchdb/default.ini.tpl.in URL: http://svn.apache.org/viewvc/couchdb/trunk/etc/couchdb/default.ini.tpl.in?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/etc/couchdb/default.ini.tpl.in (original) +++ couchdb/trunk/etc/couchdb/default.ini.tpl.in Mon Jul 26 17:21:30 2010 @@ -23,6 +23,9 @@ secure_rewrites = true vhost_global_handlers = _utils, _uuids, _session, _oauth, _users allow_jsonp = false +[ssl] +port = 6984 + [log] file = %localstatelogdir%/couch.log level = info Modified: couchdb/trunk/etc/couchdb/local.ini URL: http://svn.apache.org/viewvc/couchdb/trunk/etc/couchdb/local.ini?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/etc/couchdb/local.ini (original) +++ couchdb/trunk/etc/couchdb/local.ini Mon Jul 26 17:21:30 2010 @@ -29,6 +29,13 @@ [log] ;level = debug +[daemons] +; enable SSL support by uncommenting the following line and supply the PEM's below. +; httpsd = {couch_httpd, start_link, [https]} + +[ssl] +;cert_file = /full/path/to/server_cert.pem +;key_file = /full/path/to/server_key.pem ; To enable Virtual Hosts in CouchDB, add a vhost = path directive. All requests to ; the Virual Host will be redirected to the path. In the example below all requests Modified: couchdb/trunk/share/www/script/couch.js URL: http://svn.apache.org/viewvc/couchdb/trunk/share/www/script/couch.js?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/share/www/script/couch.js [utf-8] (original) +++ couchdb/trunk/share/www/script/couch.js [utf-8] Mon Jul 26 17:21:30 2010 @@ -402,7 +402,8 @@ CouchDB.request = function(method, uri, options.headers["Content-Type"] = options.headers["Content-Type"] || options.headers["content-type"] || "application/json"; options.headers["Accept"] = options.headers["Accept"] || options.headers["accept"] || "application/json"; var req = CouchDB.newXhr(); - if(uri.substr(0, "http://".length) != "http://") { + var proto = window.location.protocol + "//"; + if(uri.substr(0, proto.length) != proto) { uri = CouchDB.urlPrefix + uri } req.open(method, uri, false); Modified: couchdb/trunk/share/www/script/jspec/jspec.js URL: http://svn.apache.org/viewvc/couchdb/trunk/share/www/script/jspec/jspec.js?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/share/www/script/jspec/jspec.js (original) +++ couchdb/trunk/share/www/script/jspec/jspec.js Mon Jul 26 17:21:30 2010 @@ -87,7 +87,7 @@ */ Server : function(results, options) { - var uri = options.uri || 'http://' + window.location.host + '/results' + var uri = options.uri || window.location.protocol + "//" + window.location.host + '/results' JSpec.post(uri, { stats: JSpec.stats, options: options, Modified: couchdb/trunk/share/www/script/test/basics.js URL: http://svn.apache.org/viewvc/couchdb/trunk/share/www/script/test/basics.js?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/share/www/script/test/basics.js (original) +++ couchdb/trunk/share/www/script/test/basics.js Mon Jul 26 17:21:30 2010 @@ -37,9 +37,9 @@ couchTests.basics = function(debug) { TEquals(dbname, xhr.getResponseHeader("Location").substr(-dbname.length), "should return Location header to newly created document"); - - TEquals("http://", - xhr.getResponseHeader("Location").substr(0, 7), + var expected = window.location.protocol + "//"; + TEquals(expected, + xhr.getResponseHeader("Location").substr(0, expected.length), "should return absolute Location header to newly created document"); }); @@ -181,9 +181,9 @@ couchTests.basics = function(debug) { TEquals("/test_suite_db/newdoc", xhr.getResponseHeader("Location").substr(-21), "should return Location header to newly created document"); - - TEquals("http://", - xhr.getResponseHeader("Location").substr(0, 7), + var expected = window.location.protocol + "//"; + TEquals(expected, + xhr.getResponseHeader("Location").substr(0, expected.length), "should return absolute Location header to newly created document"); // deleting a non-existent doc should be 404 Modified: couchdb/trunk/share/www/script/test/config.js URL: http://svn.apache.org/viewvc/couchdb/trunk/share/www/script/test/config.js?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/share/www/script/test/config.js (original) +++ couchdb/trunk/share/www/script/test/config.js Mon Jul 26 17:21:30 2010 @@ -28,8 +28,8 @@ couchTests.config = function(debug) { Overengineering FTW. */ var server_port = CouchDB.host.split(':'); + var proto = window.location.protocol; if(server_port.length == 1 && CouchDB.inBrowser) { - var proto = window.location.protocol; if(proto == "http:") { port = 80; } @@ -40,8 +40,15 @@ couchTests.config = function(debug) { port = server_port.pop(); } + if(proto == "http:") { + config_port = config.httpd.port; + } + if(proto == "https:") { + config_port = config.ssl.port; + } + if(port) { - T(config.httpd.port == port); + TEquals(config_port, port, "ports should match"); } T(config.couchdb.database_dir); Modified: couchdb/trunk/share/www/script/test/http.js URL: http://svn.apache.org/viewvc/couchdb/trunk/share/www/script/test/http.js?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/share/www/script/test/http.js (original) +++ couchdb/trunk/share/www/script/test/http.js Mon Jul 26 17:21:30 2010 @@ -25,7 +25,7 @@ couchTests.http = function(debug) { var xhr = CouchDB.request("PUT", "/test_suite_db/test", {body: "{}"}); var host = CouchDB.host; - TEquals("http://" + host + "/test_suite_db/test", + TEquals(window.location.protocol + "//" + host + "/test_suite_db/test", xhr.getResponseHeader("Location"), "should include ip address"); @@ -34,7 +34,7 @@ couchTests.http = function(debug) { headers: {"X-Forwarded-Host": "mysite.com"} }); - TEquals("http://mysite.com/test_suite_db/test2", + TEquals(window.location.protocol + "//" + "mysite.com/test_suite_db/test2", xhr.getResponseHeader("Location"), "should include X-Forwarded-Host"); @@ -47,7 +47,7 @@ couchTests.http = function(debug) { body: "{}", headers: {"X-Host": "mysite2.com"} }); - TEquals("http://mysite2.com/test_suite_db/test3", + TEquals(window.location.protocol + "//" + "mysite2.com/test_suite_db/test3", xhr.getResponseHeader("Location"), "should include X-Host"); }); Modified: couchdb/trunk/share/www/script/test/oauth.js URL: http://svn.apache.org/viewvc/couchdb/trunk/share/www/script/test/oauth.js?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/share/www/script/test/oauth.js (original) +++ couchdb/trunk/share/www/script/test/oauth.js Mon Jul 26 17:21:30 2010 @@ -71,7 +71,7 @@ couchTests.oauth = function(debug) { var host = CouchDB.host; var dbPair = { source: { - url: "http://" + host + "/test_suite_db_a", + url: window.location.protocol + "//" + host + "/test_suite_db_a", auth: { oauth: { consumer_key: "key", @@ -82,7 +82,7 @@ couchTests.oauth = function(debug) { } }, target: { - url: "http://" + host + "/test_suite_db_b", + url: window.location.protocol + "//" + host + "/test_suite_db_b", headers: {"Authorization": adminBasicAuthHeaderValue()} } }; @@ -90,7 +90,7 @@ couchTests.oauth = function(debug) { // this function will be called on the modified server var testFun = function () { try { - CouchDB.request("PUT", "http://" + host + "/_config/admins/testadmin", { + CouchDB.request("PUT", window.location.protocol + "//" + host + "/_config/admins/testadmin", { headers: {"X-Couch-Persist": "false"}, body: JSON.stringify(testadminPassword) }); @@ -98,7 +98,7 @@ couchTests.oauth = function(debug) { waitForSuccess(function() { //loop until the couch server has processed the password i += 1; - var xhr = CouchDB.request("GET", "http://" + host + "/_config/admins/testadmin?foo="+i,{ + var xhr = CouchDB.request("GET", window.location.protocol + "//" + host + "/_config/admins/testadmin?foo="+i,{ headers: { "Authorization": adminBasicAuthHeaderValue() }}); @@ -109,7 +109,7 @@ couchTests.oauth = function(debug) { CouchDB.newUuids(2); // so we have one to make the salt - CouchDB.request("PUT", "http://" + host + "/_config/couch_httpd_auth/require_valid_user", { + CouchDB.request("PUT", window.location.protocol + "//" + host + "/_config/couch_httpd_auth/require_valid_user", { headers: { "X-Couch-Persist": "false", "Authorization": adminBasicAuthHeaderValue() @@ -157,11 +157,11 @@ couchTests.oauth = function(debug) { }; // Get request token via Authorization header - xhr = oauthRequest("GET", "http://" + host + "/_oauth/request_token", message, accessor); + xhr = oauthRequest("GET", window.location.protocol + "//" + host + "/_oauth/request_token", message, accessor); T(xhr.status == expectedCode); // GET request token via query parameters - xhr = oauthRequest("GET", "http://" + host + "/_oauth/request_token", message, accessor); + xhr = oauthRequest("GET", window.location.protocol + "//" + host + "/_oauth/request_token", message, accessor); T(xhr.status == expectedCode); responseMessage = OAuth.decodeForm(xhr.responseText); @@ -171,7 +171,7 @@ couchTests.oauth = function(debug) { //xhr = CouchDB.request("GET", authorization_url + '?oauth_token=' + responseMessage.oauth_token); //T(xhr.status == expectedCode); - xhr = oauthRequest("GET", "http://" + host + "/_session", message, accessor); + xhr = oauthRequest("GET", window.location.protocol + "//" + host + "/_session", message, accessor); T(xhr.status == expectedCode); if (xhr.status == expectedCode == 200) { data = JSON.parse(xhr.responseText); @@ -179,11 +179,11 @@ couchTests.oauth = function(debug) { T(data.roles[0] == "test"); } - xhr = oauthRequest("GET", "http://" + host + "/_session?foo=bar", message, accessor); + xhr = oauthRequest("GET", window.location.protocol + "//" + host + "/_session?foo=bar", message, accessor); T(xhr.status == expectedCode); // Test HEAD method - xhr = oauthRequest("HEAD", "http://" + host + "/_session?foo=bar", message, accessor); + xhr = oauthRequest("HEAD", window.location.protocol + "//" + host + "/_session?foo=bar", message, accessor); T(xhr.status == expectedCode); // Replication @@ -207,7 +207,7 @@ couchTests.oauth = function(debug) { oauth_version: "1.0" } }; - xhr = oauthRequest("GET", "http://" + host + "/_session?foo=bar", message, adminAccessor); + xhr = oauthRequest("GET", window.location.protocol + "//" + host + "/_session?foo=bar", message, adminAccessor); if (xhr.status == expectedCode == 200) { data = JSON.parse(xhr.responseText); T(data.name == "testadmin"); @@ -216,13 +216,13 @@ couchTests.oauth = function(debug) { // Test when the user's token doesn't exist. message.parameters.oauth_token = "not a token!"; - xhr = oauthRequest("GET", "http://" + host + "/_session?foo=bar", + xhr = oauthRequest("GET", window.location.protocol + "//" + host + "/_session?foo=bar", message, adminAccessor); T(xhr.status == 400, "Request should be invalid."); } } } finally { - var xhr = CouchDB.request("PUT", "http://" + host + "/_config/couch_httpd_auth/require_valid_user", { + var xhr = CouchDB.request("PUT", window.location.protocol + "//" + host + "/_config/couch_httpd_auth/require_valid_user", { headers: { "Authorization": adminBasicAuthHeaderValue(), "X-Couch-Persist": "false" @@ -231,7 +231,7 @@ couchTests.oauth = function(debug) { }); T(xhr.status == 200); - var xhr = CouchDB.request("DELETE", "http://" + host + "/_config/admins/testadmin", { + var xhr = CouchDB.request("DELETE", window.location.protocol + "//" + host + "/_config/admins/testadmin", { headers: { "Authorization": adminBasicAuthHeaderValue(), "X-Couch-Persist": "false" Modified: couchdb/trunk/share/www/script/test/replication.js URL: http://svn.apache.org/viewvc/couchdb/trunk/share/www/script/test/replication.js?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/share/www/script/test/replication.js (original) +++ couchdb/trunk/share/www/script/test/replication.js Mon Jul 26 17:21:30 2010 @@ -17,11 +17,11 @@ couchTests.replication = function(debug) {source:"test_suite_db_a", target:"test_suite_db_b"}, {source:"test_suite_db_a", - target:"http://" + host + "/test_suite_db_b"}, - {source:"http://" + host + "/test_suite_db_a", + target:window.location.protocol + "//" + host + "/test_suite_db_b"}, + {source:window.location.protocol + "//" + host + "/test_suite_db_a", target:"test_suite_db_b"}, - {source:"http://" + host + "/test_suite_db_a", - target:"http://" + host + "/test_suite_db_b"} + {source:window.location.protocol + "//" + host + "/test_suite_db_a", + target:window.location.protocol + "//" + host + "/test_suite_db_b"} ] var dbA = new CouchDB("test_suite_db_a", {"X-Couch-Full-Commit":"false"}); var dbB = new CouchDB("test_suite_db_b", {"X-Couch-Full-Commit":"false"}); @@ -296,7 +296,7 @@ couchTests.replication = function(debug) // remote dbB.deleteDb(); - CouchDB.replicate(dbA.name, "http://" + CouchDB.host + "/test_suite_db_b", { + CouchDB.replicate(dbA.name, window.location.protocol + "//" + CouchDB.host + "/test_suite_db_b", { body: {"create_target": true} }); TEquals("test_suite_db_b", dbB.info().db_name, @@ -372,11 +372,11 @@ couchTests.replication = function(debug) {source:"test_suite_rep_docs_db_a", target:"test_suite_rep_docs_db_b"}, {source:"test_suite_rep_docs_db_a", - target:"http://" + host + "/test_suite_rep_docs_db_b"}, - {source:"http://" + host + "/test_suite_rep_docs_db_a", + target:window.location.protocol + "//" + host + "/test_suite_rep_docs_db_b"}, + {source:window.location.protocol + "//" + host + "/test_suite_rep_docs_db_a", target:"test_suite_rep_docs_db_b"}, - {source:"http://" + host + "/test_suite_rep_docs_db_a", - target:"http://" + host + "/test_suite_rep_docs_db_b"} + {source:window.location.protocol + "//" + host + "/test_suite_rep_docs_db_a", + target:window.location.protocol + "//" + host + "/test_suite_rep_docs_db_b"} ]; var target_doc_ids = [ @@ -481,11 +481,11 @@ couchTests.replication = function(debug) {source:"test_suite_filtered_rep_db_a", target:"test_suite_filtered_rep_db_b"}, {source:"test_suite_filtered_rep_db_a", - target:"http://" + host + "/test_suite_filtered_rep_db_b"}, - {source:"http://" + host + "/test_suite_filtered_rep_db_a", + target:window.location.protocol + "//" + host + "/test_suite_filtered_rep_db_b"}, + {source:window.location.protocol + "//" + host + "/test_suite_filtered_rep_db_a", target:"test_suite_filtered_rep_db_b"}, - {source:"http://" + host + "/test_suite_filtered_rep_db_a", - target:"http://" + host + "/test_suite_filtered_rep_db_b"} + {source:window.location.protocol + "//" + host + "/test_suite_filtered_rep_db_a", + target:window.location.protocol + "//" + host + "/test_suite_filtered_rep_db_b"} ]; for (var i = 0; i < dbPairs.length; i++) { Modified: couchdb/trunk/share/www/script/test/security_validation.js URL: http://svn.apache.org/viewvc/couchdb/trunk/share/www/script/test/security_validation.js?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/share/www/script/test/security_validation.js (original) +++ couchdb/trunk/share/www/script/test/security_validation.js Mon Jul 26 17:21:30 2010 @@ -235,16 +235,16 @@ couchTests.security_validation = functio target:"test_suite_db_b"}, {source:"test_suite_db_a", - target:{url: "http://" + host + "/test_suite_db_b", + target:{url: window.location.protocol + "//" + host + "/test_suite_db_b", headers: AuthHeaders}}, - {source:{url:"http://" + host + "/test_suite_db_a", + {source:{url:window.location.protocol + "//" + host + "/test_suite_db_a", headers: AuthHeaders}, target:"test_suite_db_b"}, - {source:{url:"http://" + host + "/test_suite_db_a", + {source:{url:window.location.protocol + "//" + host + "/test_suite_db_a", headers: AuthHeaders}, - target:{url:"http://" + host + "/test_suite_db_b", + target:{url:window.location.protocol + "//" + host + "/test_suite_db_b", headers: AuthHeaders}}, ] var adminDbA = new CouchDB("test_suite_db_a", {"X-Couch-Full-Commit":"false"}); Modified: couchdb/trunk/src/couchdb/couch_httpd.erl URL: http://svn.apache.org/viewvc/couchdb/trunk/src/couchdb/couch_httpd.erl?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/src/couchdb/couch_httpd.erl (original) +++ couchdb/trunk/src/couchdb/couch_httpd.erl Mon Jul 26 17:21:30 2010 @@ -13,7 +13,7 @@ -module(couch_httpd). -include("couch_db.hrl"). --export([start_link/0, stop/0, handle_request/7]). +-export([start_link/0, start_link/1, stop/0, handle_request/7]). -export([header_value/2,header_value/3,qs_value/2,qs_value/3,qs/1,path/1,absolute_uri/2,body_length/1]). -export([verify_is_server_admin/1,unquote/1,quote/1,recv/2,recv_chunked/4,error_info/1]). @@ -28,14 +28,34 @@ -export([accepted_encodings/1,handle_request_int/5,validate_referer/1,validate_ctype/2]). start_link() -> + start_link(http). +start_link(http) -> + Port = couch_config:get("httpd", "port", "5984"), + start_link(?MODULE, [{port, Port}]); +start_link(https) -> + Port = couch_config:get("ssl", "port", "5984"), + CertFile = couch_config:get("ssl", "cert_file", nil), + KeyFile = couch_config:get("ssl", "key_file", nil), + Options = case CertFile /= nil andalso KeyFile /= nil of + true -> + [{port, Port}, + {ssl, true}, + {ssl_opts, [ + {certfile, CertFile}, + {keyfile, KeyFile}]}]; + false -> + io:format("SSL enabled but PEM certificates are missing.", []), + throw({error, missing_certs}) + end, + start_link(https, Options). +start_link(Name, Options) -> % read config and register for configuration changes % just stop if one of the config settings change. couch_server_sup % will restart us and then we will pick up the new settings. BindAddress = couch_config:get("httpd", "bind_address", any), - Port = couch_config:get("httpd", "port", "5984"), - MaxConnections = couch_config:get("httpd", "max_connections", "2048"), + %% MaxConnections = couch_config:get("httpd", "max_connections", "2048"), VirtualHosts = couch_config:get("vhosts"), VhostGlobals = re:split( couch_config:get("httpd", "vhost_global_handlers", ""), @@ -74,12 +94,10 @@ start_link() -> % and off we go - {ok, Pid} = case mochiweb_http:start([ + {ok, Pid} = case mochiweb_http:start(Options ++ [ {loop, Loop}, - {name, ?MODULE}, - {ip, BindAddress}, - {port, Port}, - {max, MaxConnections} + {name, Name}, + {ip, BindAddress} ]) of {ok, MochiPid} -> {ok, MochiPid}; {error, Reason} -> @@ -101,6 +119,8 @@ start_link() -> ("httpd_db_handlers", _) -> ?MODULE:stop(); ("vhosts", _) -> + ?MODULE:stop(); + ("ssl", _) -> ?MODULE:stop() end, Pid), @@ -430,15 +450,18 @@ absolute_uri(#httpd{mochi_req=MochiReq}= Host = host_for_request(Req), XSsl = couch_config:get("httpd", "x_forwarded_ssl", "X-Forwarded-Ssl"), Scheme = case MochiReq:get_header_value(XSsl) of - "on" -> "https"; - _ -> - XProto = couch_config:get("httpd", "x_forwarded_proto", "X-Forwarded-Proto"), - case MochiReq:get_header_value(XProto) of - % Restrict to "https" and "http" schemes only - "https" -> "https"; - _ -> "http" - end - end, + "on" -> "https"; + _ -> + XProto = couch_config:get("httpd", "x_forwarded_proto", "X-Forwarded-Proto"), + case MochiReq:get_header_value(XProto) of + %% Restrict to "https" and "http" schemes only + "https" -> "https"; + _ -> case MochiReq:get(scheme) of + https -> "https"; + http -> "http" + end + end + end, Scheme ++ "://" ++ Host ++ Path. unquote(UrlEncodedString) -> Modified: couchdb/trunk/src/mochiweb/Makefile.am URL: http://svn.apache.org/viewvc/couchdb/trunk/src/mochiweb/Makefile.am?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/src/mochiweb/Makefile.am (original) +++ couchdb/trunk/src/mochiweb/Makefile.am Mon Jul 26 17:21:30 2010 @@ -10,59 +10,79 @@ ## License for the specific language governing permissions and limitations under ## the License. -mochiwebebindir = $(localerlanglibdir)/mochiweb-r113/ebin +mochiwebebindir = $(localerlanglibdir)/mochiweb-7c2bc2/ebin mochiweb_file_collection = \ - mochifmt.erl \ - mochifmt_records.erl \ - mochifmt_std.erl \ - mochihex.erl \ - mochijson.erl \ - mochijson2.erl \ - mochinum.erl \ + mochifmt.erl \ + mochifmt_records.erl \ + mochifmt_std.erl \ + mochiglobal.erl \ + mochihex.erl \ + mochijson.erl \ + mochijson2.erl \ + mochilists.erl \ + mochilogfile2.erl \ + mochinum.erl \ + mochitemp.erl \ + mochiutf8.erl \ mochiweb.app.in \ - mochiweb.erl \ - mochiweb_app.erl \ - mochiweb_charref.erl \ - mochiweb_cookies.erl \ - mochiweb_echo.erl \ - mochiweb_headers.erl \ - mochiweb_html.erl \ - mochiweb_http.erl \ - mochiweb_multipart.erl \ - mochiweb_request.erl \ - mochiweb_response.erl \ - mochiweb_skel.erl \ - mochiweb_socket_server.erl \ - mochiweb_sup.erl \ - mochiweb_util.erl \ - reloader.erl + mochiweb.erl \ + mochiweb_acceptor.erl \ + mochiweb_app.erl \ + mochiweb_charref.erl \ + mochiweb_cookies.erl \ + mochiweb_cover.erl \ + mochiweb_echo.erl \ + mochiweb_headers.erl \ + mochiweb_html.erl \ + mochiweb_http.erl \ + mochiweb_io.erl \ + mochiweb_mime.erl \ + mochiweb_multipart.erl \ + mochiweb_request.erl \ + mochiweb_response.erl \ + mochiweb_skel.erl \ + mochiweb_socket.erl \ + mochiweb_socket_server.erl \ + mochiweb_sup.erl \ + mochiweb_util.erl \ + reloader.erl mochiwebebin_make_generated_file_list = \ - mochifmt.beam \ - mochifmt_records.beam \ - mochifmt_std.beam \ - mochihex.beam \ - mochijson.beam \ - mochijson2.beam \ - mochinum.beam \ + mochifmt.beam \ + mochifmt_records.beam \ + mochifmt_std.beam \ + mochiglobal.beam \ + mochihex.beam \ + mochijson.beam \ + mochijson2.beam \ + mochilists.beam \ + mochilogfile2.beam \ + mochinum.beam \ + mochitemp.beam \ + mochiutf8.beam \ mochiweb.app \ - mochiweb.beam \ - mochiweb_app.beam \ - mochiweb_charref.beam \ - mochiweb_cookies.beam \ - mochiweb_echo.beam \ - mochiweb_headers.beam \ - mochiweb_html.beam \ - mochiweb_http.beam \ - mochiweb_multipart.beam \ - mochiweb_request.beam \ - mochiweb_response.beam \ - mochiweb_skel.beam \ - mochiweb_socket_server.beam \ - mochiweb_sup.beam \ - mochiweb_util.beam \ - reloader.beam + mochiweb.beam \ + mochiweb_acceptor.beam \ + mochiweb_app.beam \ + mochiweb_charref.beam \ + mochiweb_cookies.beam \ + mochiweb_cover.beam \ + mochiweb_echo.beam \ + mochiweb_headers.beam \ + mochiweb_html.beam \ + mochiweb_http.beam \ + mochiweb_io.beam \ + mochiweb_mime.beam \ + mochiweb_multipart.beam \ + mochiweb_request.beam \ + mochiweb_response.beam \ + mochiweb_skel.beam \ + mochiweb_socket.beam \ + mochiweb_socket_server.beam \ + mochiweb_sup.beam \ + mochiweb_util.beam \ + reloader.beam mochiwebebin_DATA = \ $(mochiwebebin_make_generated_file_list) @@ -77,4 +97,5 @@ CLEANFILES = \ cp $< $@ %.beam: %.erl + $(ERLC) $(ERLC_FLAGS) $< Added: couchdb/trunk/src/mochiweb/internal.hrl URL: http://svn.apache.org/viewvc/couchdb/trunk/src/mochiweb/internal.hrl?rev=979368&view=auto ============================================================================== --- couchdb/trunk/src/mochiweb/internal.hrl (added) +++ couchdb/trunk/src/mochiweb/internal.hrl Mon Jul 26 17:21:30 2010 @@ -0,0 +1,3 @@ + +-define(RECBUF_SIZE, 8192). + Modified: couchdb/trunk/src/mochiweb/mochifmt.erl URL: http://svn.apache.org/viewvc/couchdb/trunk/src/mochiweb/mochifmt.erl?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/src/mochiweb/mochifmt.erl (original) +++ couchdb/trunk/src/mochiweb/mochifmt.erl Mon Jul 26 17:21:30 2010 @@ -10,7 +10,6 @@ -export([tokenize/1, format/3, get_field/3, format_field/3]). -export([bformat/2, bformat/3]). -export([f/2, f/3]). --export([test/0]). -record(conversion, {length, precision, ctype, align, fill_char, sign}). @@ -113,15 +112,6 @@ bformat(Format, Args) -> bformat(Format, Args, Module) -> iolist_to_binary(format(Format, Args, Module)). -%% @spec test() -> ok -%% @doc Run tests. -test() -> - ok = test_tokenize(), - ok = test_format(), - ok = test_std(), - ok = test_records(), - ok. - %% Internal API add_raw("", Acc) -> @@ -375,14 +365,21 @@ parse_std_conversion([$. | Spec], Acc) - parse_std_conversion([Type], Acc) -> parse_std_conversion("", Acc#conversion{ctype=ctype(Type)}). -test_tokenize() -> + +%% +%% Tests +%% +-include_lib("eunit/include/eunit.hrl"). +-ifdef(TEST). + +tokenize_test() -> {?MODULE, [{raw, "ABC"}]} = tokenize("ABC"), {?MODULE, [{format, {"0", "", ""}}]} = tokenize("{0}"), {?MODULE, [{raw, "ABC"}, {format, {"1", "", ""}}, {raw, "DEF"}]} = tokenize("ABC{1}DEF"), ok. -test_format() -> +format_test() -> <<" -4">> = bformat("{0:4}", [-4]), <<" 4">> = bformat("{0:4}", [4]), <<" 4">> = bformat("{0:{0}}", [4]), @@ -410,12 +407,12 @@ test_format() -> {{2008,5,4}, {4, 2, 2}}), ok. -test_std() -> +std_test() -> M = mochifmt_std:new(), <<"01">> = bformat("{0}{1}", [0, 1], M), ok. -test_records() -> +records_test() -> M = mochifmt_records:new([{conversion, record_info(fields, conversion)}]), R = #conversion{length=long, precision=hard, sign=peace}, long = M:get_value("length", R), @@ -424,3 +421,5 @@ test_records() -> <<"long hard">> = bformat("{length} {precision}", R, M), <<"long hard">> = bformat("{0.length} {0.precision}", [R], M), ok. + +-endif. Modified: couchdb/trunk/src/mochiweb/mochifmt_records.erl URL: http://svn.apache.org/viewvc/couchdb/trunk/src/mochiweb/mochifmt_records.erl?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/src/mochiweb/mochifmt_records.erl (original) +++ couchdb/trunk/src/mochiweb/mochifmt_records.erl Mon Jul 26 17:21:30 2010 @@ -28,3 +28,11 @@ get_rec_index(Atom, [Atom | _], Index) - Index; get_rec_index(Atom, [_ | Rest], Index) -> get_rec_index(Atom, Rest, 1 + Index). + + +%% +%% Tests +%% +-include_lib("eunit/include/eunit.hrl"). +-ifdef(TEST). +-endif. Modified: couchdb/trunk/src/mochiweb/mochifmt_std.erl URL: http://svn.apache.org/viewvc/couchdb/trunk/src/mochiweb/mochifmt_std.erl?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/src/mochiweb/mochifmt_std.erl (original) +++ couchdb/trunk/src/mochiweb/mochifmt_std.erl Mon Jul 26 17:21:30 2010 @@ -21,3 +21,10 @@ get_value(Key, Args) -> format_field(Arg, Format) -> mochifmt:format_field(Arg, Format, THIS). + +%% +%% Tests +%% +-include_lib("eunit/include/eunit.hrl"). +-ifdef(TEST). +-endif. Added: couchdb/trunk/src/mochiweb/mochiglobal.erl URL: http://svn.apache.org/viewvc/couchdb/trunk/src/mochiweb/mochiglobal.erl?rev=979368&view=auto ============================================================================== --- couchdb/trunk/src/mochiweb/mochiglobal.erl (added) +++ couchdb/trunk/src/mochiweb/mochiglobal.erl Mon Jul 26 17:21:30 2010 @@ -0,0 +1,107 @@ +%% @author Bob Ippolito +%% @copyright 2010 Mochi Media, Inc. +%% @doc Abuse module constant pools as a "read-only shared heap" (since erts 5.6) +%% [1]. +-module(mochiglobal). +-author("Bob Ippolito "). +-export([get/1, get/2, put/2, delete/1]). + +-spec get(atom()) -> any() | undefined. +%% @equiv get(K, undefined) +get(K) -> + get(K, undefined). + +-spec get(atom(), T) -> any() | T. +%% @doc Get the term for K or return Default. +get(K, Default) -> + get(K, Default, key_to_module(K)). + +get(_K, Default, Mod) -> + try Mod:term() + catch error:undef -> + Default + end. + +-spec put(atom(), any()) -> ok. +%% @doc Store term V at K, replaces an existing term if present. +put(K, V) -> + put(K, V, key_to_module(K)). + +put(_K, V, Mod) -> + Bin = compile(Mod, V), + code:purge(Mod), + code:load_binary(Mod, atom_to_list(Mod) ++ ".erl", Bin), + ok. + +-spec delete(atom()) -> boolean(). +%% @doc Delete term stored at K, no-op if non-existent. +delete(K) -> + delete(K, key_to_module(K)). + +delete(_K, Mod) -> + code:purge(Mod), + code:delete(Mod). + +-spec key_to_module(atom()) -> atom(). +key_to_module(K) -> + list_to_atom("mochiglobal:" ++ atom_to_list(K)). + +-spec compile(atom(), any()) -> binary(). +compile(Module, T) -> + {ok, Module, Bin} = compile:forms(forms(Module, T), + [verbose, report_errors]), + Bin. + +-spec forms(atom(), any()) -> [erl_syntax:syntaxTree()]. +forms(Module, T) -> + [erl_syntax:revert(X) || X <- term_to_abstract(Module, term, T)]. + +-spec term_to_abstract(atom(), atom(), any()) -> [erl_syntax:syntaxTree()]. +term_to_abstract(Module, Getter, T) -> + [%% -module(Module). + erl_syntax:attribute( + erl_syntax:atom(module), + [erl_syntax:atom(Module)]), + %% -export([Getter/0]). + erl_syntax:attribute( + erl_syntax:atom(export), + [erl_syntax:list( + [erl_syntax:arity_qualifier( + erl_syntax:atom(Getter), + erl_syntax:integer(0))])]), + %% Getter() -> T. + erl_syntax:function( + erl_syntax:atom(Getter), + [erl_syntax:clause([], none, [erl_syntax:abstract(T)])])]. + +%% +%% Tests +%% +-include_lib("eunit/include/eunit.hrl"). +-ifdef(TEST). +get_put_delete_test() -> + K = '$$test$$mochiglobal', + delete(K), + ?assertEqual( + bar, + get(K, bar)), + try + ?MODULE:put(K, baz), + ?assertEqual( + baz, + get(K, bar)), + ?MODULE:put(K, wibble), + ?assertEqual( + wibble, + ?MODULE:get(K)) + after + delete(K) + end, + ?assertEqual( + bar, + get(K, bar)), + ?assertEqual( + undefined, + ?MODULE:get(K)), + ok. +-endif. Modified: couchdb/trunk/src/mochiweb/mochihex.erl URL: http://svn.apache.org/viewvc/couchdb/trunk/src/mochiweb/mochihex.erl?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/src/mochiweb/mochihex.erl (original) +++ couchdb/trunk/src/mochiweb/mochihex.erl Mon Jul 26 17:21:30 2010 @@ -6,7 +6,7 @@ -module(mochihex). -author('bob@mochimedia.com'). --export([test/0, to_hex/1, to_bin/1, to_int/1, dehex/1, hexdigit/1]). +-export([to_hex/1, to_bin/1, to_int/1, dehex/1, hexdigit/1]). %% @type iolist() = [char() | binary() | iolist()] %% @type iodata() = iolist() | binary() @@ -46,16 +46,6 @@ hexdigit(C) when C >= 0, C =< 9 -> hexdigit(C) when C =< 15 -> C + $a - 10. -%% @spec test() -> ok -%% @doc Test this module. -test() -> - "ff000ff1" = to_hex([255, 0, 15, 241]), - <<255, 0, 15, 241>> = to_bin("ff000ff1"), - 16#ff000ff1 = to_int("ff000ff1"), - "ff000ff1" = to_hex(16#ff000ff1), - ok. - - %% Internal API to_hex(<<>>, Acc) -> @@ -73,3 +63,29 @@ to_bin([], Acc) -> to_bin([C1, C2 | Rest], Acc) -> to_bin(Rest, [(dehex(C1) bsl 4) bor dehex(C2) | Acc]). + + +%% +%% Tests +%% +-include_lib("eunit/include/eunit.hrl"). +-ifdef(TEST). + +to_hex_test() -> + "ff000ff1" = to_hex([255, 0, 15, 241]), + "ff000ff1" = to_hex(16#ff000ff1), + "0" = to_hex(16#0), + ok. + +to_bin_test() -> + <<255, 0, 15, 241>> = to_bin("ff000ff1"), + <<255, 0, 10, 161>> = to_bin("Ff000aA1"), + ok. + +to_int_test() -> + 16#ff000ff1 = to_int("ff000ff1"), + 16#ff000aa1 = to_int("FF000Aa1"), + 16#0 = to_int("0"), + ok. + +-endif. Modified: couchdb/trunk/src/mochiweb/mochijson.erl URL: http://svn.apache.org/viewvc/couchdb/trunk/src/mochiweb/mochijson.erl?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/src/mochiweb/mochijson.erl (original) +++ couchdb/trunk/src/mochiweb/mochijson.erl Mon Jul 26 17:21:30 2010 @@ -8,7 +8,6 @@ -export([decoder/1, decode/1]). -export([binary_encoder/1, binary_encode/1]). -export([binary_decoder/1, binary_decode/1]). --export([test/0]). % This is a macro to placate syntax highlighters.. -define(Q, $\"). @@ -91,10 +90,6 @@ binary_encode(Any) -> binary_decode(S) -> mochijson2:decode(S). -test() -> - test_all(), - mochijson2:test(). - %% Internal API parse_encoder_options([], State) -> @@ -145,7 +140,7 @@ json_encode_proplist([], _State) -> "{}"; json_encode_proplist(Props, State) -> F = fun ({K, V}, Acc) -> - KS = case K of + KS = case K of K when is_atom(K) -> json_encode_string_utf8(atom_to_list(K)); K when is_integer(K) -> @@ -321,12 +316,12 @@ tokenize_string([$\\, $u, C3, C2, C1, C0 % coalesce UTF-16 surrogate pair? C = dehex(C0) bor (dehex(C1) bsl 4) bor - (dehex(C2) bsl 8) bor + (dehex(C2) bsl 8) bor (dehex(C3) bsl 12), tokenize_string(Rest, ?ADV_COL(S, 6), [C | Acc]); tokenize_string([C | Rest], S, Acc) when C >= $\s; C < 16#10FFFF -> tokenize_string(Rest, ?ADV_COL(S, 1), [C | Acc]). - + tokenize_number(IoList=[C | _], Mode, S=#decoder{input_encoding=utf8}, Acc) when is_list(C); is_binary(C); C >= 16#7f -> List = xmerl_ucs:from_utf8(iolist_to_binary(IoList)), @@ -407,6 +402,13 @@ tokenize(L=[C | _], S) when C >= $0, C = {{const, list_to_float(Float)}, Rest, S1} end. + +%% +%% Tests +%% +-include_lib("eunit/include/eunit.hrl"). +-ifdef(TEST). + %% testing constructs borrowed from the Yaws JSON implementation. %% Create an object from a list of Key/Value pairs. @@ -419,7 +421,7 @@ is_obj({struct, Props}) -> true; (_) -> false - end, + end, lists:all(F, Props). obj_from_list(Props) -> @@ -462,11 +464,10 @@ equiv_list([], []) -> equiv_list([V1 | L1], [V2 | L2]) -> equiv(V1, V2) andalso equiv_list(L1, L2). -test_all() -> - test_issue33(), +e2j_vec_test() -> test_one(e2j_test_vec(utf8), 1). -test_issue33() -> +issue33_test() -> %% http://code.google.com/p/mochiweb/issues/detail?id=33 Js = {struct, [{"key", [194, 163]}]}, Encoder = encoder([{input_encoding, utf8}]), @@ -526,3 +527,5 @@ e2j_test_vec(utf8) -> {{array, [-123, "foo", obj_from_list([{"bar", {array, []}}]), null]}, "[-123,\"foo\",{\"bar\":[]},null]"} ]. + +-endif. Modified: couchdb/trunk/src/mochiweb/mochijson2.erl URL: http://svn.apache.org/viewvc/couchdb/trunk/src/mochiweb/mochijson2.erl?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/src/mochiweb/mochijson2.erl (original) +++ couchdb/trunk/src/mochiweb/mochijson2.erl Mon Jul 26 17:21:30 2010 @@ -9,7 +9,6 @@ -author('bob@mochimedia.com'). -export([encoder/1, encode/1]). -export([decoder/1, decode/1]). --export([test/0]). % This is a macro to placate syntax highlighters.. -define(Q, $\"). @@ -39,8 +38,9 @@ %% @type json_number() = integer() | float() %% @type json_array() = [json_term()] %% @type json_object() = {struct, [{json_string(), json_term()}]} +%% @type json_iolist() = {json, iolist()} %% @type json_term() = json_string() | json_number() | json_array() | -%% json_object() +%% json_object() | json_iolist() -record(encoder, {handler=null, utf8=false}). @@ -75,9 +75,6 @@ decoder(Options) -> decode(S) -> json_decode(S, #decoder{}). -test() -> - test_all(). - %% Internal API parse_encoder_options([], State) -> @@ -108,6 +105,8 @@ json_encode(Array, State) when is_list(A json_encode_array(Array, State); json_encode({struct, Props}, State) when is_list(Props) -> json_encode_proplist(Props, State); +json_encode({json, IoList}, _State) -> + IoList; json_encode(Bad, #encoder{handler=null}) -> exit({json_encode, {bad_term, Bad}}); json_encode(Bad, State=#encoder{handler=Handler}) -> @@ -202,12 +201,10 @@ json_bin_is_safe(<>) -> false; $\t -> false; - C when C >= 0, C < $\s; C >= 16#7f, C =< 16#10FFFF -> + C when C >= 0, C < $\s; C >= 16#7f -> false; C when C < 16#7f -> - json_bin_is_safe(Rest); - _ -> - false + json_bin_is_safe(Rest) end. json_encode_string_unicode([], _State, Acc) -> @@ -507,6 +504,12 @@ tokenize(B, S=#decoder{offset=O}) -> trim = S#decoder.state, {eof, S} end. +%% +%% Tests +%% +-include_lib("eunit/include/eunit.hrl"). +-ifdef(TEST). + %% testing constructs borrowed from the Yaws JSON implementation. @@ -516,19 +519,13 @@ obj_new() -> {struct, []}. is_obj({struct, Props}) -> - F = fun ({K, _}) when is_binary(K) -> - true; - (_) -> - false - end, + F = fun ({K, _}) when is_binary(K) -> true end, lists:all(F, Props). obj_from_list(Props) -> Obj = {struct, Props}, - case is_obj(Obj) of - true -> Obj; - false -> exit({json_bad_object, Obj}) - end. + ?assert(is_obj(Obj)), + Obj. %% Test for equivalence of Erlang terms. %% Due to arbitrary order of construction, equivalent objects might @@ -541,9 +538,7 @@ equiv(L1, L2) when is_list(L1), is_list( equiv_list(L1, L2); equiv(N1, N2) when is_number(N1), is_number(N2) -> N1 == N2; equiv(B1, B2) when is_binary(B1), is_binary(B2) -> B1 == B2; -equiv(true, true) -> true; -equiv(false, false) -> true; -equiv(null, null) -> true. +equiv(A, A) when A =:= true orelse A =:= false orelse A =:= null -> true. %% Object representation and traversal order is unknown. %% Use the sledgehammer and sort property lists. @@ -563,11 +558,11 @@ equiv_list([], []) -> equiv_list([V1 | L1], [V2 | L2]) -> equiv(V1, V2) andalso equiv_list(L1, L2). -test_all() -> +decode_test() -> [1199344435545.0, 1] = decode(<<"[1199344435545.0,1]">>), - <<16#F0,16#9D,16#9C,16#95>> = decode([34,"\\ud835","\\udf15",34]), - test_encoder_utf8(), - test_input_validation(), + <<16#F0,16#9D,16#9C,16#95>> = decode([34,"\\ud835","\\udf15",34]). + +e2j_vec_test() -> test_one(e2j_test_vec(utf8), 1). test_one([], _N) -> @@ -624,7 +619,7 @@ e2j_test_vec(utf8) -> ]. %% test utf8 encoding -test_encoder_utf8() -> +encoder_utf8_test() -> %% safe conversion case (default) [34,"\\u0001","\\u0442","\\u0435","\\u0441","\\u0442",34] = encode(<<1,"\321\202\320\265\321\201\321\202">>), @@ -634,11 +629,11 @@ test_encoder_utf8() -> [34,"\\u0001",[209,130],[208,181],[209,129],[209,130],34] = Enc(<<1,"\321\202\320\265\321\201\321\202">>). -test_input_validation() -> +input_validation_test() -> Good = [ - {16#00A3, <>}, % pound - {16#20AC, <>}, % euro - {16#10196, <>} % denarius + {16#00A3, <>}, %% pound + {16#20AC, <>}, %% euro + {16#10196, <>} %% denarius ], lists:foreach(fun({CodePoint, UTF8}) -> Expect = list_to_binary(xmerl_ucs:to_utf8(CodePoint)), @@ -646,15 +641,146 @@ test_input_validation() -> end, Good), Bad = [ - % 2nd, 3rd, or 4th byte of a multi-byte sequence w/o leading byte + %% 2nd, 3rd, or 4th byte of a multi-byte sequence w/o leading byte <>, - % missing continuations, last byte in each should be 80-BF + %% missing continuations, last byte in each should be 80-BF <>, <>, <>, - % we don't support code points > 10FFFF per RFC 3629 + %% we don't support code points > 10FFFF per RFC 3629 <> ], - lists:foreach(fun(X) -> - ok = try decode(X) catch invalid_utf8 -> ok end - end, Bad). + lists:foreach( + fun(X) -> + ok = try decode(X) catch invalid_utf8 -> ok end, + %% could be {ucs,{bad_utf8_character_code}} or + %% {json_encode,{bad_char,_}} + {'EXIT', _} = (catch encode(X)) + end, Bad). + +inline_json_test() -> + ?assertEqual(<<"\"iodata iodata\"">>, + iolist_to_binary( + encode({json, [<<"\"iodata">>, " iodata\""]}))), + ?assertEqual({struct, [{<<"key">>, <<"iodata iodata">>}]}, + decode( + encode({struct, + [{key, {json, [<<"\"iodata">>, " iodata\""]}}]}))), + ok. + +big_unicode_test() -> + UTF8Seq = list_to_binary(xmerl_ucs:to_utf8(16#0001d120)), + ?assertEqual( + <<"\"\\ud834\\udd20\"">>, + iolist_to_binary(encode(UTF8Seq))), + ?assertEqual( + UTF8Seq, + decode(iolist_to_binary(encode(UTF8Seq)))), + ok. + +custom_decoder_test() -> + ?assertEqual( + {struct, [{<<"key">>, <<"value">>}]}, + (decoder([]))("{\"key\": \"value\"}")), + F = fun ({struct, [{<<"key">>, <<"value">>}]}) -> win end, + ?assertEqual( + win, + (decoder([{object_hook, F}]))("{\"key\": \"value\"}")), + ok. + +atom_test() -> + %% JSON native atoms + [begin + ?assertEqual(A, decode(atom_to_list(A))), + ?assertEqual(iolist_to_binary(atom_to_list(A)), + iolist_to_binary(encode(A))) + end || A <- [true, false, null]], + %% Atom to string + ?assertEqual( + <<"\"foo\"">>, + iolist_to_binary(encode(foo))), + ?assertEqual( + <<"\"\\ud834\\udd20\"">>, + iolist_to_binary(encode(list_to_atom(xmerl_ucs:to_utf8(16#0001d120))))), + ok. + +key_encode_test() -> + %% Some forms are accepted as keys that would not be strings in other + %% cases + ?assertEqual( + <<"{\"foo\":1}">>, + iolist_to_binary(encode({struct, [{foo, 1}]}))), + ?assertEqual( + <<"{\"foo\":1}">>, + iolist_to_binary(encode({struct, [{<<"foo">>, 1}]}))), + ?assertEqual( + <<"{\"foo\":1}">>, + iolist_to_binary(encode({struct, [{"foo", 1}]}))), + ?assertEqual( + <<"{\"\\ud834\\udd20\":1}">>, + iolist_to_binary( + encode({struct, [{[16#0001d120], 1}]}))), + ?assertEqual( + <<"{\"1\":1}">>, + iolist_to_binary(encode({struct, [{1, 1}]}))), + ok. + +unsafe_chars_test() -> + Chars = "\"\\\b\f\n\r\t", + [begin + ?assertEqual(false, json_string_is_safe([C])), + ?assertEqual(false, json_bin_is_safe(<>)), + ?assertEqual(<>, decode(encode(<>))) + end || C <- Chars], + ?assertEqual( + false, + json_string_is_safe([16#0001d120])), + ?assertEqual( + false, + json_bin_is_safe(list_to_binary(xmerl_ucs:to_utf8(16#0001d120)))), + ?assertEqual( + [16#0001d120], + xmerl_ucs:from_utf8( + binary_to_list( + decode(encode(list_to_atom(xmerl_ucs:to_utf8(16#0001d120))))))), + ?assertEqual( + false, + json_string_is_safe([16#110000])), + ?assertEqual( + false, + json_bin_is_safe(list_to_binary(xmerl_ucs:to_utf8([16#110000])))), + %% solidus can be escaped but isn't unsafe by default + ?assertEqual( + <<"/">>, + decode(<<"\"\\/\"">>)), + ok. + +int_test() -> + ?assertEqual(0, decode("0")), + ?assertEqual(1, decode("1")), + ?assertEqual(11, decode("11")), + ok. + +large_int_test() -> + ?assertEqual(<<"-2147483649214748364921474836492147483649">>, + iolist_to_binary(encode(-2147483649214748364921474836492147483649))), + ?assertEqual(<<"2147483649214748364921474836492147483649">>, + iolist_to_binary(encode(2147483649214748364921474836492147483649))), + ok. + +float_test() -> + ?assertEqual(<<"-2147483649.0">>, iolist_to_binary(encode(-2147483649.0))), + ?assertEqual(<<"2147483648.0">>, iolist_to_binary(encode(2147483648.0))), + ok. + +handler_test() -> + ?assertEqual( + {'EXIT',{json_encode,{bad_term,{}}}}, + catch encode({})), + F = fun ({}) -> [] end, + ?assertEqual( + <<"[]">>, + iolist_to_binary((encoder([{handler, F}]))({}))), + ok. + +-endif. Added: couchdb/trunk/src/mochiweb/mochilists.erl URL: http://svn.apache.org/viewvc/couchdb/trunk/src/mochiweb/mochilists.erl?rev=979368&view=auto ============================================================================== --- couchdb/trunk/src/mochiweb/mochilists.erl (added) +++ couchdb/trunk/src/mochiweb/mochilists.erl Mon Jul 26 17:21:30 2010 @@ -0,0 +1,104 @@ +%% @copyright Copyright (c) 2010 Mochi Media, Inc. +%% @author David Reid + +%% @doc Utility functions for dealing with proplists. + +-module(mochilists). +-author("David Reid "). +-export([get_value/2, get_value/3, is_defined/2, set_default/2, set_defaults/2]). + +%% @spec set_default({Key::term(), Value::term()}, Proplist::list()) -> list() +%% +%% @doc Return new Proplist with {Key, Value} set if not is_defined(Key, Proplist). +set_default({Key, Value}, Proplist) -> + case is_defined(Key, Proplist) of + true -> + Proplist; + false -> + [{Key, Value} | Proplist] + end. + +%% @spec set_defaults([{Key::term(), Value::term()}], Proplist::list()) -> list() +%% +%% @doc Return new Proplist with {Key, Value} set if not is_defined(Key, Proplist). +set_defaults(DefaultProps, Proplist) -> + lists:foldl(fun set_default/2, Proplist, DefaultProps). + + +%% @spec is_defined(Key::term(), Proplist::list()) -> bool() +%% +%% @doc Returns true if Propist contains at least one entry associated +%% with Key, otherwise false is returned. +is_defined(Key, Proplist) -> + lists:keyfind(Key, 1, Proplist) =/= false. + + +%% @spec get_value(Key::term(), Proplist::list()) -> term() | undefined +%% +%% @doc Return the value of Key or undefined +get_value(Key, Proplist) -> + get_value(Key, Proplist, undefined). + +%% @spec get_value(Key::term(), Proplist::list(), Default::term()) -> term() +%% +%% @doc Return the value of Key or Default +get_value(_Key, [], Default) -> + Default; +get_value(Key, Proplist, Default) -> + case lists:keyfind(Key, 1, Proplist) of + false -> + Default; + {Key, Value} -> + Value + end. + +%% +%% Tests +%% +-include_lib("eunit/include/eunit.hrl"). +-ifdef(TEST). + +set_defaults_test() -> + ?assertEqual( + [{k, v}], + set_defaults([{k, v}], [])), + ?assertEqual( + [{k, v}], + set_defaults([{k, vee}], [{k, v}])), + ?assertEqual( + lists:sort([{kay, vee}, {k, v}]), + lists:sort(set_defaults([{k, vee}, {kay, vee}], [{k, v}]))), + ok. + +set_default_test() -> + ?assertEqual( + [{k, v}], + set_default({k, v}, [])), + ?assertEqual( + [{k, v}], + set_default({k, vee}, [{k, v}])), + ok. + +get_value_test() -> + ?assertEqual( + undefined, + get_value(foo, [])), + ?assertEqual( + undefined, + get_value(foo, [{bar, baz}])), + ?assertEqual( + bar, + get_value(foo, [{foo, bar}])), + ?assertEqual( + default, + get_value(foo, [], default)), + ?assertEqual( + default, + get_value(foo, [{bar, baz}], default)), + ?assertEqual( + bar, + get_value(foo, [{foo, bar}], default)), + ok. + +-endif. + Added: couchdb/trunk/src/mochiweb/mochilogfile2.erl URL: http://svn.apache.org/viewvc/couchdb/trunk/src/mochiweb/mochilogfile2.erl?rev=979368&view=auto ============================================================================== --- couchdb/trunk/src/mochiweb/mochilogfile2.erl (added) +++ couchdb/trunk/src/mochiweb/mochilogfile2.erl Mon Jul 26 17:21:30 2010 @@ -0,0 +1,140 @@ +%% @author Bob Ippolito +%% @copyright 2010 Mochi Media, Inc. + +%% @doc Write newline delimited log files, ensuring that if a truncated +%% entry is found on log open then it is fixed before writing. Uses +%% delayed writes and raw files for performance. +-module(mochilogfile2). +-author('bob@mochimedia.com'). + +-export([open/1, write/2, close/1, name/1]). + +%% @spec open(Name) -> Handle +%% @doc Open the log file Name, creating or appending as necessary. All data +%% at the end of the file will be truncated until a newline is found, to +%% ensure that all records are complete. +open(Name) -> + {ok, FD} = file:open(Name, [raw, read, write, delayed_write, binary]), + fix_log(FD), + {?MODULE, Name, FD}. + +%% @spec name(Handle) -> string() +%% @doc Return the path of the log file. +name({?MODULE, Name, _FD}) -> + Name. + +%% @spec write(Handle, IoData) -> ok +%% @doc Write IoData to the log file referenced by Handle. +write({?MODULE, _Name, FD}, IoData) -> + ok = file:write(FD, [IoData, $\n]), + ok. + +%% @spec close(Handle) -> ok +%% @doc Close the log file referenced by Handle. +close({?MODULE, _Name, FD}) -> + ok = file:sync(FD), + ok = file:close(FD), + ok. + +fix_log(FD) -> + {ok, Location} = file:position(FD, eof), + Seek = find_last_newline(FD, Location), + {ok, Seek} = file:position(FD, Seek), + ok = file:truncate(FD), + ok. + +%% Seek backwards to the last valid log entry +find_last_newline(_FD, N) when N =< 1 -> + 0; +find_last_newline(FD, Location) -> + case file:pread(FD, Location - 1, 1) of + {ok, <<$\n>>} -> + Location; + {ok, _} -> + find_last_newline(FD, Location - 1) + end. + +%% +%% Tests +%% +-include_lib("eunit/include/eunit.hrl"). +-ifdef(TEST). +name_test() -> + D = mochitemp:mkdtemp(), + FileName = filename:join(D, "open_close_test.log"), + H = open(FileName), + ?assertEqual( + FileName, + name(H)), + close(H), + file:delete(FileName), + file:del_dir(D), + ok. + +open_close_test() -> + D = mochitemp:mkdtemp(), + FileName = filename:join(D, "open_close_test.log"), + OpenClose = fun () -> + H = open(FileName), + ?assertEqual( + true, + filelib:is_file(FileName)), + ok = close(H), + ?assertEqual( + {ok, <<>>}, + file:read_file(FileName)), + ok + end, + OpenClose(), + OpenClose(), + file:delete(FileName), + file:del_dir(D), + ok. + +write_test() -> + D = mochitemp:mkdtemp(), + FileName = filename:join(D, "write_test.log"), + F = fun () -> + H = open(FileName), + write(H, "test line"), + close(H), + ok + end, + F(), + ?assertEqual( + {ok, <<"test line\n">>}, + file:read_file(FileName)), + F(), + ?assertEqual( + {ok, <<"test line\ntest line\n">>}, + file:read_file(FileName)), + file:delete(FileName), + file:del_dir(D), + ok. + +fix_log_test() -> + D = mochitemp:mkdtemp(), + FileName = filename:join(D, "write_test.log"), + file:write_file(FileName, <<"first line good\nsecond line bad">>), + F = fun () -> + H = open(FileName), + write(H, "test line"), + close(H), + ok + end, + F(), + ?assertEqual( + {ok, <<"first line good\ntest line\n">>}, + file:read_file(FileName)), + file:write_file(FileName, <<"first line bad">>), + F(), + ?assertEqual( + {ok, <<"test line\n">>}, + file:read_file(FileName)), + F(), + ?assertEqual( + {ok, <<"test line\ntest line\n">>}, + file:read_file(FileName)), + ok. + +-endif. Modified: couchdb/trunk/src/mochiweb/mochinum.erl URL: http://svn.apache.org/viewvc/couchdb/trunk/src/mochiweb/mochinum.erl?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/src/mochiweb/mochinum.erl (original) +++ couchdb/trunk/src/mochiweb/mochinum.erl Mon Jul 26 17:21:30 2010 @@ -11,7 +11,7 @@ -module(mochinum). -author("Bob Ippolito "). --export([digits/1, frexp/1, int_pow/2, int_ceil/1, test/0]). +-export([digits/1, frexp/1, int_pow/2, int_ceil/1]). %% IEEE 754 Float exponent bias -define(FLOAT_BIAS, 1022). @@ -120,7 +120,7 @@ digits1(Float, Exp, Frac) -> case Exp >= 0 of true -> BExp = 1 bsl Exp, - case (Frac /= ?BIG_POW) of + case (Frac =/= ?BIG_POW) of true -> scale((Frac * BExp * 2), 2, BExp, BExp, Round, Round, Float); @@ -129,7 +129,7 @@ digits1(Float, Exp, Frac) -> Round, Round, Float) end; false -> - case (Exp == ?MIN_EXP) orelse (Frac /= ?BIG_POW) of + case (Exp =:= ?MIN_EXP) orelse (Frac =/= ?BIG_POW) of true -> scale((Frac * 2), 1 bsl (1 - Exp), 1, 1, Round, Round, Float); @@ -228,14 +228,13 @@ log2floor(Int, N) -> log2floor(Int bsr 1, 1 + N). -test() -> - ok = test_frexp(), - ok = test_int_ceil(), - ok = test_int_pow(), - ok = test_digits(), - ok. +%% +%% Tests +%% +-include_lib("eunit/include/eunit.hrl"). +-ifdef(TEST). -test_int_ceil() -> +int_ceil_test() -> 1 = int_ceil(0.0001), 0 = int_ceil(0.0), 1 = int_ceil(0.99), @@ -244,7 +243,7 @@ test_int_ceil() -> -2 = int_ceil(-2.0), ok. -test_int_pow() -> +int_pow_test() -> 1 = int_pow(1, 1), 1 = int_pow(1, 0), 1 = int_pow(10, 0), @@ -253,17 +252,58 @@ test_int_pow() -> 1000 = int_pow(10, 3), ok. -test_digits() -> - "0" = digits(0), - "0.0" = digits(0.0), - "1.0" = digits(1.0), - "-1.0" = digits(-1.0), - "0.1" = digits(0.1), - "0.01" = digits(0.01), - "0.001" = digits(0.001), +digits_test() -> + ?assertEqual("0", + digits(0)), + ?assertEqual("0.0", + digits(0.0)), + ?assertEqual("1.0", + digits(1.0)), + ?assertEqual("-1.0", + digits(-1.0)), + ?assertEqual("0.1", + digits(0.1)), + ?assertEqual("0.01", + digits(0.01)), + ?assertEqual("0.001", + digits(0.001)), + ?assertEqual("1.0e+6", + digits(1000000.0)), + ?assertEqual("0.5", + digits(0.5)), + ?assertEqual("4503599627370496.0", + digits(4503599627370496.0)), + %% small denormalized number + %% 4.94065645841246544177e-324 + <> = <<0,0,0,0,0,0,0,1>>, + ?assertEqual("4.9406564584124654e-324", + digits(SmallDenorm)), + ?assertEqual(SmallDenorm, + list_to_float(digits(SmallDenorm))), + %% large denormalized number + %% 2.22507385850720088902e-308 + <> = <<0,15,255,255,255,255,255,255>>, + ?assertEqual("2.225073858507201e-308", + digits(BigDenorm)), + ?assertEqual(BigDenorm, + list_to_float(digits(BigDenorm))), + %% small normalized number + %% 2.22507385850720138309e-308 + <> = <<0,16,0,0,0,0,0,0>>, + ?assertEqual("2.2250738585072014e-308", + digits(SmallNorm)), + ?assertEqual(SmallNorm, + list_to_float(digits(SmallNorm))), + %% large normalized number + %% 1.79769313486231570815e+308 + <> = <<127,239,255,255,255,255,255,255>>, + ?assertEqual("1.7976931348623157e+308", + digits(LargeNorm)), + ?assertEqual(LargeNorm, + list_to_float(digits(LargeNorm))), ok. -test_frexp() -> +frexp_test() -> %% zero {0.0, 0} = frexp(0.0), %% one @@ -287,3 +327,5 @@ test_frexp() -> <> = <<127,239,255,255,255,255,255,255>>, {0.99999999999999989, 1024} = frexp(LargeNorm), ok. + +-endif. Added: couchdb/trunk/src/mochiweb/mochitemp.erl URL: http://svn.apache.org/viewvc/couchdb/trunk/src/mochiweb/mochitemp.erl?rev=979368&view=auto ============================================================================== --- couchdb/trunk/src/mochiweb/mochitemp.erl (added) +++ couchdb/trunk/src/mochiweb/mochitemp.erl Mon Jul 26 17:21:30 2010 @@ -0,0 +1,310 @@ +%% @author Bob Ippolito +%% @copyright 2010 Mochi Media, Inc. + +%% @doc Create temporary files and directories. Requires crypto to be started. + +-module(mochitemp). +-export([gettempdir/0]). +-export([mkdtemp/0, mkdtemp/3]). +-export([rmtempdir/1]). +%% -export([mkstemp/4]). +-define(SAFE_CHARS, {$a, $b, $c, $d, $e, $f, $g, $h, $i, $j, $k, $l, $m, + $n, $o, $p, $q, $r, $s, $t, $u, $v, $w, $x, $y, $z, + $A, $B, $C, $D, $E, $F, $G, $H, $I, $J, $K, $L, $M, + $N, $O, $P, $Q, $R, $S, $T, $U, $V, $W, $X, $Y, $Z, + $0, $1, $2, $3, $4, $5, $6, $7, $8, $9, $_}). +-define(TMP_MAX, 10000). + +-include_lib("kernel/include/file.hrl"). + +%% TODO: An ugly wrapper over the mktemp tool with open_port and sadness? +%% We can't implement this race-free in Erlang without the ability +%% to issue O_CREAT|O_EXCL. I suppose we could hack something with +%% mkdtemp, del_dir, open. +%% mkstemp(Suffix, Prefix, Dir, Options) -> +%% ok. + +rmtempdir(Dir) -> + case file:del_dir(Dir) of + {error, eexist} -> + ok = rmtempdirfiles(Dir), + ok = file:del_dir(Dir); + ok -> + ok + end. + +rmtempdirfiles(Dir) -> + {ok, Files} = file:list_dir(Dir), + ok = rmtempdirfiles(Dir, Files). + +rmtempdirfiles(_Dir, []) -> + ok; +rmtempdirfiles(Dir, [Basename | Rest]) -> + Path = filename:join([Dir, Basename]), + case filelib:is_dir(Path) of + true -> + ok = rmtempdir(Path); + false -> + ok = file:delete(Path) + end, + rmtempdirfiles(Dir, Rest). + +mkdtemp() -> + mkdtemp("", "tmp", gettempdir()). + +mkdtemp(Suffix, Prefix, Dir) -> + mkdtemp_n(rngpath_fun(Suffix, Prefix, Dir), ?TMP_MAX). + + + +mkdtemp_n(RngPath, 1) -> + make_dir(RngPath()); +mkdtemp_n(RngPath, N) -> + try make_dir(RngPath()) + catch throw:{error, eexist} -> + mkdtemp_n(RngPath, N - 1) + end. + +make_dir(Path) -> + case file:make_dir(Path) of + ok -> + ok; + E={error, eexist} -> + throw(E) + end, + %% Small window for a race condition here because dir is created 777 + ok = file:write_file_info(Path, #file_info{mode=8#0700}), + Path. + +rngpath_fun(Prefix, Suffix, Dir) -> + fun () -> + filename:join([Dir, Prefix ++ rngchars(6) ++ Suffix]) + end. + +rngchars(0) -> + ""; +rngchars(N) -> + [rngchar() | rngchars(N - 1)]. + +rngchar() -> + rngchar(crypto:rand_uniform(0, tuple_size(?SAFE_CHARS))). + +rngchar(C) -> + element(1 + C, ?SAFE_CHARS). + +%% @spec gettempdir() -> string() +%% @doc Get a usable temporary directory using the first of these that is a directory: +%% $TMPDIR, $TMP, $TEMP, "/tmp", "/var/tmp", "/usr/tmp", ".". +gettempdir() -> + gettempdir(gettempdir_checks(), fun normalize_dir/1). + +gettempdir_checks() -> + [{fun os:getenv/1, ["TMPDIR", "TMP", "TEMP"]}, + {fun gettempdir_identity/1, ["/tmp", "/var/tmp", "/usr/tmp"]}, + {fun gettempdir_cwd/1, [cwd]}]. + +gettempdir_identity(L) -> + L. + +gettempdir_cwd(cwd) -> + {ok, L} = file:get_cwd(), + L. + +gettempdir([{_F, []} | RestF], Normalize) -> + gettempdir(RestF, Normalize); +gettempdir([{F, [L | RestL]} | RestF], Normalize) -> + case Normalize(F(L)) of + false -> + gettempdir([{F, RestL} | RestF], Normalize); + Dir -> + Dir + end. + +normalize_dir(False) when False =:= false orelse False =:= "" -> + %% Erlang doesn't have an unsetenv, wtf. + false; +normalize_dir(L) -> + Dir = filename:absname(L), + case filelib:is_dir(Dir) of + false -> + false; + true -> + Dir + end. + +%% +%% Tests +%% +-include_lib("eunit/include/eunit.hrl"). +-ifdef(TEST). +pushenv(L) -> + [{K, os:getenv(K)} || K <- L]. +popenv(L) -> + F = fun ({K, false}) -> + %% Erlang doesn't have an unsetenv, wtf. + os:putenv(K, ""); + ({K, V}) -> + os:putenv(K, V) + end, + lists:foreach(F, L). + +gettempdir_fallback_test() -> + ?assertEqual( + "/", + gettempdir([{fun gettempdir_identity/1, ["/--not-here--/"]}, + {fun gettempdir_identity/1, ["/"]}], + fun normalize_dir/1)), + ?assertEqual( + "/", + %% simulate a true os:getenv unset env + gettempdir([{fun gettempdir_identity/1, [false]}, + {fun gettempdir_identity/1, ["/"]}], + fun normalize_dir/1)), + ok. + +gettempdir_identity_test() -> + ?assertEqual( + "/", + gettempdir([{fun gettempdir_identity/1, ["/"]}], fun normalize_dir/1)), + ok. + +gettempdir_cwd_test() -> + {ok, Cwd} = file:get_cwd(), + ?assertEqual( + normalize_dir(Cwd), + gettempdir([{fun gettempdir_cwd/1, [cwd]}], fun normalize_dir/1)), + ok. + +rngchars_test() -> + crypto:start(), + ?assertEqual( + "", + rngchars(0)), + ?assertEqual( + 10, + length(rngchars(10))), + ok. + +rngchar_test() -> + ?assertEqual( + $a, + rngchar(0)), + ?assertEqual( + $A, + rngchar(26)), + ?assertEqual( + $_, + rngchar(62)), + ok. + +mkdtemp_n_failonce_test() -> + crypto:start(), + D = mkdtemp(), + Path = filename:join([D, "testdir"]), + %% Toggle the existence of a dir so that it fails + %% the first time and succeeds the second. + F = fun () -> + case filelib:is_dir(Path) of + true -> + file:del_dir(Path); + false -> + file:make_dir(Path) + end, + Path + end, + try + %% Fails the first time + ?assertThrow( + {error, eexist}, + mkdtemp_n(F, 1)), + %% Reset state + file:del_dir(Path), + %% Succeeds the second time + ?assertEqual( + Path, + mkdtemp_n(F, 2)) + after rmtempdir(D) + end, + ok. + +mkdtemp_n_fail_test() -> + {ok, Cwd} = file:get_cwd(), + ?assertThrow( + {error, eexist}, + mkdtemp_n(fun () -> Cwd end, 1)), + ?assertThrow( + {error, eexist}, + mkdtemp_n(fun () -> Cwd end, 2)), + ok. + +make_dir_fail_test() -> + {ok, Cwd} = file:get_cwd(), + ?assertThrow( + {error, eexist}, + make_dir(Cwd)), + ok. + +mkdtemp_test() -> + crypto:start(), + D = mkdtemp(), + ?assertEqual( + true, + filelib:is_dir(D)), + ?assertEqual( + ok, + file:del_dir(D)), + ok. + +rmtempdir_test() -> + crypto:start(), + D1 = mkdtemp(), + ?assertEqual( + true, + filelib:is_dir(D1)), + ?assertEqual( + ok, + rmtempdir(D1)), + D2 = mkdtemp(), + ?assertEqual( + true, + filelib:is_dir(D2)), + ok = file:write_file(filename:join([D2, "foo"]), <<"bytes">>), + D3 = mkdtemp("suffix", "prefix", D2), + ?assertEqual( + true, + filelib:is_dir(D3)), + ok = file:write_file(filename:join([D3, "foo"]), <<"bytes">>), + ?assertEqual( + ok, + rmtempdir(D2)), + ?assertEqual( + {error, enoent}, + file:consult(D3)), + ?assertEqual( + {error, enoent}, + file:consult(D2)), + ok. + +gettempdir_env_test() -> + Env = pushenv(["TMPDIR", "TEMP", "TMP"]), + FalseEnv = [{"TMPDIR", false}, {"TEMP", false}, {"TMP", false}], + try + popenv(FalseEnv), + popenv([{"TMPDIR", "/"}]), + ?assertEqual( + "/", + os:getenv("TMPDIR")), + ?assertEqual( + "/", + gettempdir()), + {ok, Cwd} = file:get_cwd(), + popenv(FalseEnv), + popenv([{"TMP", Cwd}]), + ?assertEqual( + normalize_dir(Cwd), + gettempdir()) + after popenv(Env) + end, + ok. + +-endif. Added: couchdb/trunk/src/mochiweb/mochiutf8.erl URL: http://svn.apache.org/viewvc/couchdb/trunk/src/mochiweb/mochiutf8.erl?rev=979368&view=auto ============================================================================== --- couchdb/trunk/src/mochiweb/mochiutf8.erl (added) +++ couchdb/trunk/src/mochiweb/mochiutf8.erl Mon Jul 26 17:21:30 2010 @@ -0,0 +1,316 @@ +%% @copyright 2010 Mochi Media, Inc. +%% @author Bob Ippolito + +%% @doc Algorithm to convert any binary to a valid UTF-8 sequence by ignoring +%% invalid bytes. + +-module(mochiutf8). +-export([valid_utf8_bytes/1, codepoint_to_bytes/1, bytes_to_codepoints/1]). +-export([bytes_foldl/3, codepoint_foldl/3, read_codepoint/1, len/1]). + +%% External API + +-type unichar_low() :: 0..16#d7ff. +-type unichar_high() :: 16#e000..16#10ffff. +-type unichar() :: unichar_low() | unichar_high(). + +-spec codepoint_to_bytes(unichar()) -> binary(). +%% @doc Convert a unicode codepoint to UTF-8 bytes. +codepoint_to_bytes(C) when (C >= 16#00 andalso C =< 16#7f) -> + %% U+0000 - U+007F - 7 bits + <>; +codepoint_to_bytes(C) when (C >= 16#080 andalso C =< 16#07FF) -> + %% U+0080 - U+07FF - 11 bits + <<0:5, B1:5, B0:6>> = <>, + <<2#110:3, B1:5, + 2#10:2, B0:6>>; +codepoint_to_bytes(C) when (C >= 16#0800 andalso C =< 16#FFFF) andalso + (C < 16#D800 orelse C > 16#DFFF) -> + %% U+0800 - U+FFFF - 16 bits (excluding UTC-16 surrogate code points) + <> = <>, + <<2#1110:4, B2:4, + 2#10:2, B1:6, + 2#10:2, B0:6>>; +codepoint_to_bytes(C) when (C >= 16#010000 andalso C =< 16#10FFFF) -> + %% U+10000 - U+10FFFF - 21 bits + <<0:3, B3:3, B2:6, B1:6, B0:6>> = <>, + <<2#11110:5, B3:3, + 2#10:2, B2:6, + 2#10:2, B1:6, + 2#10:2, B0:6>>. + +-spec codepoints_to_bytes([unichar()]) -> binary(). +%% @doc Convert a list of codepoints to a UTF-8 binary. +codepoints_to_bytes(L) -> + <<<<(codepoint_to_bytes(C))/binary>> || C <- L>>. + +-spec read_codepoint(binary()) -> {unichar(), binary(), binary()}. +read_codepoint(Bin = <<2#0:1, C:7, Rest/binary>>) -> + %% U+0000 - U+007F - 7 bits + <> = Bin, + {C, B, Rest}; +read_codepoint(Bin = <<2#110:3, B1:5, + 2#10:2, B0:6, + Rest/binary>>) -> + %% U+0080 - U+07FF - 11 bits + case <> of + <> when C >= 16#80 -> + <> = Bin, + {C, B, Rest} + end; +read_codepoint(Bin = <<2#1110:4, B2:4, + 2#10:2, B1:6, + 2#10:2, B0:6, + Rest/binary>>) -> + %% U+0800 - U+FFFF - 16 bits (excluding UTC-16 surrogate code points) + case <> of + <> when (C >= 16#0800 andalso C =< 16#FFFF) andalso + (C < 16#D800 orelse C > 16#DFFF) -> + <> = Bin, + {C, B, Rest} + end; +read_codepoint(Bin = <<2#11110:5, B3:3, + 2#10:2, B2:6, + 2#10:2, B1:6, + 2#10:2, B0:6, + Rest/binary>>) -> + %% U+10000 - U+10FFFF - 21 bits + case <> of + <> when (C >= 16#010000 andalso C =< 16#10FFFF) -> + <> = Bin, + {C, B, Rest} + end. + +-spec codepoint_foldl(fun((unichar(), _) -> _), _, binary()) -> _. +codepoint_foldl(F, Acc, <<>>) when is_function(F, 2) -> + Acc; +codepoint_foldl(F, Acc, Bin) -> + {C, _, Rest} = read_codepoint(Bin), + codepoint_foldl(F, F(C, Acc), Rest). + +-spec bytes_foldl(fun((binary(), _) -> _), _, binary()) -> _. +bytes_foldl(F, Acc, <<>>) when is_function(F, 2) -> + Acc; +bytes_foldl(F, Acc, Bin) -> + {_, B, Rest} = read_codepoint(Bin), + bytes_foldl(F, F(B, Acc), Rest). + +-spec bytes_to_codepoints(binary()) -> [unichar()]. +bytes_to_codepoints(B) -> + lists:reverse(codepoint_foldl(fun (C, Acc) -> [C | Acc] end, [], B)). + +-spec len(binary()) -> non_neg_integer(). +len(<<>>) -> + 0; +len(B) -> + {_, _, Rest} = read_codepoint(B), + 1 + len(Rest). + +-spec valid_utf8_bytes(B::binary()) -> binary(). +%% @doc Return only the bytes in B that represent valid UTF-8. Uses +%% the following recursive algorithm: skip one byte if B does not +%% follow UTF-8 syntax (a 1-4 byte encoding of some number), +%% skip sequence of 2-4 bytes if it represents an overlong encoding +%% or bad code point (surrogate U+D800 - U+DFFF or > U+10FFFF). +valid_utf8_bytes(B) when is_binary(B) -> + binary_skip_bytes(B, invalid_utf8_indexes(B)). + +%% Internal API + +-spec binary_skip_bytes(binary(), [non_neg_integer()]) -> binary(). +%% @doc Return B, but skipping the 0-based indexes in L. +binary_skip_bytes(B, []) -> + B; +binary_skip_bytes(B, L) -> + binary_skip_bytes(B, L, 0, []). + +%% @private +-spec binary_skip_bytes(binary(), [non_neg_integer()], non_neg_integer(), iolist()) -> binary(). +binary_skip_bytes(B, [], _N, Acc) -> + iolist_to_binary(lists:reverse([B | Acc])); +binary_skip_bytes(<<_, RestB/binary>>, [N | RestL], N, Acc) -> + binary_skip_bytes(RestB, RestL, 1 + N, Acc); +binary_skip_bytes(<>, L, N, Acc) -> + binary_skip_bytes(RestB, L, 1 + N, [C | Acc]). + +-spec invalid_utf8_indexes(binary()) -> [non_neg_integer()]. +%% @doc Return the 0-based indexes in B that are not valid UTF-8. +invalid_utf8_indexes(B) -> + invalid_utf8_indexes(B, 0, []). + +%% @private. +-spec invalid_utf8_indexes(binary(), non_neg_integer(), [non_neg_integer()]) -> [non_neg_integer()]. +invalid_utf8_indexes(<>, N, Acc) when C < 16#80 -> + %% U+0000 - U+007F - 7 bits + invalid_utf8_indexes(Rest, 1 + N, Acc); +invalid_utf8_indexes(<>, N, Acc) + when C1 band 16#E0 =:= 16#C0, + C2 band 16#C0 =:= 16#80 -> + %% U+0080 - U+07FF - 11 bits + case ((C1 band 16#1F) bsl 6) bor (C2 band 16#3F) of + C when C < 16#80 -> + %% Overlong encoding. + invalid_utf8_indexes(Rest, 2 + N, [1 + N, N | Acc]); + _ -> + %% Upper bound U+07FF does not need to be checked + invalid_utf8_indexes(Rest, 2 + N, Acc) + end; +invalid_utf8_indexes(<>, N, Acc) + when C1 band 16#F0 =:= 16#E0, + C2 band 16#C0 =:= 16#80, + C3 band 16#C0 =:= 16#80 -> + %% U+0800 - U+FFFF - 16 bits + case ((((C1 band 16#0F) bsl 6) bor (C2 band 16#3F)) bsl 6) bor + (C3 band 16#3F) of + C when (C < 16#800) orelse (C >= 16#D800 andalso C =< 16#DFFF) -> + %% Overlong encoding or surrogate. + invalid_utf8_indexes(Rest, 3 + N, [2 + N, 1 + N, N | Acc]); + _ -> + %% Upper bound U+FFFF does not need to be checked + invalid_utf8_indexes(Rest, 3 + N, Acc) + end; +invalid_utf8_indexes(<>, N, Acc) + when C1 band 16#F8 =:= 16#F0, + C2 band 16#C0 =:= 16#80, + C3 band 16#C0 =:= 16#80, + C4 band 16#C0 =:= 16#80 -> + %% U+10000 - U+10FFFF - 21 bits + case ((((((C1 band 16#0F) bsl 6) bor (C2 band 16#3F)) bsl 6) bor + (C3 band 16#3F)) bsl 6) bor (C4 band 16#3F) of + C when (C < 16#10000) orelse (C > 16#10FFFF) -> + %% Overlong encoding or invalid code point. + invalid_utf8_indexes(Rest, 4 + N, [3 + N, 2 + N, 1 + N, N | Acc]); + _ -> + invalid_utf8_indexes(Rest, 4 + N, Acc) + end; +invalid_utf8_indexes(<<_, Rest/binary>>, N, Acc) -> + %% Invalid char + invalid_utf8_indexes(Rest, 1 + N, [N | Acc]); +invalid_utf8_indexes(<<>>, _N, Acc) -> + lists:reverse(Acc). + +%% +%% Tests +%% +-include_lib("eunit/include/eunit.hrl"). +-ifdef(TEST). + +binary_skip_bytes_test() -> + ?assertEqual(<<"foo">>, + binary_skip_bytes(<<"foo">>, [])), + ?assertEqual(<<"foobar">>, + binary_skip_bytes(<<"foo bar">>, [3])), + ?assertEqual(<<"foo">>, + binary_skip_bytes(<<"foo bar">>, [3, 4, 5, 6])), + ?assertEqual(<<"oo bar">>, + binary_skip_bytes(<<"foo bar">>, [0])), + ok. + +invalid_utf8_indexes_test() -> + ?assertEqual( + [], + invalid_utf8_indexes(<<"unicode snowman for you: ", 226, 152, 131>>)), + ?assertEqual( + [0], + invalid_utf8_indexes(<<128>>)), + ?assertEqual( + [57,59,60,64,66,67], + invalid_utf8_indexes(<<"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; (", + 167, 65, 170, 186, 73, 83, 80, 166, 87, 186, 217, 41, 41>>)), + ok. + +codepoint_to_bytes_test() -> + %% U+0000 - U+007F - 7 bits + %% U+0080 - U+07FF - 11 bits + %% U+0800 - U+FFFF - 16 bits (excluding UTC-16 surrogate code points) + %% U+10000 - U+10FFFF - 21 bits + ?assertEqual( + <<"a">>, + codepoint_to_bytes($a)), + ?assertEqual( + <<16#c2, 16#80>>, + codepoint_to_bytes(16#80)), + ?assertEqual( + <<16#df, 16#bf>>, + codepoint_to_bytes(16#07ff)), + ?assertEqual( + <<16#ef, 16#bf, 16#bf>>, + codepoint_to_bytes(16#ffff)), + ?assertEqual( + <<16#f4, 16#8f, 16#bf, 16#bf>>, + codepoint_to_bytes(16#10ffff)), + ok. + +bytes_foldl_test() -> + ?assertEqual( + <<"abc">>, + bytes_foldl(fun (B, Acc) -> <> end, <<>>, <<"abc">>)), + ?assertEqual( + <<"abc", 226, 152, 131, 228, 184, 173, 194, 133, 244,143,191,191>>, + bytes_foldl(fun (B, Acc) -> <> end, <<>>, + <<"abc", 226, 152, 131, 228, 184, 173, 194, 133, 244,143,191,191>>)), + ok. + +bytes_to_codepoints_test() -> + ?assertEqual( + "abc" ++ [16#2603, 16#4e2d, 16#85, 16#10ffff], + bytes_to_codepoints(<<"abc", 226, 152, 131, 228, 184, 173, 194, 133, 244,143,191,191>>)), + ok. + +codepoint_foldl_test() -> + ?assertEqual( + "cba", + codepoint_foldl(fun (C, Acc) -> [C | Acc] end, [], <<"abc">>)), + ?assertEqual( + [16#10ffff, 16#85, 16#4e2d, 16#2603 | "cba"], + codepoint_foldl(fun (C, Acc) -> [C | Acc] end, [], + <<"abc", 226, 152, 131, 228, 184, 173, 194, 133, 244,143,191,191>>)), + ok. + +len_test() -> + ?assertEqual( + 29, + len(<<"unicode snowman for you: ", 226, 152, 131, 228, 184, 173, 194, 133, 244, 143, 191, 191>>)), + ok. + +codepoints_to_bytes_test() -> + ?assertEqual( + iolist_to_binary(lists:map(fun codepoint_to_bytes/1, lists:seq(1, 1000))), + codepoints_to_bytes(lists:seq(1, 1000))), + ok. + +valid_utf8_bytes_test() -> + ?assertEqual( + <<"invalid U+11ffff: ">>, + valid_utf8_bytes(<<"invalid U+11ffff: ", 244, 159, 191, 191>>)), + ?assertEqual( + <<"U+10ffff: ", 244, 143, 191, 191>>, + valid_utf8_bytes(<<"U+10ffff: ", 244, 143, 191, 191>>)), + ?assertEqual( + <<"overlong 2-byte encoding (a): ">>, + valid_utf8_bytes(<<"overlong 2-byte encoding (a): ", 2#11000001, 2#10100001>>)), + ?assertEqual( + <<"overlong 2-byte encoding (!): ">>, + valid_utf8_bytes(<<"overlong 2-byte encoding (!): ", 2#11000000, 2#10100001>>)), + ?assertEqual( + <<"mu: ", 194, 181>>, + valid_utf8_bytes(<<"mu: ", 194, 181>>)), + ?assertEqual( + <<"bad coding bytes: ">>, + valid_utf8_bytes(<<"bad coding bytes: ", 2#10011111, 2#10111111, 2#11111111>>)), + ?assertEqual( + <<"low surrogate (unpaired): ">>, + valid_utf8_bytes(<<"low surrogate (unpaired): ", 237, 176, 128>>)), + ?assertEqual( + <<"high surrogate (unpaired): ">>, + valid_utf8_bytes(<<"high surrogate (unpaired): ", 237, 191, 191>>)), + ?assertEqual( + <<"unicode snowman for you: ", 226, 152, 131>>, + valid_utf8_bytes(<<"unicode snowman for you: ", 226, 152, 131>>)), + ?assertEqual( + <<"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; (AISPW))">>, + valid_utf8_bytes(<<"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; (", + 167, 65, 170, 186, 73, 83, 80, 166, 87, 186, 217, 41, 41>>)), + ok. + +-endif. Modified: couchdb/trunk/src/mochiweb/mochiweb.app.in URL: http://svn.apache.org/viewvc/couchdb/trunk/src/mochiweb/mochiweb.app.in?rev=979368&r1=979367&r2=979368&view=diff ============================================================================== --- couchdb/trunk/src/mochiweb/mochiweb.app.in (original) +++ couchdb/trunk/src/mochiweb/mochiweb.app.in Mon Jul 26 17:21:30 2010 @@ -1,6 +1,6 @@ {application, mochiweb, [{description, "MochiMedia Web Server"}, - {vsn, "113"}, + {vsn, "7c2bc2"}, {modules, [ mochihex, mochijson, Added: couchdb/trunk/src/mochiweb/mochiweb.app.src URL: http://svn.apache.org/viewvc/couchdb/trunk/src/mochiweb/mochiweb.app.src?rev=979368&view=auto ============================================================================== --- couchdb/trunk/src/mochiweb/mochiweb.app.src (added) +++ couchdb/trunk/src/mochiweb/mochiweb.app.src Mon Jul 26 17:21:30 2010 @@ -0,0 +1,9 @@ +%% This is generated from src/mochiweb.app.src +{application, mochiweb, + [{description, "MochiMedia Web Server"}, + {vsn, "7c2bc2"}, + {modules, []}, + {registered, []}, + {mod, {mochiweb_app, []}}, + {env, []}, + {applications, [kernel, stdlib, crypto, inets]}]}.