Return-Path: X-Original-To: apmail-couchdb-user-archive@www.apache.org Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AF778925A for ; Thu, 31 May 2012 23:52:42 +0000 (UTC) Received: (qmail 59586 invoked by uid 500); 31 May 2012 23:52:41 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 59556 invoked by uid 500); 31 May 2012 23:52:41 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 59547 invoked by uid 99); 31 May 2012 23:52:41 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 31 May 2012 23:52:41 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [128.18.84.133] (HELO brightmail-internal4.sri.com) (128.18.84.133) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 31 May 2012 23:52:30 +0000 X-AuditID: 80125485-b7efc6d000006bab-59-4fc804273e37 Received: from exchange-hub03.SRI.COM (exchange-hub03.SRI.COM [128.18.23.155]) (using TLS with cipher AES128-SHA (AES128-SHA/128 bits)) (Client did not present a certificate) by brightmail-internal4.sri.com (SRI Internal SMTP Gateway) with SMTP id 8D.A6.27563.72408CF4; Thu, 31 May 2012 16:52:07 -0700 (PDT) Received: from EXCHANGE-DB08.SRI.COM ([fe80::a11e:7c21:6886:9a20]) by exchange-hub03.SRI.COM ([fe80::8c0e:cf22:fef8:cb20%15]) with mapi id 14.02.0298.004; Thu, 31 May 2012 16:51:59 -0700 From: Jim Klo To: "" , "" Subject: Re: View access during indexing Thread-Topic: View access during indexing Thread-Index: AQHNP4bU3IuB+IIzYEaGoxAUhzX2ZZblBw8A Date: Thu, 31 May 2012 23:51:58 +0000 Message-ID: <630350F9-8969-4FE9-AEED-A067BAC71C68@sri.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [192.12.16.206] Content-Type: multipart/signed; boundary="Apple-Mail-47--576125974"; protocol="application/pkcs7-signature"; micalg=sha1 MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrNIsWRmVeSWpSXmKPExsXSICQ+W1ed5YS/wcyLuhaf585ltOjcs5fN gclj44fjjB47Z91lD2CK4rJJSc3JLEst0rdL4Mo49q6RseBUaMX39rtMDYybfLsYOTkkBEwk ljWdZIKwxSQu3FvP1sXIxSEksJNJ4sbDF6wQzj5GiWUPjzGDVLEJyEsc3v4AzBYRKJDomfGN DcQWFtCQuLNiHxNEXFOi4dw7FgjbSGLptgNg9SwCqhJfDx5mBbF5Bawkzk1+CGYLCQRIHG6e C2ZzCgRKbHt0CsxmBLro+6k1YDOZBcQlbj2ZD3WpiMTDi6fZIGxRiZeP/7FC2IoSn2c8YIOo r5K4/WUHI8QuQYmTM5+wTGAUmYVk1CwkZbOQlEHEPSTm/FvMCGFrSyxb+Jp5FiMHkK0jMXkh mjCE/fH8EaiRphKvj36EqrGWmPHrIBuErSgxpfsh+wJG7lWMMklFmekZJbmJmTm6sIg10Ssu ytRLzs/dxAiO4pDWHYwr9hgeYhTgYFTi4TWaddxfiDWxrLgy9xCjBAezkggvwxugEG9KYmVV alF+fFFpTmrxIUZpDhYlcd5v5vz+QgLpiSWp2ampBalFMFkmDk6pBsZjnAknFRmfvv/TLnR3 +sW3FmtuqlpJfHNUsTyWfTFj98VLvpvORi5MqEq0uqyqlD+37pXQ3Ef8NuwqB2xaOzl4jz9w /3Ez9L7uov+M59f9uPfEP9n9kIKGbXuB/pSwP84pgm++B4dEOjB6Hi+O89mfGXcj/M7Uub9m nHyY1njZOzsiQVZE45sSS3FGoqEWc1FxIgDmkwJN3gIAAA== --Apple-Mail-47--576125974 Content-Type: multipart/alternative; boundary=Apple-Mail-46--576126011 --Apple-Mail-46--576126011 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii It should be limited to views within a single design doc. There's some = storage efficiency I think for placing all your views into a single = design doc, but all views in a design doc get rebuilt when you modify = any of the views. If your views aren't dependent upon each other, and you aren't concerned = about disk utilization, you could create each view in a separate design = doc to bypass the rebuild. Other 'hackery' that I understand is = possible is to push the modified view to a copy of the design doc with a = different _id and begin the index process (query the last document in = the view). Then once the indexing is complete, update the original = design doc to match the alternate and CouchDb should assume using the = view that was 'backgrounded'. This will keep all views accessible until = you swap in the changes after the modified view is indexed. - Jim Jim Klo Senior Software Engineer Center for Software Engineering SRI International On May 31, 2012, at 4:40 PM, Matthias Eck wrote: > Hello, >=20 > One of my databases in CouchDB has close to 1,000,000 documents with a > size of about 200GB. > If I change one of the views, the indexing takes about 2 hours. >=20 > This time would be not a problem, but I am wondering why I cannot > access the other views, that were previously generated during the > indexing time. > Currently none of the views is available, taking my web application > down during this time. >=20 > Is there a parameter that can be adjusted to allow view access during = indexing? >=20 > Thanks, > Matthias --Apple-Mail-46--576126011 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii It = should be limited to views within a single design doc.  There's = some storage efficiency I think for placing all your views into a single = design doc, but all views in a design doc get rebuilt when you modify = any of the views.

If your views aren't dependent upon = each other, and you aren't concerned about disk utilization, you could = create each view in a separate design doc to bypass the rebuild. =  Other 'hackery' that I understand is possible is to push the = modified view to a copy of the design doc with a different _id and begin = the index process (query the last document in the view). Then once the = indexing is complete, update the original design doc to match the = alternate and CouchDb should assume using the view that was = 'backgrounded'. This will keep all views accessible until you swap in = the changes after the modified view is = indexed.

- Jim

Senior Software = Engineer
Center for Software = Engineering
SRI = International

On May 31, 2012, at 4:40 PM, Matthias Eck wrote:

Hello,

One of my databases in CouchDB has = close to 1,000,000 documents with a
size of about 200GB.
If I = change one of the views, the indexing takes about 2 hours.

This = time would be not a problem, but I am wondering why I cannot
access = the other views, that were previously generated during the
indexing = time.
Currently none of the views is available, taking my web = application
down during this time.

Is there a parameter that = can be adjusted to allow view access during = indexing?

Thanks,
Matthias

= --Apple-Mail-46--576126011-- --Apple-Mail-47--576125974 Content-Disposition: attachment; filename="smime.p7s" Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJ/zCCBO0w ggRWoAMCAQICEBX3i1OyIZLyYjv7fwx/UYkwDQYJKoZIhvcNAQEFBQAwgdgxCzAJBgNVBAYTAlVT MRowGAYDVQQKExFTUkkgSW50ZXJuYXRpb25hbDEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1c3QgTmV0 d29yazE7MDkGA1UECxMyVGVybXMgb2YgdXNlIGF0IGh0dHBzOi8vd3d3LnZlcmlzaWduLmNvbS9y cGEgKGMpMDIxMDAuBgNVBAsTJ0NsYXNzIDIgT25TaXRlIEluZGl2aWR1YWwgU3Vic2NyaWJlciBD QTEdMBsGA1UEAxMUU1JJIEludGVybmF0aW9uYWwgQ0EwHhcNMTIwMTAzMDAwMDAwWhcNMTMwMTAy MjM1OTU5WjCBwjEaMBgGA1UEChQRU1JJIEludGVybmF0aW9uYWwxKDAmBgNVBAsUH0luZm9ybWF0 aW9uIFRlY2hub2xvZ3kgU2VydmljZXMxRjBEBgNVBAsTPXd3dy52ZXJpc2lnbi5jb20vcmVwb3Np dG9yeS9DUFMgSW5jb3JwLiBieSBSZWYuLExJQUIuTFREKGMpOTkxEjAQBgNVBAMTCUphbWVzIEts bzEeMBwGCSqGSIb3DQEJARYPamltLmtsb0BzcmkuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A MIIBCgKCAQEA2utx7vCP7eb6FitPXlP4Oo4fm2Bsx/lz7X7rHvqZRFdNkLtZjmppsofuWMRdrIMj xCW0lCQb2mvKwA/VSKvoyd4MBSIDYT/jVMz7OeCzNk0VhGKRwqXBlkvlirqhKOo4O24RU6C33c5p il3TDla/YwVbkFmKqGWNKnSddhUKpRVfQW3xJfbzjALWyx0OpLpxLmns6wrnKr6aYMWHOXZmCQ7J jwLWosKJgjlhLJOI+ZSK0JcrK7u2I9pIfYeVjJari4tPBbmoFV8S8vDFxWYryqvQuul7UVHO8VDC dP4jraUzOXZUIhzqmejClwmDsvvuNGsXpW+FaZJ7MwX8j3C5uQIDAQABo4IBRjCCAUIwCQYDVR0T BAIwADCBrAYDVR0gBIGkMIGhMIGeBgtghkgBhvhFAQcXAjCBjjAoBggrBgEFBQcCARYcaHR0cHM6 Ly93d3cudmVyaXNpZ24uY29tL0NQUzBiBggrBgEFBQcCAjBWMBUWDlZlcmlTaWduLCBJbmMuMAMC AQEaPVZlcmlTaWduJ3MgQ1BTIGluY29ycC4gYnkgcmVmZXJlbmNlIGxpYWIuIGx0ZC4gKGMpOTcg VmVyaVNpZ24wCwYDVR0PBAQDAgWgMBEGCWCGSAGG+EIBAQQEAwIHgDBmBgNVHR8EXzBdMFugWaBX hlVodHRwOi8vb25zaXRlY3JsLnZlcmlzaWduLmNvbS9TUklJbnRlcm5hdGlvbmFsSW5mb3JtYXRp b25UZWNobm9sb2d5U2VydmljZXMvTGF0ZXN0Q1JMMA0GCSqGSIb3DQEBBQUAA4GBAI7wVCjyQVMr YkTs+2zjKpjh9Oamq0rcbwyPAHQKJtz23JO0s/cVjsukw+lHvxaMSu8oCnsTa0NOc1a/n7PEoI7n e4j5XH3L6tUsEnNc+t237NoBrJP66my/2FSDpWkLGJ4sxioNEPonl0I0IuE8DiCP1JAdP8vJsXrE 2a5p2y8/MIIFCjCCBHOgAwIBAgIQdRD9LNvKRXBSboyDbAKnbDANBgkqhkiG9w0BAQUFADCBwTEL MAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMTwwOgYDVQQLEzNDbGFzcyAyIFB1 YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5IC0gRzIxOjA4BgNVBAsTMShjKSAx OTk4IFZlcmlTaWduLCBJbmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxHzAdBgNVBAsTFlZl cmlTaWduIFRydXN0IE5ldHdvcmswHhcNMDIwOTIzMDAwMDAwWhcNMTIwOTIyMjM1OTU5WjCB2DEL MAkGA1UEBhMCVVMxGjAYBgNVBAoTEVNSSSBJbnRlcm5hdGlvbmFsMR8wHQYDVQQLExZWZXJpU2ln biBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2UgYXQgaHR0cHM6Ly93d3cudmVy aXNpZ24uY29tL3JwYSAoYykwMjEwMC4GA1UECxMnQ2xhc3MgMiBPblNpdGUgSW5kaXZpZHVhbCBT dWJzY3JpYmVyIENBMR0wGwYDVQQDExRTUkkgSW50ZXJuYXRpb25hbCBDQTCBnzANBgkqhkiG9w0B AQEFAAOBjQAwgYkCgYEAzvnUwmuZmBSSAFVb0qoC0hhUL1a6f+AIHw5UpxW5oRTjsDtUzsCa+6Yg GvKUlisrnI/tPZFrupvHVNQjRj05fhHiABFinwlnCA7J80x3gZlBMwHrgoKYribJ1GTVmc1R0FmA B4KYzBeZjJZiNpqLEsEb0ORdzJYb2/UZazjL/fkCAwEAAaOCAegwggHkMBIGA1UdEwEB/wQIMAYB Af8CAQAwRAYDVR0gBD0wOzA5BgtghkgBhvhFAQcXAjAqMCgGCCsGAQUFBwIBFhxodHRwczovL3d3 dy52ZXJpc2lnbi5jb20vcnBhMDQGA1UdHwQtMCswKaAnoCWGI2h0dHA6Ly9jcmwudmVyaXNpZ24u Y29tL3BjYTItZzIuY3JsMAsGA1UdDwQEAwIBBjARBglghkgBhvhCAQEEBAMCAQYwKAYDVR0RBCEw H6QdMBsxGTAXBgNVBAMTEFByaXZhdGVMYWJlbDItODIwHQYDVR0OBBYEFC1OfgnwbUVBEaxx2j87 9iZKf2RkMIHoBgNVHSMEgeAwgd2hgcekgcQwgcExCzAJBgNVBAYTAlVTMRcwFQYDVQQKEw5WZXJp U2lnbiwgSW5jLjE8MDoGA1UECxMzQ2xhc3MgMiBQdWJsaWMgUHJpbWFyeSBDZXJ0aWZpY2F0aW9u IEF1dGhvcml0eSAtIEcyMTowOAYDVQQLEzEoYykgMTk5OCBWZXJpU2lnbiwgSW5jLiAtIEZvciBh dXRob3JpemVkIHVzZSBvbmx5MR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrghEAuS9g zIifoXpGCbhbcGyKrzANBgkqhkiG9w0BAQUFAAOBgQAowFJw4GZ/4dbI1ncxPAvPGrV/aIB5Z8mZ e9tmn/CH+OcKSVI02h/Q5qbUD+P2hWMW3hBaQeCUG/YMWDgUXXEQKSeZYVGLpGdxkSAzV8VOQLIG JX3/1Lo4oo067Z8qZ0NLf6IH2SzZDEcDuFHGuc5Z0OM3Cghvwo6OX1oO37MiszGCBHswggR3AgEB MIHtMIHYMQswCQYDVQQGEwJVUzEaMBgGA1UEChMRU1JJIEludGVybmF0aW9uYWwxHzAdBgNVBAsT FlZlcmlTaWduIFRydXN0IE5ldHdvcmsxOzA5BgNVBAsTMlRlcm1zIG9mIHVzZSBhdCBodHRwczov L3d3dy52ZXJpc2lnbi5jb20vcnBhIChjKTAyMTAwLgYDVQQLEydDbGFzcyAyIE9uU2l0ZSBJbmRp dmlkdWFsIFN1YnNjcmliZXIgQ0ExHTAbBgNVBAMTFFNSSSBJbnRlcm5hdGlvbmFsIENBAhAV94tT siGS8mI7+38Mf1GJMAkGBSsOAwIaBQCgggJiMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJ KoZIhvcNAQkFMQ8XDTEyMDUzMTIzNTIwN1owIwYJKoZIhvcNAQkEMRYEFF15xwMUQeE+WzheTR9R ttZGn5sHMIH+BgkrBgEEAYI3EAQxgfAwge0wgdgxCzAJBgNVBAYTAlVTMRowGAYDVQQKExFTUkkg SW50ZXJuYXRpb25hbDEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1c3QgTmV0d29yazE7MDkGA1UECxMy VGVybXMgb2YgdXNlIGF0IGh0dHBzOi8vd3d3LnZlcmlzaWduLmNvbS9ycGEgKGMpMDIxMDAuBgNV BAsTJ0NsYXNzIDIgT25TaXRlIEluZGl2aWR1YWwgU3Vic2NyaWJlciBDQTEdMBsGA1UEAxMUU1JJ IEludGVybmF0aW9uYWwgQ0ECEBX3i1OyIZLyYjv7fwx/UYkwggEABgsqhkiG9w0BCRACCzGB8KCB 7TCB2DELMAkGA1UEBhMCVVMxGjAYBgNVBAoTEVNSSSBJbnRlcm5hdGlvbmFsMR8wHQYDVQQLExZW ZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2UgYXQgaHR0cHM6Ly93 d3cudmVyaXNpZ24uY29tL3JwYSAoYykwMjEwMC4GA1UECxMnQ2xhc3MgMiBPblNpdGUgSW5kaXZp ZHVhbCBTdWJzY3JpYmVyIENBMR0wGwYDVQQDExRTUkkgSW50ZXJuYXRpb25hbCBDQQIQFfeLU7Ih kvJiO/t/DH9RiTANBgkqhkiG9w0BAQEFAASCAQB//noJarMF2KCWQKYAnCuvd2+r/mLJ/hwWoizR 5luNQMT7z8+jwMl8lCQPTjgJv4yi0ocueoCXf7NEP42DUV8+C7JBZzBDC8OUkrO1p7sHdbNfU9Lc CWHbQy7UaKdzmLfwid1EQPr8j3jC/J/KLvdfBoMlVHZuNjQjWaG1vFuHbDYtCGYPl494YWQrvfB2 czMeo2k4b2/AeRnrZj4I8/x854l1/NTw8V/UR9aIplyPseylmw+BcEsdB7H3gdZ+2W64xDfhIc0t lQ2m1t5kimFZJiLhZvxweuu3Ve3+zx/QamXvrvrNyl5lJaXNn/22qlcagu1819EMlncCnzpsHEe8 AAAAAAAA --Apple-Mail-47--576125974--