From user-return-19675-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Sun Jan 29 22:21:29 2012 Return-Path: X-Original-To: apmail-couchdb-user-archive@www.apache.org Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AE5339DC9 for ; Sun, 29 Jan 2012 22:21:29 +0000 (UTC) Received: (qmail 15521 invoked by uid 500); 29 Jan 2012 22:21:28 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 15493 invoked by uid 500); 29 Jan 2012 22:21:27 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 15481 invoked by uid 99); 29 Jan 2012 22:21:26 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 29 Jan 2012 22:21:26 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [128.18.84.114] (HELO mailgate-internal4.sri.com) (128.18.84.114) by apache.org (qpsmtpd/0.29) with SMTP; Sun, 29 Jan 2012 22:21:20 +0000 Received: from brightmail-internal2.sri.com (128.18.84.122) by mailgate-internal4.sri.com with SMTP; 29 Jan 2012 22:20:59 -0000 X-AuditID: 8012547a-b7bf0ae000000cc7-63-4f25c64bbb15 Received: from mars.esd.sri.com (mars.esd.sri.com [128.18.26.200]) by brightmail-internal2.sri.com (Symantec Brightmail Gateway) with SMTP id 48.1B.03271.B46C52F4; Sun, 29 Jan 2012 14:20:59 -0800 (PST) MIME-version: 1.0 Received: from [192.168.1.14] (adsl-75-15-136-75.dsl.snlo01.sbcglobal.net [75.15.136.75]) by mars.esd.sri.com (Sun Java(tm) System Messaging Server 6.3-8.05 (built Sep 1 2009; 64bit)) with ESMTPSA id <0LYK00CIAZEXN000@mars.esd.sri.com> for user@couchdb.apache.org; Sun, 29 Jan 2012 14:20:58 -0800 (PST) From: Jim Klo Content-type: multipart/signed; boundary=Apple-Mail-104--471376083; protocol="application/pkcs7-signature"; micalg=sha1 Subject: Re: Are CouchDB filters limited to the _changes feed? Date: Sun, 29 Jan 2012 14:20:58 -0800 In-reply-to: To: user@couchdb.apache.org References: Message-id: <70DE84CA-BF4C-4D4B-9DD9-DE7C681728B6@sri.com> X-Mailer: Apple Mail (2.1084) X-Brightmail-Tracker: AAAAAA== --Apple-Mail-104--471376083 Content-Type: multipart/alternative; boundary=Apple-Mail-103--471376137 --Apple-Mail-103--471376137 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 I can see filters applied to views and replication, since lists are = essentially a filter with formatting. I understood the proposal being = the other way around using view (or map) for replication filtering. Chained filtering would be nice - albeit, I'm not sure how efficient it = would be since the worst possible case you'd be O =3D n*m where n is the = number of filters, m is the number of docs. Jim Klo Senior Software Engineer Center for Software Engineering SRI International On Jan 29, 2012, at 2:07 PM, Alexander Shorin wrote: > IMHO, that's nice idea to apply filter or set of filters to view = result. >=20 > Some examples: > Your database contains set of articles. Your view looks like: > function(doc){ > if (doc.type =3D=3D 'article') > emit(doc.title, doc) > } > Fine. Now you'd like to get articles that have been created by user > group managers? No problem, just create new view with user group > check. Wanted to emit only those who had any comments? Get another > one. And so on, and so on. Instead of this, you may just apply set of > filters to view result and customize output it on fly! Like this: > = /_design/articles/_view/by_title?filters=3D[only_managers,has_comments,has= _positive_rating] > If you're ready to pay some performance hit for this, you've got 4 > views by cost(disk-space) of single one. >=20 > Now let's take a look a little closer... > Common filter function takes two arguments: doc to filter and request > information: > function(doc, req){ > ... > } > and should return boolean to mark has doc passed or not. >=20 > View internals could surely say what document could be processed in > map mode. If you apply reduce function this information is lost for > oblivious reasons. So this idea could live for only map views if we > operating with documents. This means, that this filters should be > applied after map function execution and before reduce one. Since the > fact that reduced view result calculated for each request and doesn't > stored in view index file(am I wrong?) this feature is quite possible > to be real, theoretically(: >=20 > P.S. There is nothing about replications due to views are already > could be used as filters. Creating mix of views/filters I've found a > little crazy. >=20 > -- > ,,,^..^,,, >=20 >=20 >=20 > On Mon, Jan 30, 2012 at 1:39 AM, Jim Klo wrote: >> I could see it as a bad idea in that views usually have a different = collation from the changes feed. Views also can emit more than 1 = key/value per doc, which I'm not sure how that works with filters. I = suppose it's possible, but you'd still have to follow the changes = sequence, but when applying the map function, you replicate only the id = emitted once. >>=20 >> - Jim >>=20 >> Sent from my iPad >>=20 >> On Jan 29, 2012, at 11:35 AM, Dave Cottlehuber = wrote: >>=20 >>> On 29 January 2012 18:13, Benoit Chesneau = wrote: >>>> On Sun, Jan 29, 2012 at 8:54 AM, Daniel Gonzalez = wrote: >>>>=20 >>>>>=20 >>>>> - Am I doing something wrong in the curl calls? >>>>> - Is it at all possible to use views together with filters, or are >>>>> filters limited to the _changes feed? I have seen no examples of >>>>> filters except related to _changes >>>>>=20 >>>>=20 >>>>=20 >>>> filters are only expected for _changes. >>>>=20 >>>> - beno=EEt >>>=20 >>> Although it might be interesting to factor out filters (in changes = and >>> replication) and to manage them the same way as views, including = being >>> able to use views as replication targets, and avoid needing to >>> reprocess the filter each time. >>>=20 >>> I'm sure that when I think about this after my wine, I'll come up = with >>> some good reasons why thats not a great idea. >>>=20 >>> A+ >>> Dave --Apple-Mail-103--471376137 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1 I can = see filters applied to views and replication, since lists are = essentially a filter with formatting. I understood the proposal being = the other way around using view (or map) for replication = filtering.

Chained filtering would be nice - albeit, = I'm not sure how efficient it would be since the worst possible case = you'd be O =3D n*m where n is the number of filters, m is the number of = docs.

Jim Klo
Senior Software Engineer
Center for Software Engineering
SRI = International




On Jan 29, 2012, at 2:07 PM, Alexander Shorin = wrote:

IMHO, that's nice idea to apply filter or set of = filters to view result.

Some examples:
Your database contains = set of articles. Your view looks like:
function(doc){
 if = (doc.type =3D=3D 'article')
   emit(doc.title, = doc)
}
Fine. Now you'd like to get articles that have been created = by user
group managers? No problem, just create new view with user = group
check. Wanted to emit only those who had any comments? Get = another
one. And so on, and so on. Instead of this, you may just = apply set of
filters to view result and customize output it on fly! = Like = this:
/_design/articles/_view/by_title?filters=3D[only_managers,has_com= ments,has_positive_rating]
If you're ready to pay some performance = hit for this, you've got 4
views by cost(disk-space) of single = one.

Now let's take a look a little closer...
Common filter = function takes two arguments: doc to filter and = request
information:
function(doc, req){
 ...
}
and = should return boolean to mark has doc passed or not.

View = internals could surely say what document could be processed in
map = mode. If you apply reduce function this information is lost = for
oblivious reasons. So this idea could live for only map views if = we
operating with documents.  This means, that this filters = should be
applied after map function execution and before reduce one. = Since the
fact that reduced view result calculated for each request = and doesn't
stored in view index file(am I wrong?) this feature is = quite possible
to be real, theoretically(:

P.S. There is = nothing about replications due to views are already
could be used as = filters. Creating mix of views/filters I've found a
little = crazy.

--
,,,^..^,,,



On Mon, Jan 30, 2012 at = 1:39 AM, Jim Klo <jim.klo@sri.com> = wrote:
I could see it as a bad idea in that = views usually have a different collation from the changes feed. Views = also can emit more than 1 key/value per doc, which I'm not sure how that = works with filters.  I suppose it's possible, but you'd still have = to follow the changes sequence, but when applying the map function, you = replicate only the id emitted once.

- = Jim

Sent from my iPad

On Jan 29, = 2012, at 11:35 AM, Dave Cottlehuber <dave@muse.net.nz> = wrote:

On 29 January 2012 18:13, Benoit Chesneau <bchesneau@gmail.com> = wrote:
On Sun, Jan 29, 2012 at 8:54 AM, = Daniel Gonzalez <gonvaled@gonvaled.com> = wrote:


- Am I doing something wrong in = the curl = calls?
- Is it at all possible to use = views together with filters, or = are
filters limited to the _changes = feed? I have seen no examples = of
filters except related to = _changes



filters = are only expected for = _changes.

- = beno=EEt

Although it might be interesting = to factor out filters (in changes = and
replication) and to manage them the same way as views, = including being
able to use views as replication = targets, and avoid needing to
reprocess the filter each = time.

I'm sure that when I think about = this after my wine, I'll come up = with
some good reasons why thats not a great = idea.

A+
Dave
<= br>
= --Apple-Mail-103--471376137-- --Apple-Mail-104--471376083 Content-Disposition: attachment; filename=smime.p7s Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJ/zCCBO0w ggRWoAMCAQICEBX3i1OyIZLyYjv7fwx/UYkwDQYJKoZIhvcNAQEFBQAwgdgxCzAJBgNVBAYTAlVT MRowGAYDVQQKExFTUkkgSW50ZXJuYXRpb25hbDEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1c3QgTmV0 d29yazE7MDkGA1UECxMyVGVybXMgb2YgdXNlIGF0IGh0dHBzOi8vd3d3LnZlcmlzaWduLmNvbS9y cGEgKGMpMDIxMDAuBgNVBAsTJ0NsYXNzIDIgT25TaXRlIEluZGl2aWR1YWwgU3Vic2NyaWJlciBD QTEdMBsGA1UEAxMUU1JJIEludGVybmF0aW9uYWwgQ0EwHhcNMTIwMTAzMDAwMDAwWhcNMTMwMTAy MjM1OTU5WjCBwjEaMBgGA1UEChQRU1JJIEludGVybmF0aW9uYWwxKDAmBgNVBAsUH0luZm9ybWF0 aW9uIFRlY2hub2xvZ3kgU2VydmljZXMxRjBEBgNVBAsTPXd3dy52ZXJpc2lnbi5jb20vcmVwb3Np dG9yeS9DUFMgSW5jb3JwLiBieSBSZWYuLExJQUIuTFREKGMpOTkxEjAQBgNVBAMTCUphbWVzIEts bzEeMBwGCSqGSIb3DQEJARYPamltLmtsb0BzcmkuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A MIIBCgKCAQEA2utx7vCP7eb6FitPXlP4Oo4fm2Bsx/lz7X7rHvqZRFdNkLtZjmppsofuWMRdrIMj xCW0lCQb2mvKwA/VSKvoyd4MBSIDYT/jVMz7OeCzNk0VhGKRwqXBlkvlirqhKOo4O24RU6C33c5p il3TDla/YwVbkFmKqGWNKnSddhUKpRVfQW3xJfbzjALWyx0OpLpxLmns6wrnKr6aYMWHOXZmCQ7J jwLWosKJgjlhLJOI+ZSK0JcrK7u2I9pIfYeVjJari4tPBbmoFV8S8vDFxWYryqvQuul7UVHO8VDC dP4jraUzOXZUIhzqmejClwmDsvvuNGsXpW+FaZJ7MwX8j3C5uQIDAQABo4IBRjCCAUIwCQYDVR0T BAIwADCBrAYDVR0gBIGkMIGhMIGeBgtghkgBhvhFAQcXAjCBjjAoBggrBgEFBQcCARYcaHR0cHM6 Ly93d3cudmVyaXNpZ24uY29tL0NQUzBiBggrBgEFBQcCAjBWMBUWDlZlcmlTaWduLCBJbmMuMAMC AQEaPVZlcmlTaWduJ3MgQ1BTIGluY29ycC4gYnkgcmVmZXJlbmNlIGxpYWIuIGx0ZC4gKGMpOTcg VmVyaVNpZ24wCwYDVR0PBAQDAgWgMBEGCWCGSAGG+EIBAQQEAwIHgDBmBgNVHR8EXzBdMFugWaBX hlVodHRwOi8vb25zaXRlY3JsLnZlcmlzaWduLmNvbS9TUklJbnRlcm5hdGlvbmFsSW5mb3JtYXRp b25UZWNobm9sb2d5U2VydmljZXMvTGF0ZXN0Q1JMMA0GCSqGSIb3DQEBBQUAA4GBAI7wVCjyQVMr YkTs+2zjKpjh9Oamq0rcbwyPAHQKJtz23JO0s/cVjsukw+lHvxaMSu8oCnsTa0NOc1a/n7PEoI7n e4j5XH3L6tUsEnNc+t237NoBrJP66my/2FSDpWkLGJ4sxioNEPonl0I0IuE8DiCP1JAdP8vJsXrE 2a5p2y8/MIIFCjCCBHOgAwIBAgIQdRD9LNvKRXBSboyDbAKnbDANBgkqhkiG9w0BAQUFADCBwTEL MAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMTwwOgYDVQQLEzNDbGFzcyAyIFB1 YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5IC0gRzIxOjA4BgNVBAsTMShjKSAx OTk4IFZlcmlTaWduLCBJbmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxHzAdBgNVBAsTFlZl cmlTaWduIFRydXN0IE5ldHdvcmswHhcNMDIwOTIzMDAwMDAwWhcNMTIwOTIyMjM1OTU5WjCB2DEL MAkGA1UEBhMCVVMxGjAYBgNVBAoTEVNSSSBJbnRlcm5hdGlvbmFsMR8wHQYDVQQLExZWZXJpU2ln biBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2UgYXQgaHR0cHM6Ly93d3cudmVy aXNpZ24uY29tL3JwYSAoYykwMjEwMC4GA1UECxMnQ2xhc3MgMiBPblNpdGUgSW5kaXZpZHVhbCBT dWJzY3JpYmVyIENBMR0wGwYDVQQDExRTUkkgSW50ZXJuYXRpb25hbCBDQTCBnzANBgkqhkiG9w0B AQEFAAOBjQAwgYkCgYEAzvnUwmuZmBSSAFVb0qoC0hhUL1a6f+AIHw5UpxW5oRTjsDtUzsCa+6Yg GvKUlisrnI/tPZFrupvHVNQjRj05fhHiABFinwlnCA7J80x3gZlBMwHrgoKYribJ1GTVmc1R0FmA B4KYzBeZjJZiNpqLEsEb0ORdzJYb2/UZazjL/fkCAwEAAaOCAegwggHkMBIGA1UdEwEB/wQIMAYB Af8CAQAwRAYDVR0gBD0wOzA5BgtghkgBhvhFAQcXAjAqMCgGCCsGAQUFBwIBFhxodHRwczovL3d3 dy52ZXJpc2lnbi5jb20vcnBhMDQGA1UdHwQtMCswKaAnoCWGI2h0dHA6Ly9jcmwudmVyaXNpZ24u Y29tL3BjYTItZzIuY3JsMAsGA1UdDwQEAwIBBjARBglghkgBhvhCAQEEBAMCAQYwKAYDVR0RBCEw H6QdMBsxGTAXBgNVBAMTEFByaXZhdGVMYWJlbDItODIwHQYDVR0OBBYEFC1OfgnwbUVBEaxx2j87 9iZKf2RkMIHoBgNVHSMEgeAwgd2hgcekgcQwgcExCzAJBgNVBAYTAlVTMRcwFQYDVQQKEw5WZXJp U2lnbiwgSW5jLjE8MDoGA1UECxMzQ2xhc3MgMiBQdWJsaWMgUHJpbWFyeSBDZXJ0aWZpY2F0aW9u IEF1dGhvcml0eSAtIEcyMTowOAYDVQQLEzEoYykgMTk5OCBWZXJpU2lnbiwgSW5jLiAtIEZvciBh dXRob3JpemVkIHVzZSBvbmx5MR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrghEAuS9g zIifoXpGCbhbcGyKrzANBgkqhkiG9w0BAQUFAAOBgQAowFJw4GZ/4dbI1ncxPAvPGrV/aIB5Z8mZ e9tmn/CH+OcKSVI02h/Q5qbUD+P2hWMW3hBaQeCUG/YMWDgUXXEQKSeZYVGLpGdxkSAzV8VOQLIG JX3/1Lo4oo067Z8qZ0NLf6IH2SzZDEcDuFHGuc5Z0OM3Cghvwo6OX1oO37MiszGCBHswggR3AgEB MIHtMIHYMQswCQYDVQQGEwJVUzEaMBgGA1UEChMRU1JJIEludGVybmF0aW9uYWwxHzAdBgNVBAsT FlZlcmlTaWduIFRydXN0IE5ldHdvcmsxOzA5BgNVBAsTMlRlcm1zIG9mIHVzZSBhdCBodHRwczov L3d3dy52ZXJpc2lnbi5jb20vcnBhIChjKTAyMTAwLgYDVQQLEydDbGFzcyAyIE9uU2l0ZSBJbmRp dmlkdWFsIFN1YnNjcmliZXIgQ0ExHTAbBgNVBAMTFFNSSSBJbnRlcm5hdGlvbmFsIENBAhAV94tT siGS8mI7+38Mf1GJMAkGBSsOAwIaBQCgggJiMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJ KoZIhvcNAQkFMQ8XDTEyMDEyOTIyMjA1OFowIwYJKoZIhvcNAQkEMRYEFEfU0WgdECMNrIIYPrba gN0eWtpyMIH+BgkrBgEEAYI3EAQxgfAwge0wgdgxCzAJBgNVBAYTAlVTMRowGAYDVQQKExFTUkkg SW50ZXJuYXRpb25hbDEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1c3QgTmV0d29yazE7MDkGA1UECxMy VGVybXMgb2YgdXNlIGF0IGh0dHBzOi8vd3d3LnZlcmlzaWduLmNvbS9ycGEgKGMpMDIxMDAuBgNV BAsTJ0NsYXNzIDIgT25TaXRlIEluZGl2aWR1YWwgU3Vic2NyaWJlciBDQTEdMBsGA1UEAxMUU1JJ IEludGVybmF0aW9uYWwgQ0ECEBX3i1OyIZLyYjv7fwx/UYkwggEABgsqhkiG9w0BCRACCzGB8KCB 7TCB2DELMAkGA1UEBhMCVVMxGjAYBgNVBAoTEVNSSSBJbnRlcm5hdGlvbmFsMR8wHQYDVQQLExZW ZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2UgYXQgaHR0cHM6Ly93 d3cudmVyaXNpZ24uY29tL3JwYSAoYykwMjEwMC4GA1UECxMnQ2xhc3MgMiBPblNpdGUgSW5kaXZp ZHVhbCBTdWJzY3JpYmVyIENBMR0wGwYDVQQDExRTUkkgSW50ZXJuYXRpb25hbCBDQQIQFfeLU7Ih kvJiO/t/DH9RiTANBgkqhkiG9w0BAQEFAASCAQDRov1ROBvxxR7loUzlIAFj39xeVcowDs7HgNDH 3s2wmbx7z/hWB4tnUTCKRJH3wrlSBchHBc+ZYLUsjLDhlnlvVdSprWMADcNwAqG/lsNXsJq+Ztzd MbmonfiN9gcmmr5ayFE4t7KKMFEoDyj73QbR9UdkaYCObwU5h60d/xFf/nMl3B8b07lBxah//BvY wHkawRhHHno+8htj2QMoB9fKo/gR3z3NIBdBi532XwjzkyU/BAPqBP3O0VpccAxyQpchVfvvTJ3E hZbN5IMKRAh4mFuCrcz70t3JHieeqH8VsBGOKRcmUrqoR6Fqy2lfgrkwkc4Xx2Ir3smuFCBmKllI AAAAAAAA --Apple-Mail-104--471376083--