couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: CommonJS in map and reduce
Date Sat, 25 Sep 2010 21:56:47 GMT
On Sat, Sep 25, 2010 at 5:20 PM, Chris Anderson <jchris@apache.org> wrote:
> On Mon, Sep 20, 2010 at 8:53 AM, Mikeal Rogers <mikeal.rogers@gmail.com> wrote:
>> Great writeup.
>>
>> The only thing I think it's lacking is how the require namespaces will
>> differ.
>>
>> So, refresher, require('foo') means import a module that is a top level
>> attribute in the design doc named "foo". Basically the commonjs module
>> namespace is scoped to the ddoc root. This will stay the same for require
>> outside of map/reduce.
>>
>
> So the way I've implemented it so far is that to require code that it
> stored in ddoc.views.lib.foo.bar, you'd write:
>
> var barModule = require("views/lib/foo/bar');
>
> This is less surprising, I think, than having the call look like:
>
> var barModule = require("foo/bar');
>
> Even though the second one is shorter, I think it is cleaner to have
> it be consistent what require() means regardless of if you are in a
> view or a show/list function.
>
> One other issue: currently I haven't yet implemented commonjs for
> reduce functions. The reason I have not is that reduce functions are
> not batched, so library code would have to be sent and eval'd every
> single time the reduce function is run. On top of this, 99% of the
> time the reduce function won't make use of the library code, so it
> will just be dead weight. (I have a hard time thinking of when a
> reduce function will need to require library code).
>
> So until someone rewrites the reduce runner to use batching, I will
> don't think it is a good idea to support commonjs in reduce functions.
>

http://github.com/davisp/couchdb/commit/3cec76a37d3e652e539c8b416e1e5a930364eb43

View updates are done and use the same os process that the map docs
functions use. I didn't do this for view reads as it would change more
behavior that people would run into with lots of concurrent reduce
view readers.

That patch set is quite big and I need to go back through and
rearrange the patches to do all the refactoring I did before I import
emonk, but the theory is all there even if I have to manually pull the
various parts apart. The last major bit that I need to finish on that
is the new app server code for OS processes. The idea is sound, I just
didn't get to it before CouchCamp.

HTH,
Paul Davis

> The code is on this branch. I will merge it to trunk in a couple of
> days once people have had a chance to look at it.
>
> http://github.com/jchris/couchdb/tree/commonjs_views
>
> Chris
>
>> In map/reduce all we'll do is re-scope the root namespace for require to the
>> ddoc "views" attribute.
>>
>> All the "relative" imports, like require('./module'), will work the same but
>> we'll probably have some safe guard from doing '../../../' to get out of the
>> views sandbox.
>>
>> -Mikeal
>>
>> On Mon, Sep 20, 2010 at 4:55 PM, Chris Anderson <jchris@apache.org> wrote:
>>
>>> Devs,
>>>
>>> Mikeal and I were talking and we came up with a sane way to do
>>> CommonJS in map and reduce.
>>>
>>> Refresher: we don't have CommonJS require there now because the
>>> current CommonJS implementation is scoped to the whole design doc, and
>>> giving views access to load code from anywhere in the design doc would
>>> mean we'd have to blow away your view index anytime you changed
>>> anything. Having to rebuild views from scratch just because you
>>> changed some CSS or a show function isn't fun,so we avoided the issue
>>> by keeping CommonJS require out of map and reduce altogether.
>>>
>>> The solution we came up with is to allow CommonJS inside map and
>>> reduce funs, but only of libraries that are stored inside the views
>>> part of the design doc.
>>>
>>> So you could continue to access CommonJS code in design_doc.foo, from
>>> your list functions etc, but we'd add the ability to require CommonJS
>>> modules within map and reduce, but only from design_doc.views.lib
>>>
>>> There's no worry here about namespace collisions, as Couch just plucks
>>> views.*.map and views.*.reduce out of the design doc. So you could
>>> have a view called "lib" if you wanted, and still have CommonJS stored
>>> in views.lib.sha1 and views.lib.stemmer if you wanted.
>>>
>>> We could allow CommonJS modules to be stored anywhere in ddoc.views,
>>> but I think it will simplify the implementation to enforce that they
>>> be stored in views.lib -- if people think that is too restrictive,
>>> please speak up now, otherwise I'll start to implement this.
>>>
>>> Chris
>>>
>>>
>>> --
>>> Chris Anderson
>>> http://jchrisa.net
>>> http://couch.io
>>>
>>
>
>
>
> --
> Chris Anderson
> http://jchrisa.net
> http://couch.io
>

Mime
View raw message