incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christoph Zrenner (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (COUCHDB-249) Treat output rows of views as documents for other views to build upon
Date Sat, 04 Feb 2012 13:53:53 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200441#comment-13200441
] 

Christoph Zrenner commented on COUCHDB-249:
-------------------------------------------

Hi all, I've been using CouchDB for about 6 months now (an amazing technology) and we implemented
a "prediction engine" using a bayesian classifier inside a view. The parameters for the machine
learning algorithm is in a commonjs array and every 24 hours we modify the parameters based
on updates from the manual verified training data added that day (like in spam/ham classification),
so then the view is rebuilt. It seems like this might be a use case that would be well served
with chained views, so I'm adding it here to this ticket:

So after the predictions are calculated, there are then a bunch of further analytics calculation
that happen based on the predicted data, but the predictions are only temporary (the prediction
parameters are updated every 24h). Right now, I'm doing the same bayes prediction calculation
in each of the analytics views (inside a commonjs function "BayesPredictor") but this means
that the same calculations are performed for each analytics view.

Chaining the temporary and computationally intense prediction calculation output with a subsequent
analysis view "feels" like it might be a good solution for this problem. I'm reluctant to
write the predictions into another database as in the cloudant solution, if I was to go that
route, then I think I may as well just keep updating my source documents by going through
all_documents and updating the predictions on each document every 24h.

Would very much appreciate any views on whether this is a "valid" use-case for native chained
views and any advice on how I might implement this! Thanks.
                
> Treat output rows of views as documents for other views to build upon
> ---------------------------------------------------------------------
>
>                 Key: COUCHDB-249
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-249
>             Project: CouchDB
>          Issue Type: New Feature
>          Components: Database Core, Infrastructure, JavaScript View Server
>            Reporter: Joey Lawrance
>         Attachments: couch_view_updaer.erl.patch.txt, couch_view_updater.erl
>
>
> Unless I manually copy the JSON rows of a view into a new document, I am unable to create
new views that are computed from existing views. That is, it seems as if views are second
class citizens compared to first-class documents.
> Suppose I wanted to find the spread between the cheapest suppliers and the most expensive
suppliers of each fruit. I know it's possible to use one map/reduce to compute such a view,
but I'd like to be able to re-use my existing "cheapest" and "costliest" views. That is, I'd
like to use the document output of these views as input into another view.
> I started with the simple fruit store example in the CouchDB book. I developed a simple
view called "cheapest" with the following map and reduce functions (the "costliest" view is
the same as "cheapest" but except the reduce function's comparison is the other way around):
> function(doc) {
>     var store, price, key;
>     if (doc.item && doc.prices) {
>         for (store in doc.prices) {
>             price = doc.prices[store];
>             key = doc.item;
>             emit(key, {store:store, price:price});
>         }
>     }
> }
> function(item,store) {
>   var m = store[0];
>   for (i in store) {
>     if (m.price > store[i].price) m = store[i];
>   }
>   return m;
> }
> The output is as follows:
> {"rows":[
> {"key":"apple","value":{"store":"Apples Express","price":0.79}},
> {"key":"banana","value":{"store":"Price Max","price":079}},
> {"key":"orange","value":{"store":"Citrus Circus","price":1.09}}
> ]}
> I'd like to develop a new view whose input is the output of the view above, but as far
as I can tell, views only operate on documents, not the output of existing views. Am I missing
something?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message