couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zachary Zolton <zachary.zol...@gmail.com>
Subject Re: M/R/M, again
Date Fri, 29 Jan 2010 03:50:42 GMT
@Markus,

Not sure I quite understood the semantics of the proposed "merge"
concept... Perhaps another example usage?

In the meantime, however, I'd try to solve your initial problem by
joining related docs, from a view, via a _list function:

function(head, req){
  var row;
  var customer;
  var products = [];
  while(row = getRow()) {
    if (row.doc.type == 'customer') {
      customer = row.doc;
    } else if (row.doc.type == 'product') {
      products.push(row.doc);
    }
  }
  customer.products = products;
  return toJSON(customer);
}

I'm not sure this particular hack is explicitly mentioned in the wiki.


Cheers,

Zach

On Thu, Jan 28, 2010 at 8:11 PM, Nicholas Orr <nicholas.orr@zxgen.net> wrote:
> I could probably use m/r/m - however each time I thought I needed this I
> solved my perceived problem by rethinking what I wanted to achieve in the
> first place and then ended up just merging docs together with references to
> one other doc with extra data copied into my new all in one doc.
>
> I think a detailed use case needs to be supplied with actual solved problems
> and why doing it any other way is not feasible or practical in the
> demonstrated use case.
>
> Even with the products, user, purchases use case I have a similar use case.
> Generating "quotes" (finance industry) based on who did it, what product,
> what rates, what lender etc.
> When I first did this I did it in mysql, everything works great, ideal RDBMS
> logic applied. Everything referencing everything else, not much data copied.
> Then I moved to couchdb because at the end of the day I want a document that
> is a quote and re-arranged what I was doing to suit CouchDB's way of getting
> things done. As such I copy a lot of data from my RDBMS
> lender/product/rate structure into my final CouchDB quote doc.
>
> Using map/reduce I can query all these docs anyway I want.
> As they are also quotes they should not be changing because I updated a rate
> in the RDBMS rate structure. (I need to move these rate plans into some sort
> of CouchDB doc as well and the latest doc as defined by some sort of date is
> used, the rates change weekly and are documents in either pdf, excel or
> word)
>
> Everything in CouchDB as far as I can tell is developed due to an actual
> need, without it x is not possible and implementing the feature provides a
> net win for the project. I've actually managed to solve all my issues with
> the existing feature set and ultimately figuring out if CouchDB is the best
> fit for the thing I'm trying to do. CouchDB isn't a do everything tool.
>
> A detailed need with everything mapped out would go a long way, "making my
> life easier" probably doesn't qualify - if you want easy just go back to
> paying M$ or Oracle for a RDBMS product (and associated hardware costs).
> Everything is doable and relatively simple with those tools. I've found its
> easier for me to read and learn more than to figure out where I get a chunk
> of coin to pay M$ (hardware/hosting costs)...
>
> Cheers,
>
> Nick
>
> 2010/1/29 Markus Jelsma <markus@buyways.nl>
>
>> Nicholas, thanks for your reply!
>>
>>
>> I completely agree with you that CouchDB users need to leave their RDBMS
>> knowledge aside, i tell the same to my co-workers but somehow i tend to
>> feel a need for keeping separate documents. Perhaps i'm biased by some
>> wiki pages that also tend to do the same. You'll recognize the case with
>> products and purchases where each purchase is a separate document.
>> Although the use-case is not entirely the same, you can get much
>> information on purchases in that scenario with simple map/reduce
>> functions.
>>
>> But, if map/reduce/merge can simplify my life and others, why shouldn't it
>> be implemented? I'm quite sure many of us will be very grateful if such a
>> feature was to be available, don't you agree?
>>
>> If it is not being implemented or even suggested as a real proposal - as
>> the list of proposals on the wiki tells us - i would have no other choice
>> than following the principle you speak about. For my use case, it would
>> not be such a bad choice because these kind of updates are only initiated
>> by that same user only, so there is no worry about the
>> multiple-document-transaction paradigm which does not exist in CouchDB.
>>
>> Nevertheless, i, and possibly the core developers, sure would like to know
>> if there are more users that'd like to utilize a feature like me.
>>
>> Perhaps i just need to be either patient or bend the use-case even more
>> towards a pure document oriented database which will not allow us to merge
>> documents in a view.
>>
>>
>> Cheers,
>>
>> Nicholas Orr said:
>> > The other thing you could do is merge the two docs together.
>> > I've done this recently and boy did it make life simple.
>> >
>> > CouchDB requires a different approach than RDBMS.
>> > Applying RDBMS concepts to CouchDB will leave you wanting RDBMS
>> > features. Consider stepping back for a moment and forget RDBMS way of
>> > doing things and solve what you're trying to solve with CouchDB
>> > (Document) concepts.
>> >
>> > From what I can tell you want a username/email to map to a primaryid
>> > depending on the app in question.
>> >
>> > Would this work?
>> >
>> > {
>> > "_id": "a@a.com",
>> > "username": "adam",
>> > "type": "profile"
>> > "apps":
>> >
>> [{"applicationId":"app2","primaryId":18},{"applicationId":"app1","primaryId":17}]
>> > }
>> >
>> > This usually has far reaching implications when converting from RDBMS to
>> > Document Store.
>> > I've made choices like, well if applicationId changes then I'm going to
>> > have to update somehow, there isn't a cascade or atomic op in CouchDB,
>> > so how about applicationId is not allowed to change only allowed to
>> > create new ones, that works :) (then if I want to delete I have a
>> > active/disabled flag)
>> >
>> > Food for thought
>> >
>> > Nick
>> >
>> > 2010/1/29 Markus Jelsma <markus@buyways.nl>
>> >
>> >> Hi Jan,
>> >>
>> >>
>> >> Thanks for your reply, but i'm afraid that i have provided a lousy
>> >> explanation
>> >> of the case i run in to. Let me explain with actual examples for i
>> >> believe Damien's examples do not fit my use case.
>> >>
>> >> I have a tiny database with two types of documents, profile and
>> >> profileApplication. The profile type has an ID which is the user's
>> >> e-mail address and a simply username field, nothing more (see below
>> >> for anatomy of both document types).
>> >>
>> >> {
>> >>   "_id": "markus@buyways.nl",
>> >>   "_rev": "1-5f7718ae8a627f4cf5b93b63420b7e1f",
>> >>   "type": "profile",
>> >>   "username": "markus17"
>> >> }
>> >> {
>> >>   "_id": "1d2d9db700029557666e5d260b2ea038",
>> >>   "_rev": "2-279daa538abc5cbb4b1524d29ce4ab53",
>> >>   "type": "profileApplication",
>> >>   "applicationId": "app2",
>> >>   "profileId": "markus@buyways.nl",
>> >>   "primaryId": 18
>> >> }
>> >>
>> >> The documents with profileApplication type are related to both an
>> >> application
>> >> (which i have omitted for now) and a profile. In RDBMS terms its
>> >> purpose would
>> >> be a common link table.
>> >>
>> >> The purpose for this relation is that a single profile can have a
>> >> different primaryId for different applications. My profile
>> >> (markus@buyways.nl) would have primaryId=18 for app2 and primaryId=17
>> >> for app1 etc.
>> >>
>> >> The goal would be to retrieve both my profile document _and_ the
>> >> primaryId that goes with my profile for app1 or app2, ideally the
>> >> query would be key=["markus@buyways.nl", "app1"], but this is
>> >> currently not possible.
>> >>
>> >> There are two things i can do now:
>> >> 1) retrieve the profile first and then fetch the primaryId for the
>> >> application
>> >> i need, but this takes two requests and manually merging of the
>> >> profile data
>> >> and primaryId;
>> >>
>> >>
>> >> http request 1:
>> >> http://localhost:5984/test/markus@buyways.nl
>> >>
>> >> output:
>> >> {"_id":"markus@buyways.nl
>> >>
>> ","_rev":"1-5f7718ae8a627f4cf5b93b63420b7e1f","type":"profile","username":"markus17"}
>> >>
>> >> http request 2:
>> >>
>> >>
>> http://localhost:5984/test/_design/profiles/_view/getPrimaryByEmailAndApplication?key=[%22markus@buyways.nl%22
>> >> ,
>> >> %20%22app1%22]
>> >>
>> >> output:
>> >> {"total_rows":4,"offset":2,"rows":[
>> >> {"id":"f51b92f4a59de0e28641375637a73050","key":["markus@buyways.nl
>> >> ","app1"],"value":17}
>> >> ]}
>> >>
>> >> It's clear that i need to merge the value of the second request with
>> >> the document received by the first.
>> >>
>> >>
>> >> 2) fetch the profile and all related primaryIds in one go, this is one
>> >> single
>> >> requests but i also get primaryId's for apps that i don't need so this
>> >> fetches
>> >> more data and also needs clientside merging after i filtered out the
>> >> app i need.
>> >>
>> >>
>> >> http request 1:
>> >>
>> >>
>> http://localhost:5984/test/_design/profiles/_view/getProfileApplications?startkey=[%22markus@buyways.nl%22]&endkey=[%22markus@buyways.nl%22
>> >> ,
>> >> %20%22zzz%22]
>> >>
>> >> output:
>> >> {"total_rows":6,"offset":3,"rows":[
>> >> {"id":"markus@buyways.nl","key":["markus@buyways.nl",1],"value":null},
>> >> {"id":"f51b92f4a59de0e28641375637a73050","key":["markus@buyways.nl
>> >> ","app1",2],"value":17},
>> >> {"id":"1d2d9db700029557666e5d260b2ea038","key":["markus@buyways.nl
>> >> ","app2",2],"value":18}
>> >> ]}
>> >>
>> >> It's clear that i need to filter my profile document and the
>> >> profileApplication document for the app i want (app1). The bad thing
>> >> here is
>> >> that i do not get my profile document in the value (although i can
>> >> emit it but
>> >> that's), if i include_docs i'll also get a lot of extra data on the
>> >> documents
>> >> i don't need, here it's just one document but i can be many.
>> >>
>> >>
>> >>
>> >> Both techniques work and have their pros and cons. But do you agree
>> >> that it would be much more convenient if we could simply construct
>> >> views that carry merged or combined documents using
>> >> key=["markus@buyways.nl","app1"].
>> >>
>> >> Am i correct to assume i cannot achieve the goal stated above without
>> >> either
>> >> Chris' technique or merging of documents in one single view?
>> >>
>> >> Please forgive me if i somehow didn't understand Damien's example but
>> >> i believe that deals with arithmetic instead of merging complex data
>> >> structures.
>> >> I also didn't (yet?) feel that the new 0.11 linked documents feature
>> >> will help
>> >> me out here. Also, i wish to keep this data in separate documents,
>> >> keeping an
>> >> array within the profile document isn't really the best approach i
>> >> think.
>> >>
>> >>
>> >>
>> >> Cheers,
>> >>
>> >>
>> >> >See http://damienkatz.net/2008/02/incremental_map.html and
>> >> >http://damienkatz.net/2008/02/incremental_map_1.html and the
>> >> >comments on both.
>> >>
>> >> Markus Jelsma - Technisch Architect - Buyways BV
>> >> http://www.linkedin.com/in/markus17
>> >> 050-8536620 / 06-50258350
>>
>>
>>
>>
>

Mime
View raw message