incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <matthew-m.r...@ubs.com>
Subject RE: Lookup views
Date Mon, 16 Jan 2012 08:04:12 GMT
>-----Original Message-----
>From: Dave Cottlehuber [mailto:dave@muse.net.nz] 
>Sent: 13 January 2012 20:47
>To: user@couchdb.apache.org
>Subject: Re: Lookup views
>
>On 13 January 2012 12:56, Nils Breunese <N.Breunese@vpro.nl> wrote:
>> Matt <matthew-m.rose@ubs.com> wrote:
>>
>>> Suppose we buy an sell an awful lot of fruit, mostly apples and oranges but sometimes
bananas, pears and anything else that shoppers want. We take a feed (from another system)
of every trade and load them all into couch as we go along, then define a view to aggregate
them up by various different aspects of the sale. We can control the grouping level to produce
many different reports out the same view and it works very well for us. Some of the attributes
on the trades documents are not on the original feed so we enrich the data before PUTting
them in couch. This is also fine.
>>>
>>> Unfortunately, sometimes the enrichment changes. For example each trade has a
fruit field and we have a mapping from fruit to fruit type. Today apples, oranges and pears
are classified as round fruit whereas bananas are long fruit so we can put a fruit type field
on the document as we load it in but tomorrow that classification might change and pears might
become lumpy fruit instead of round fruit. We could go back and update every pear trade in
the database but we'd rather not have fruit type on the document at all and maintain a separate
mapping "table".
>>>
>>> We thought that perhaps the map function could reference another database that
was keyed on fruit and had documents containing the fruit type but don't seem to be able to
perform this lookup in the mapping. We could get the document itself on the result but not
extract just one field to augment the original documents. We also considered generating the
view's design dynamically out of a mapping table and replicating the mapping into explicit
logic in the javascript but this seemed like an abuse.
>>>
>>> In short, how would you implement a pre-aggregation lookup table?
>>
>> I think I'd just put the lookup table in the map function. Another option is indeed
to update the documents and change the fruit type. Which of these two yields best performance
probably depends on how large your lookup table would be and how many documents you'd have
to update. Map functions need to be completely standalone in order to ensure things keep working
when you start doing things like distributing your data, so calls to external stuff is a no-go.
>>
>> Nils.
>> ----------------------------------------------------------------------
>> --
>>  VPRO   www.vpro.nl
>> ----------------------------------------------------------------------
>> --
>
>Its worth noting that changing the map functions in either way will require the view to
be rebuilt before it can be used which might be an expensive or timeconsuming operation. This
is a common workaround for reducing the operational impact of doing that http://wiki.apache.org/couchdb/How_to_deploy_view_changes_in_a_live_environment
>
>I'd not update the docs directly as this will increase the DB size, requiring more frequent
compaction. In comparison a view generation would likely be less expensive.
>
>A+
>Dave
>
>

Thanks everyone. We were happy to live with the view rebuild time but that's a nice neat workaround
you have there so we'll probably adopt it.

Regards,

Matt
Visit our website at http://www.ubs.com 

This message contains confidential information and is intended only 
for the individual named. If you are not the named addressee you 
should not disseminate, distribute or copy this e-mail. Please 
notify the sender immediately by e-mail if you have received this 
e-mail by mistake and delete this e-mail from your system. 

E-mails are not encrypted and cannot be guaranteed to be secure or 
error-free as information could be intercepted, corrupted, lost, 
destroyed, arrive late or incomplete, or contain viruses. The sender 
therefore does not accept liability for any errors or omissions in the 
contents of this message which arise as a result of e-mail transmission. 
If verification is required please request a hard-copy version. This 
message is provided for informational purposes and should not be 
construed as a solicitation or offer to buy or sell any securities 
or related financial instruments. 

UBS Limited is a company limited by shares incorporated in the United 
Kingdom registered in England and Wales with number 2035362. 
Registered office: 1 Finsbury Avenue, London EC2M 2PP.  UBS Limited 
is authorised and regulated by the Financial Services Authority. 

UBS AG is a public company incorporated with limited liability in 
Switzerland domiciled in the Canton of Basel-City and the Canton of 
Zurich respectively registered at the Commercial Registry offices in 
those Cantons with Identification No: CH-270.3.004.646-4 and having 
respective head offices at Aeschenvorstadt 1, 4051 Basel and 
Bahnhofstrasse 45, 8001 Zurich, Switzerland.  Registered in the 
United Kingdom as a foreign company with No: FC021146 and having a 
UK Establishment registered at Companies House, Cardiff, with No:  
BR 004507.  The principal office of UK Establishment: 1 Finsbury Avenue, 
London EC2M 2PP.  In the United Kingdom, UBS AG is authorised and 
regulated by the Financial Services Authority.

UBS reserves the right to retain all messages. Messages are protected 
and accessed only in legally justified cases. 

Mime
View raw message