Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 41698 invoked from network); 9 Apr 2010 18:47:56 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 9 Apr 2010 18:47:56 -0000 Received: (qmail 58034 invoked by uid 500); 9 Apr 2010 18:47:54 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 57970 invoked by uid 500); 9 Apr 2010 18:47:54 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 57962 invoked by uid 99); 9 Apr 2010 18:47:54 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 09 Apr 2010 18:47:54 +0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of andrew.melo@gmail.com designates 209.85.222.183 as permitted sender) Received: from [209.85.222.183] (HELO mail-pz0-f183.google.com) (209.85.222.183) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 09 Apr 2010 18:47:47 +0000 Received: by pzk13 with SMTP id 13so2895610pzk.13 for ; Fri, 09 Apr 2010 11:47:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:received:message-id:subject:from:to:content-type :content-transfer-encoding; bh=YQI+3CmLzpsvY+/aDQRIPZ5iXwEZTsysRLUAbIDNkYM=; b=SJmfrIEMifr1C248cYrLjDf3Msnk2jvFeZDY5pRtR+wnVM6DFkjC2tLy4IRVkY2nnK 4ZzcpOhMtnOYzdyC0PIAqYmQoSKibAovoD8Laa+JwJY0kqRMAr6ox9MD+x9oX3P6UT+Z ba8tIDULYgzSoPBcXJ8Z26l7BVeRLRlYwYPL4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=GzxoYy4Qpnc5K17LokHj7kQ9K/pY4I2aAPuueOEwL5EQEW5qh8MSC6VJWxXDITjQD9 6iKihQNf7BY78omorufKkBgPp4qPKHk0tgPq/5glyU4PBCxvw0Nwzl0hIs0IURekQZ3G VXm125/7eei7F6p+y9edYLbD0PGvGLOdNPgHI= MIME-Version: 1.0 Received: by 10.231.172.133 with HTTP; Fri, 9 Apr 2010 11:47:26 -0700 (PDT) In-Reply-To: References: <641CEC1C-ECF8-479C-8603-622D1FA8E8D3@gmail.com> Date: Fri, 9 Apr 2010 13:47:26 -0500 Received: by 10.140.88.33 with SMTP id l33mr889055rvb.4.1270838846161; Fri, 09 Apr 2010 11:47:26 -0700 (PDT) Message-ID: Subject: Re: About denormalization and keep consistent From: Andrew Melo To: user@couchdb.apache.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org On Fri, Apr 9, 2010 at 1:44 PM, faust 1111 wrote: > Thanks Chris. > >> Your idea to do it in the app server, as the author changes their master= record, is troubling because it can lead to race conditions. The changes m= ethod, where a name-update is an asynchronous process, is more robust, beca= use you can know for sure that *eventually* the author's name will be chang= ed everywhere it appears. >> > > You told about more robust method, you mean run backend process listen > _changes feed for Authors, > and when changes come try update all contents related to author? > If i get you right, i don't understand you >>because you can know for sure that *eventually* the author's name will be= changed everywhere it appears. > What you mean? > > >> My method is to have a view of docs by author, and then query that view = for the old author's name, updating any docs that appear. This way if new w= rites come in with the old name (due to there being out of date replicas of= the master record lingering, for instance) they will be eventually updated= as well. You could have a time-to-live of something like 5 minutes (or lon= ger if your system is giant) for the process which is running the query for= docs-that-say-Joe-but-should-say-Joseph and updating them. >> > > Probably i don't get you: > I track _changes feed, when change come you suggest query that view > for the old author's name, but how i know old name, author doc all > ready with newest name. When you spawn the background process, you'll have to pass it that information somehow. > > Sorry for getting your time, to answer for stupid questions. > > > 2010/4/9 J Chris Anderson : >> >> On Apr 8, 2010, at 11:55 PM, faust 1111 wrote: >> >>> Yes i understand that listen _changes is better to get round race condi= tions. >>> >>> Cannot get your suggesting about >>> how i can track that all contents related to author was updated not 5 >>> of 50 but all. >>> >> >> I think your question is valid. The answer is also simple. There is no w= ay to transactionally ensure that the author's name is updated everywhere i= t appears. >> >> Your idea to do it in the app server, as the author changes their master= record, is troubling because it can lead to race conditions. The changes m= ethod, where a name-update is an asynchronous process, is more robust, beca= use you can know for sure that *eventually* the author's name will be chang= ed everywhere it appears. >> >> It is probably best to make this clear through the UI with a message lik= e: "Your name has been changed in the master record. It could take a few mi= nutes for the change to appear throughout the site." >> >> In actuality, this is probably no different than in a relational databas= e (as in a relational database, you'd probably have a caching layer that ta= kes a few minutes to expire anyway.) >> >>> Thats ok. >>> I don't understand if listen feed _chenges, feed give me info only >>> about id & rev of changed doc, how i can get that author name is >>> changed? >>> >> >> My method is to have a view of docs by author, and then query that view = for the old author's name, updating any docs that appear. This way if new w= rites come in with the old name (due to there being out of date replicas of= the master record lingering, for instance) they will be eventually updated= as well. You could have a time-to-live of something like 5 minutes (or lon= ger if your system is giant) for the process which is running the query for= docs-that-say-Joe-but-should-say-Joseph and updating them. >> >> _changes is just a convenient way to trigger that view query (so that yo= u aren't polling the view when nothing has happened in the database.) With = filtered changes, you can even be sure that you are only polling the view w= hen there will be something relevant to see. However, all this _changes stu= ff is really just an optimization over brute force polling the view once ev= ery N seconds, so you can add it later, when your app is big enough that lo= ad starts to matter. >> >> Chris >> >>> >>> 2010/4/9 Nicholas Orr : >>>> i don't think you are getting what the above people are suggesting... >>>> >>>> Go read up on the _changes API :) >>>> >>>> The basics are, every single change in the database is pushed into thi= s >>>> feed. All race conditions that are caused by your ruby way (via the fi= lter) >>>> are averted :) >>>> >>>> Nick >>>> >>>> On Fri, Apr 9, 2010 at 4:34 AM, faust 1111 wrote: >>>> >>>>> i means >>>>> when i do >>>>> Content.by_author(self).each {|content| >>>>> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0content.author_name =3D self.name; >>>>> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0content.save(bulk=3Dtrue) >>>>> =C2=A0 =C2=A0 =C2=A0 } >>>>> >>>>> i don't sure that all contents will updated may be only 5 and then >>>>> process crushed. >>>>> >>>>> 2010/4/8 Andrew Melo : >>>>>> On Thu, Apr 8, 2010 at 12:53 PM, faust 1111 wro= te: >>>>>>> What difference? >>>>>>> if do >>>>>>> Author >>>>>>> =C2=A0after_save >>>>>>> =C2=A0 =C2=A0 if name_changed? >>>>>>> =C2=A0 =C2=A0 =C2=A0 =C2=A0Content.by_author(self).each {|content| >>>>>>> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 content.author_name =3D self.nam= e; >>>>>>> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 content.save(bulk=3Dtrue) >>>>>>> =C2=A0 =C2=A0 =C2=A0 } >>>>>>> >>>>>>> or i start backend process to track Author _changes. >>>>>>> >>>>>>> This code not guarantee that all contents will updated. >>>>>> >>>>>> I don't get your question. You asked how to make sure that you could >>>>>> change a number of documents consistently, we suggested that you wat= ch >>>>>> _changes to catch any silly race conditions. Then, you told us you >>>>>> didn't need to use _changes, but you were worried that things would = be >>>>>> inconsistent. >>>>>> >>>>>> Even with your code above, you get a race condition (if I understand >>>>>> your ruby right, I don't know ruby much at all). Something could >>>>>> happen between when you check to see if a document needs to be chang= ed >>>>>> and the actual change occurs. Then, you're gonna get a conflict and >>>>>> have to write up the logic to handle that intelligently. >>>>>> >>>>>> best, >>>>>> Andrew >>>>>> >>>>>> >>>>>>> >>>>>>> 2010/4/8 Andrew Melo : >>>>>>>> On Thu, Apr 8, 2010 at 12:29 PM, faust 1111 >>>>> wrote: >>>>>>>>> I can catch changes in my app before save author, may be backend >>>>>>>>> process is surplus in my case. >>>>>>>>> i need consistent, when i update author name i must know that all >>>>>>>>> contents with author was updated success. >>>>>>>> >>>>>>>> Then their suggestion of watching _changes works for you. Start >>>>>>>> watching _changes. Make all your changes to the documents' authors= . >>>>>>>> Any changes that come through on _changes, make sure they have the >>>>>>>> proper author. Keep watching _changes until you're sure that nobod= y >>>>>>>> has stale data they're waiting submit. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> 2010/4/8 Zachary Zolton : >>>>>>>>>> I suggest you check out the _changes API: >>>>>>>>>> http://books.couchdb.org/relax/reference/change-notifications >>>>>>>>>> >>>>>>>>>> Basically, if you have doc types A & B, where B maintains a deno= rmed >>>>>>>>>> bit of A, then you can watch the _changes feed in a backend proc= ess. >>>>>>>>>> When an A gets updated, hit a view of all B's related to that >>>>>>>>>> particular A, and update the dernomed data. >>>>>>>>>> >>>>>>>>>> On Thu, Apr 8, 2010 at 10:20 AM, faust 1111 >>>>> wrote: >>>>>>>>>>> Hi guy's >>>>>>>>>>> I return back to my problem with denormalization. >>>>>>>>>>> >>>>>>>>>>> is it possible to keep consistent when apply denormalization? >>>>>>>>>>> For example >>>>>>>>>>> Content >>>>>>>>>>> =C2=A0 have author (we store author name and id in Content) >>>>>>>>>>> >>>>>>>>>>> When author name changed(that's happens not frequently) >>>>>>>>>>> i need find all content belong to this author and update author= name >>>>>>>>>>> but what if this operation not finished (not all docs was updat= ed) >>>>>>>>>>> >>>>>>>>>>> What i can do in this case? >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> -- >>>>>>>> Andrew Melo >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> -- >>>>>> Andrew Melo >>>>>> >>>>> >>>> >> >> > --=20 -- Andrew Melo